Lead innovative AI inference strategies as a Senior Software Engineer. Work directly with customers to optimize LLM serving within Kubernetes and Slurm for groundbreaking performance.
In this pivotal role, you’ll employ your systems expertise, bolstered by over 5 years of experience, to enhance the deployment of AI inference workloads. You'll guide technical partnerships and solve complex problems, while documenting and sharing valuable insights across teams. Collaboration is key to driving effective solutions in both customer-facing and internal environments.
Key Responsibilities:
• Implement end-to-end benchmarking for LLM architectures
• Operate and optimize vLLM on GPU clusters
• Develop comprehensive performance plans
• Share technical documentation and insights
• Foster collaboration with kernel engineering teams
Requirements:
• 5+ years of relevant engineering experience
• Advanced degrees in Computer Science or similar
• Hands-on with Kubernetes...