Product Manager - AI Infrastructure

About the job

About the Role We are building a next-generation AI inference platform that virtualizes the full AI stack — from GPU infrastructure and runtime optimization to multi-tenant model serving. We are looking for a highly technical Product Manager who can operate at the intersection of distributed systems, GPU infrastructure, and AI runtime engineering. This role requires deep systems understanding. You will partner directly with platform and runtime engineers to define architecture, platform capabilities, and economic strategy for our AI execution fabric. What You Will Own · Product strategy for AI inference and GPU platform services · Roadmap for: • Multi-tenant inference • GPU orchestration and GPU scheduling • Inference runtime capabilities • Model sharding and dynamic batching • KV cache optimization • Kubernetes-based resource scheduling · Define SLOs for latency (p95/p99), throughput, and availability · Drive GPU utilization improvements and cost-per-token optimization · Shape autoscaling and resource scheduler strategies · Define pricing models (usage-based, token-based, throughput tiers) · Partner with engineering on control plane vs data plane design decisions · Ensure enterprise-grade capabilities (IAM, isolation, audit, observability) What We’re Looking For Required • 6–12+ years of product/ engineering experience in building cloud infrastructure, compute, or platform products • Strong understanding of distributed systems and multi-tenant architectures • Familiarity with GPU platforms (CUDA, MIG, NVLink, NCCL) • Exposure to AI inference runtimes (TensorRT, ONNX Runtime, Triton, vLLM) • Experience with Kubernetes, autoscaling, and resource schedulers • Ability to reason about: • Model sharding • Dynamic batching • KV cache management • GPU memory constraints • Experience defining platform SLOs and capacity planning • Experience with usage-based or infrastructure pricing models Preferred • Experience at a hyperscaler or cloud infrastructure company • Experience with AI inference platforms or ML infrastructure • Exposure to Kubernetes, scheduling systems, or multi-tenant platforms • Experience defining usage-based pricing models • Background in engineering (CS degree or prior software engineering role) What This Role Is Not • Not an AI research role • Not a consumer AI product role • Not an LLM application PM role • Not a purely commercial or marketing product role • Not a AI implementation PM role