Featured Job

Member of Technical Staff — Model Optimization and Inference (Experienced)

Seattle, Washington Full-time On-site $250k — $350k per year 11/17/2026 Job ID: 000568
Apply Now
LLM inference optimization KV cache strategies Eviction policies Compression Memory-efficient attention

Summary

What you’ll impact

The Lead Inference Engineer owns end-to-end optimization of LLM, audio, and diffusion model inference to achieve sub-500 ms latency. The role uses Python, PyTorch, CUDA/Triton kernels, and serving frameworks like vLLM, SGLang, and TensorRT-LLM on GPUs. Key tasks include KV-cache policy design, INT8/INT4 quantization (GPTQ, AWQ), custom kernel work, and profiling tools to remove bottlenecks. They collaborate with research and infrastructure teams to ship optimized models and set performance standards across the stack.

Responsibilities

What you'll do

  • Own end-to-end inference optimization across our model stack — LLMs, audio models, and diffusion-based components
  • Implement and tune KV cache strategies for long-context conversations, including eviction policies, compression, and memory-efficient attention
  • Evaluate, deploy, and extend inference serving frameworks (vLLM, SGLang, TensorRT-LLM, etc.) for our specific workloads
  • Profile and benchmark end-to-end latency and throughput; identify and systematically eliminate bottlenecks
  • Build internal tooling that makes optimization work faster and more rigorous — profiling viewers, end-to-end inference test harnesses, and other infrastructure that helps the team move quickly
  • Accelerate diffusion model inference — consistency models, step distillation, caching strategies, and custom kernel optimizations
  • Apply and develop quantization techniques (INT8, INT4, GPTQ, AWQ, and beyond) to reduce memory footprint and increase throughput without meaningfully degrading quality
  • Work closely with research and infrastructure to ensure new models ship with optimized serving from day one

Requirements

What you’ll bring

  • Significant hands-on experience with LLM inference optimization — you’ve shipped work on KV caching, memory layout, attention kernels, or batching strategies in a production or high-traffic research context
  • Proven proficiency with inference serving frameworks — vLLM, SGLang, TensorRT-LLM, or similar — including going well beyond default configurations and adapting them to non-standard workloads
  • Experience optimizing diffusion model inference (latency reduction, step distillation, caching, or kernel-level work)
  • Strong Python and PyTorch skills; comfort reading and writing CUDA or Triton kernels is a significant plus
  • A systematic approach to profiling and optimization — you measure first, then optimize
  • Familiarity with speculative decoding or other inference-time acceleration techniques

Ready to Move Forward?

Apply now and our recruiting team will reach out with next steps, interview guidance, and client insights tailored to this role.