Principal Systems Engineer, RELIC Compute Architecture Full-time Mountain View, CA Hybrid 260K - 300K USD a year View job

Mountain View, California Full-time Hybrid 11/17/2026 Job ID: 000564
Apply Now
Systems architecture for AI, HPC or hyperscale compute NVLink/NVSwitch InfiniBand RoCEv2 (PFC, ECN, DCQCN) UALink

Summary

What you’ll impact

The Principal Systems Engineer - AI Compute defines and owns the compute architecture for next‑generation AI inference platforms. Key duties include selecting GPUs, CPUs, NICs, DPUs and fabric technologies (NVLink, NVSwitch, RoCEv2), designing node‑scale and rack‑scale systems, building performance and TCO models, and co‑designing the inference stack with SGLang, vLLM, or TensorRT‑LLM. The engineer collaborates with vendors and cross‑functional teams, authors architecture specifications, and mentors peers on system‑level design.

Responsibilities

What you'll do

  • You will own the compute architecture of next-generation RELIC: what goes in the node, how nodes connect, and how the serving stack maps onto both.
  • Reporting to the CTO, you set the technical direction for every GPU, CPU, NIC, switch and fabric decision we make over the next two hardware generations.
  • This is a systems role, not a component role. You will reason from model and workload characteristics down to the scale-up domain, the scale-out fabric, memory hierarchy and power envelope, then back up to tokens per second per megawatt and cost per million tokens.
  • Define the next-gen RELIC compute platform. Evaluate and select accelerators, host CPUs, NICs and DPUs across NVIDIA (Blackwell Ultra, Rubin), AMD (MI3xx/MI4xx) and emerging inference silicon. Make the call on HGX-class nodes versus rack-scale NVL systems for each site.
  • Architect the scale-up domain. Size NVLink/NVSwitch domains and partitioning (e.g. NVL72-class racks), and track open alternatives such as UALink and Ethernet-based scale-up. Decide what domain size each workload actually needs.
  • Architect the scale-out and front-end fabrics. Own topology (rail-optimized, fat-tree, multi-plane), RoCEv2 congestion control, Ultra Ethernet readiness, optics and transceiver strategy, and storage and KV-cache networks.
  • Co-design the inference stack with the hardware. Map disaggregated prefill/decode, KV-cache transfer and offload, MoE expert parallelism, tensor/pipeline parallelism and speculative decoding onto the platform. Work hands-on with SGLang, vLLM, TensorRT-LLM and NVIDIA Dynamo.
  • Build the models that drive decisions. Own performance, roofline and TCO models covering throughput, latency SLOs, perf/W, $/token and tokens/s/MW, and keep them honest against measured data from our clusters.
  • Own the roadmap and vendor technical relationships. Lead deep-dives with NVIDIA, AMD, OEMs and network vendors; influence early-access programs; write the architecture specs that procurement and site teams build to.
  • Stay close to the metal. Lead bring-up and validation of new platforms and drive root cause on fabric, NIC and node-level issues when they block production.

Requirements

What you’ll bring

  • 10+ years in systems architecture for AI, HPC or hyperscale compute, including at least one platform you took from paper to production at cluster scale.
  • Scale-up vs. scale-out, deeply. You can explain when a workload needs a large NVLink domain versus a well-tuned RDMA fabric, and quantify the cost of getting it wrong. Hands-on with NVLink/NVSwitch, InfiniBand and RoCEv2 (PFC, ECN, DCQCN), and familiar with UALink and Ultra Ethernet.
  • GPU architecture fluency. SM and tensor-core design, HBM capacity and bandwidth, low-precision formats (FP8, FP4/MXFP4), and how each shows up in prefill versus decode.
  • CPU and host architecture. Arm and x86 server cores (e.g. Grace, Neoverse, EPYC, Xeon), NUMA, PCIe Gen5/Gen6, CXL, and the host's role in data movement and KV-cache tiering.
  • Modern inference software. Working knowledge of disaggregated serving, continuous batching, paged attention, KV-cache reuse and transfer, and MoE parallelism, with hands-on time in at least one of SGLang, vLLM, TensorRT-LLM or Dynamo.
  • Quantitative modeling. You build performance and TCO models from first principles and defend them with benchmarks.
  • Embedded in the AI/HPC community. You follow and contribute to the ecosystem: papers, open-source projects, OCP or MLPerf work, conference talks, or deep vendor relationships.
  • Builder's temperament. Comfortable owning ambiguous, cross-disciplinary problems at an early-stage company and getting hands dirty in a cluster when needed.

Ready to Move Forward?

Apply now and our recruiting team will reach out with next steps, interview guidance, and client insights tailored to this role.