Summary
What you’ll impact
The Senior Full-Stack Engineer designs, builds, and maintains a distributed platform for evaluating large language models at scale. They use Python/Django for backend services, React with TypeScript for the frontend, and AWS with CDK for cloud infrastructure and IaC. Responsibilities include creating evaluation pipelines, developing full‑stack features, deploying and monitoring cloud resources, and performing code and architecture reviews. The role works closely with research and product teams, sets engineering best practices, and mentors teammates through informal feedback and formal reviews.
Responsibilities
What you'll do
- Build distributed systems to run evaluations across multiple models, benchmarks, and machines at scale
- Deploy cloud infrastructure using IAC, including deployment pipelines, servers, logging, monitoring, etc.
- Contribute to the internal and external libraries we maintain, including our public model library
- Develop full-stack features for our platform using React/TypeScript on the frontend and Python/Django on the backend
- Perform code and architecture reviews for other members of the team
- Help establish engineering best practices across the organization
- Collaborate closely with the research team to ensure our infrastructure meets their needs
Requirements
What you’ll bring
- Strong engineering fundamentals: You can build and ship quickly with high quality. You should have a track record of building things of significant scope (within jobs, side projects, open source, etc.)
- Python expertise: Significant experience in Python, especially in a professional setting.
- System Design Experience: You should be familiar with common concepts like VMs, containerization, load balancers, databases, etc. and when to use them appropriately.
- Familiarity with LLMs: You should have experience working with LLM APIs, and understand concepts like temperature, tokenization, reasoning, etc.
- Team collaboration: Experience working with development sprints, Git workflows, and pull request reviews.
- Communication: Strong ability to provide and receive feedback effectively. This includes both spoken and written communication (e.g. design docs).
- Comfort with Ambiguity: You will often be the one taking a fuzzy problem and breaking it down into clear and actionable steps.
- Iteration speed: A tenacity to develop and iterate quickly. If you are coming from a large organization, the speed at which we ship might have a learning curve.
- Location: We are an in-person team based in San Francisco. We will support your relocation or transportation as needed.