Responsibilities
What you'll do
- Design and deploy production AI agents that leverage the latest advances in large language models, reasoning, retrieval, memory, and tool use.
- Architect intelligent systems that combine LLMs, traditional machine learning, structured knowledge, enterprise data, and deterministic software into reliable production workflows.
- Engineer customer intelligence layers, retrieval pipelines, memory systems, and knowledge representations that allow agents to reason over large, heterogeneous enterprise data.
- Develop multi-agent systems that coordinate reasoning, planning, tool execution, and human oversight.
- Translate frontier AI research into production systems by rapidly evaluating new models, prompting techniques, reasoning paradigms, and agent architectures.
- Own the full experimentation lifecycle, from hypothesis generation to production rollout.
- Design rigorous evaluation frameworks using offline benchmarks, online A/B experiments, golden datasets, regression suites, LLM-as-a-Judge, and human evaluation.
- Run controlled experiments and ablation studies to understand the contribution of different models, prompts, retrieval strategies, reasoning techniques, memory systems, and agent architectures.
- Continuously evaluate newly released frontier models and determine where they meaningfully improve quality, latency, reliability, or cost.
- Develop confidence estimation, reflection, and continuous learning systems that improve agents over time using real-world feedback.
- Measure success through business outcomes, not benchmark scores.
- Build production-quality AI systems with a strong emphasis on reliability, observability, latency, safety, and cost.
- Design agent guardrails, fallback strategies, tracing, monitoring, and evaluation pipelines that enable safe deployment in high‑stakes environments.
- Collaborate with infrastructure engineers to deploy AI systems securely within enterprise cloud environments.
- Build human-in-the-loop workflows that effectively combine AI automation with expert oversight.
- Partner directly with enterprise customers to understand their business, data, and operational challenges.
- Translate ambiguous customer problems into production AI architectures.
- Rapidly prototype new ideas, validate them with customers, and evolve successful solutions into scalable production systems.
- Identify reusable patterns that become core capabilities across many enterprise deployments.
- Designing reasoning and agent architectures.
- Building retrieval, memory, and customer intelligence systems.
- Developing predictive models that work alongside LLMs.
- Running experiments and ablation studies.
- Shipping production systems into enterprise environments.
- Measuring business impact through online experimentation.
- Continuously improving agents using real‑world feedback.
Requirements
What you’ll bring
- 4+ years of software engineering, machine learning, or applied AI experience.
- Strong Python programming skills.
- Experience building production AI systems using LLMs.
- Experience with modern AI tooling, including OpenAI, Claude, MCP, agent frameworks, vector databases, or retrieval systems.
- Strong understanding of machine learning fundamentals and modern language models.
- Experience designing or evaluating AI systems using quantitative metrics.
- Excellent communication skills and the ability to work directly with enterprise customers.