Summary
What you’ll impact
The Lead Data Engineer designs, builds, and scales the core data infrastructure that powers customer‑facing analytics and AI evaluation pipelines. They work with PostgreSQL, Redis, Typesense, and BullMQ while evolving the system toward change‑data‑capture, event modeling, and a schema‑flexible graph data model. Key responsibilities include improving query performance under row‑level security, implementing zero‑downtime schema migrations, and building reliable ingestion pipelines with strong freshness and audit guarantees. They set technical direction, define abstractions and ownership boundaries, and mentor peers while collaborating closely with product and engineering teams.
Responsibilities
What you'll do
- Scale the analytics engine behind customer-facing dashboards, tackling query performance under row-level security, workload isolation, read architecture, and observability as data volume grows.
- Design the ingestion paths, event models, schemas, and query patterns that move data from transactional writes into search, dashboards, and history—with clear guarantees around freshness, correctness, replay, and failure recovery.
- Evolve our schema-flexible, graph-shaped data model so customer-defined objects, attributes, and relationships remain fast to query as their size and complexity grow.
- Build the foundations for historical reporting and auditability, including attribute versioning, relationship history, and change capture.
- Build reliable data systems for usage metering, pipeline generation, and AI evaluation, where errors have direct customer, product, or financial consequences.
- Decide when our existing architecture remains the right foundation and when new analytical, streaming, or workflow systems earn their added complexity.
- Set the technical direction, abstractions, ownership boundaries, and engineering practices for data systems as the company grows.
- Zero-downtime schema migrations for an 18-collection Typesense search deployment.
- A usage-metering pipeline for consumption billing.
- Historical and audit data modeling.
- Evaluation data infrastructure for our AI agents.
Requirements
What you’ll bring
- 7+ years experience.
- Strong software engineering fundamentals.
- Experience owning production data systems where query plans, replication lag, backfills, data freshness, schema evolution, or data correctness had real user-facing consequences.
- Comfort debugging across multiple layers of the stack.
- Good judgment about when to make a tactical fix and when to invest in a more durable platform or architecture change.
- Product orientation: you care about how data infrastructure decisions affect customers, users, and engineering velocity.
- Clear communication, strong ownership, and a bias toward practical tradeoffs.