Summary
What you’ll impact
The Kafka Platform Engineer will design, install, secure, and operate enterprise Apache Kafka clusters and develop event-driven applications and integrations. The role also involves automating deployments, monitoring performance, and providing operational tooling and mentorship to development teams.
Responsibilities
What you'll do
- Install, configure, upgrade, patch, and operate Apache Kafka clusters using KRaft mode across Dev, QA, UAT, Production, and disaster-recovery environments.
- Configure brokers, controllers, topics, partitions, replication factors, retention policies, quotas, consumer groups, and MirrorMaker 2 replication.
- Implement TLS/mTLS, SASL/SCRAM, Kafka ACLs, least-privilege access, certificate lifecycle management, secrets integration, audit logging, and security hardening.
- Coordinate DNS, routing, firewall, load balancer, proxy, port, and network-segmentation requirements; troubleshoot connectivity, latency, certificate, and authentication issues.
- Develop and support Java/Spring Boot Kafka producers and consumers, Kafka Streams applications, Kafka Connect integrations, serialization/deserialization, retry handling, dead-letter topics, idempotency, and error recovery.
- Implement schema governance using Avro, JSON Schema, or Protobuf and support compatible schema evolution through an approved schema registry.
- Build deployment and configuration automation using Ansible, Terraform, shell scripting, Git, and CI/CD pipelines.
- Configure dashboards, logging, metrics, and alerts for broker health, under-replicated partitions, consumer lag, replication lag, storage, throughput, availability, and security events.
- Perform functional, performance, failover, recovery, and security validation; diagnose production incidents and complete root-cause analysis.
- Produce design documents, configuration standards, build guides, operating procedures, runbooks, test evidence, and knowledge-transfer materials; mentor developers and review code and configurations.
Requirements
What you’ll bring
- Experience: 15+years Must
- Visa: Only US Citizens/ H1B’S
- Strong hands-on experience with Apache Kafka architecture, KRaft, producers/consumers, consumer groups, partitioning, replication, offsets, delivery semantics, and cluster troubleshooting.
- Proficiency in Java, Spring Boot, REST APIs, distributed systems, and event-driven design; working knowledge of Kafka Streams and Kafka Connect.
- Practical experience with Linux/RHEL administration, TCP/IP, DNS, routing, firewalls, PKI, TLS certificates, keystores/truststores, authentication, and authorization.
- Experience with monitoring and logging platforms such as Prometheus, Grafana, Splunk, or equivalent enterprise tools.
- Experience with Git, CI/CD, infrastructure/configuration automation, performance tuning, DR testing, and production support.
- Strong analytical, documentation, collaboration, communication, and mentoring skills.