Summary
What you’ll impact
The Platform Engineer IV will design, build, and maintain AWS infrastructure for the data platform, including the data lake, AI agent runtime environments, CI/CD pipelines, and graph database systems. The role ensures production environments are stable, scalable, and secure while enabling data science and autonomous AI workloads to operate reliably at scale.
Responsibilities
What you'll do
- Design and manage AWS infrastructure for the IIA Data Lake including S3 storage, Glue data catalog, Athena query engine, and EMR compute clusters.
- Manage cross-account connectivity, VPC networking, security groups, and IAM roles/policies to enable secure data flow between IIA, upstream data providers, and downstream consumers.
- Build and maintain infrastructure for AI agent runtime environments, including compute resources for LangGraph agents deployed via LangSmith Deployments.
- Support deployment and operation of AWS Neptune for the network topology graph (digital twin), including capacity planning, schema design support, and performance tuning.
- Implement and manage infrastructure-as-code (Terraform, CloudFormation) for repeatable, auditable environment provisioning.
- Manage IAM access key rotations, secrets management (AWS Secrets Manager, Delinea), and security compliance for on-premises and cloud integrations (e.g., Splunk Edge Processor).
- Build and maintain CI/CD pipelines for AI agent deployments using GitLab CI/CD, Docker, and Artifactory.
- Manage container lifecycle for agents deployed via LangSmith Deployments, including image builds, versioning, and rollback procedures.
- Automate deployment workflows to enable rapid, reliable promotion of agents from development through production.
- Coordinate with SpecGPT platform team on AI Gateway integration, cross-account deployment, and connectivity requirements.
- Develop and maintain utilitarian application code: scripts, CLIs, small services, and automation utilities (primarily Python) that support data ingestion, deployment, environment provisioning, and operational workflows.
- Write integration code and glue services that connect IIA systems with upstream data providers, downstream consumers, and external platforms.
- Ensure production environment stability through monitoring, alerting, and incident response.
- Maintain SLAs for data pipeline availability and agent uptime.
- Implement production monitoring and alerting for deployed agents (health checks, error rates, latency, resource utilization).
- Coordinate with upstream data teams and platform teams (SpecGPT, Splunk, Public Cloud) on connectivity, firewall requests, and integration requirements.
- Support data engineering team with infrastructure needs for new data source onboarding (storage provisioning, access controls, pipeline compute).
- Perform other duties as required.
Requirements
What you’ll bring
- Ability to read, write, speak and understand English
- Strong communication skills with ability to explain infrastructure decisions to non-infrastructure stakeholders
- Expert-level experience with AWS services: EC2, S3, IAM, VPC, Glue, Athena, EMR, Secrets Manager, CloudWatch
- Strong experience with infrastructure-as-code (Terraform preferred, CloudFormation acceptable)
- Experience managing cross-account AWS architectures, VPC peering, PrivateLink, and transit gateway configurations
- Experience with IAM policy design, least-privilege access patterns, and service account management