Summary
What you’ll impact
The organization is seeking a strong Data Platform Engineer to architect and scale the infrastructure powering the next generation of software-defined electric vehicles. In this role, you will be responsible for the design and optimization of our core data processing and analytics infrastructure, with a specific emphasis on multi-region Databricks orchestration.
Responsibilities
What you'll do
- Infrastructure Management: Design, deploy, and maintain our data platform infrastructure using Terraform and other Infrastructure as Code (IaC) principles.
- Databricks Platform Expertise: Manage and optimize our Databricks platform, ensuring it remains secure, performant, and cost-effective at a global scale.
- System Design & Architecture: Lead the technical design of data systems, considering factors like scalability, reliability, and performance. You'll be expected to make informed decisions that align with our long-term architectural vision.
- Stakeholder Collaboration: Communicate effectively with data producers, data consumers, and business leaders to gather requirements, provide technical guidance, and ensure the data platform meets their technical and legal needs.
- Troubleshooting & Issue Resolution: Proactively identify and resolve issues that may arise with the Databricks platform.
- Staying Ahead of the Curve: Stay up-to-date with the latest advancements in big data technologies, cloud computing, and DevOps practices.
Requirements
What you’ll bring
- Education: Bachelor's or Master's degree in Computer Science, Data Science, or a related field.
- Experience: 3+ years of hands-on experience in a similar role, with a proven track record of building and maintaining large-scale data processing systems.
- Databricks Platform Expertise: Solid understanding of the Databricks platform and core concepts such as Delta, Spark, Unity Catalog, data governance, and workspace administration.
- Terraform and DevOps Background: History of using Terraform to manage large scale systems via Infrastructure as Code, and utilizing other technologies to build and maintain CI/CD pipelines.
- Problem-Solving Skills: Exceptional analytical and problem-solving skills, with the ability to break down complex problems into manageable components.
- Communication & Collaboration: Excellent communication and interpersonal skills, with the ability to collaborate effectively with cross-functional teams around the globe.
- Cloud Computing: Experience working with cloud-based technologies (e.g., AWS, GCP, Azure)
- PySpark Expertise: Background designing scalable data pipelines using the Spark ecosystem (e.g., Spark SQL, Spark Streaming).
- AWS proficiency: Familiarity with core services such as IAM permissions, VPC networking, EC2 compute, and S3 storage.