Job Description We are seeking an experienced Senior Data Pipeline Engineer to design, build, and operate secure, reliable, and cost-efficient data pipelines supporting our Android connected and infotainment experiences. You will lead Python-based ETL/ELT development, Airflow orchestration, and data platform operations across cloud environments, working closely with Android, backend, and product teams.
Responsibilities - Build and maintain Airflow 2.x DAGs (TaskFlow, dynamic DAGs, deferrable operators, providers).
- Develop robust Python ETL/ELT ingesting from APIs, storage, message buses, and databases.
- Operate Airflow on Azure/Kubernetes; support blue/green and canary DAG releases.
- Implement data quality testing, monitoring, observability, and lineage.
- Design scalable batch and streaming pipelines with strong schema management.
- Manage SQL and blob data stores (partitioning, clustering, retention).