Sr Engineer, Data

Job Description

Expert proficiency in PySpark or Spark SQL for large-scale data transformation on distributed compute platforms. Hands-on experience with Databricks including Delta Lake, Unity Catalog, and workflow orchestration. Experience with Microsoft Fabric or Azure Synapse Analytics; familiarity with Fabric IQ and OneLake is a plus. Proficiency with pipeline orchestration tools such as Azure Data Factory, Databricks Workflows, or Apache Airflow. Solid SQL skills for data validation, transformation logic, and ad-hoc source system analysis. Experience building and maintaining ingestion pipelines from enterprise operational systems such as ERP, WMS, TMS, or comparable platforms. Strong understanding of data quality frameworks, including implementing checks, alerting on anomalies, and maintaining SLA-compliant pipeline health. Experience with DevOps practices for data pipelines including version control (Git), CI/CD, and automated testing. Ability to operate with moderate independence on complex technical problems, escalating appropriately and maintaining clear technical documentation. Nice to Have: Experience with real-time streaming technologies including Kafka, Azure Event Hub, Delta Live Tables, or Spark Structured Streaming. Familiarity with Oracle, ERP ingestion patterns, or large-scale ERP migration programs. Background in supply chain, logistics, reverse logistics, or procurement data domains. Experience with legacy platform migration and decommission programs. Exposure to T-Mobile's Omni lakehouse platform or MagentaBuilt integrations. Python development skills beyond PySpark, including utility scripting and framework contributions.

Required Skills