Databricks Architect

Job Description

Develop and maintain ETL/ELT pipelines using Databricks (PySpark, SQL, Notebooks). Design end‑to‑end Lakehouse architecture including ingestion, transformation, modeling, and consumption. Define and implement Medallion Architecture (Bronze/Silver/Gold) using Delta Lake. Build high-performance ETL/ELT pipelines using PySpark, Spark SQL, Delta Lake. Architect and optimize batch and streaming data pipelines. Perform advanced Spark optimization, cluster sizing, and cost management. Implement CI/CD, version control, and DevOps best practices within Databricks.

Required Skills