We are seeking an experienced Databricks Consultant to design, develop, and optimize enterprise-grade data platforms. The candidate will play a key role in solution implementation, performance tuning, stakeholder communication, and mentoring junior engineers.
Key Responsibilities
- Design scalable and secure data architectures using Databricks Lakehouse.
- Develop advanced ETL/ELT pipelines and optimize Spark workloads.
- Lead data migration projects from traditional warehouses to Databricks.
- Implement Delta Lake best practices (partitioning, Z-ordering, optimization).
- Design data models (star schema, snowflake schema), Data Vaults.
- Ensure governance, security, and access control (Unity Catalog, RBAC).
- Collaborate with business stakeholders to translate requirements into technical solutions.
- Implement CI/CD pipelines and DevOps best practices for data engineering.
- Mentor Associate Consultants and conduct code reviews.
Requirements
Required Skills
- Bachelor’s degree in Computer Science, Information Technology, or a related field.
- 5–7 years of experience in data engineering.
- Strong hands-on expertise in Databricks and Apache Spark.
- Deep understanding of performance tuning and Spark optimization techniques.
- Strong Experience in at least one major cloud (AWS / Azure / GCP).
- Strong SQL, PySpark, Python and Data Modelling.
- Experience with orchestration tools (Airflow, ADF, etc.).
- Knowledge of data governance and security frameworks.
Good to Have
- Experience with real-time data processing.
- Exposure to DLT pipelines implementation in Databricks.
- Databricks Professional certification.
Signs you may be a great fit
- Impact: Play a pivotal role in shaping a rapidly growing venture studio.
- Culture: Thrive in a collaborative, innovative environment that values creativity and ownership.
- Growth: Access professional development opportunities and mentorship.
- Benefits: Competitive salary, health/wellness packages, and flexible work options.
