We are looking for a motivated Associate Consultant – Databricks with strong experience in data engineering and cloud platforms. The role involves building and supporting scalable data pipelines, working with large datasets, and contributing to analytics and data platform initiatives using Databricks.
Key Responsibilities
- Develop and maintain ETL/ELT pipelines using Databricks (PySpark / Spark SQL).
- Work with structured and semi-structured data (JSON, Parquet, CSV, Delta).
- Assist in designing and implementing data models in the Lakehouse architecture.
- Support data ingestion from various sources (APIs, databases, cloud storage).
- Optimize Spark jobs for performance and cost efficiency.
- Implement data quality checks and monitoring mechanisms.
- Collaborate with analytics, BI, and data science teams to deliver datasets.
- Participate in code reviews and follow engineering best practices.
Requirements
Required Skills
- Bachelor’s degree in Computer Science, Information Technology, or a related field.
- 2–4 years of experience in data engineering or related roles.
- Hands-on experience with Databricks and Apache Spark (PySpark preferred).
- Strong SQL and Python skills.
- Experience with at least one cloud platform (AWS / Azure / GCP).
- Understanding of data warehousing concepts and ETL processes.
- Basic knowledge of Delta Lake and Lakehouse architecture.
- Familiarity with Git and CI/CD fundamentals.
Good to Have
- Experience with Airflow / ADF / Glue.
- Exposure to streaming data (Kafka, Spark Streaming).
- Databricks certification Associate/Professionals.
Signs you may be a great fit
- Impact: Play a pivotal role in shaping a rapidly growing venture studio.
- Culture: Thrive in a collaborative, innovative environment that values creativity and ownership.
- Growth: Access professional development opportunities and mentorship.
- Benefits: Competitive salary, health/wellness packages, and flexible work options.
