Description

Defined.ai is a leading provider of high-quality, ethically sourced data for Artificial Intelligence (AI) and Machine Learning (ML) model training. We host the world's largest AI marketplace and offer end-to-end services to help companies accelerate their AI solutions. Backed by significant funding and recognized globally for our commitment to ethical AI, we operate in a fast-paced, innovative environment with offices in Seattle and Lisbon.

This is a hybrid position in Lisbon, or remote position (outside Lisbon).

What will you do?

Pipeline Orchestration

• Design and maintain end-to-end data workflows using Dagster , handling complex dependencies, retries, backfills, and observability.
• Build asset-based pipelines with clear ownership, lineage, and SLAs.

Data Transformation

• Develop modular dbt models (staging, intermediate, marts) to transform raw data into clean, production-grade datasets.
• Apply best practices in testing, documentation, and versioning.

Data Ingestion & Python Development

• Write robust Python-based ingestion pipelines for APIs, event streams, files, and third-party systems.
• Build reusable frameworks for validation, enrichment, and complex preprocessing (using Pandas, PySpark, or similar).

Data Modeling & Analytics Enablement

• Implement dimensional models (Star Schema) and Medallion Architecture (Bronze/Silver/Gold).
• Deliver datasets optimized for BI tools, analytics, and downstream consumers.

Quality, Observability & Reliability

• Enforce data quality using dbt tests , freshness checks, and schema validation.
• Implement monitoring and alerting through Dagster and cloud-native tooling.

Collaboration & Product Thinking

• Work closely with Product Managers, Analytics, and Engineering teams to translate business needs into scalable data models.
• Contribute to roadmap planning, technical trade-offs, and prioritization.

Who are we looking for?

We’re looking for a proactive, product-minded data engineer who enjoys building reliable systems, improving developer experience, and turning raw data into trusted insights.

• BSc or MSc in Computer Science, Engineering, or a related field
• 5+ years of experience in data engineering or analytics engineering roles
• Strong Python skills (OOP, data processing, APIs)
• Advanced SQL skills, including query optimization and performance tuning

Hands-on experience with:

• Dagster (assets, schedules, sensors) or similar orchestration tools
• dbt for transformations, testing, and documentation

Solid understanding of modern data architectures:

• Data Lake, Data Warehouse, Lakehouse
• Medallion Architecture, ELT/ETL patterns

Experience with at least one major cloud platform:

• AWS (S3, Glue, Athena), GCP , or Azure

• Familiarity with CI/CD pipelines (GitHub Actions, Azure DevOps, etc.)

Nice to have

• Experience with cloud data warehouses (Snowflake, BigQuery, Redshift)
• Exposure to data observability tools (e.g. Monte Carlo, OpenLineage, OpenTelemetry)
• Experience supporting analytics or BI tools (Power BI, Looker, Tableau.

Benefits

You spend a lot of your time at work, so it should be challenging, fun and interesting. At Defined.ai it will be all of those things and more. Here’s what we offer:

• Flexible working schedule and hybrid model. We know comfort can boost creativity and performance, so you can manage your schedule and work both from one of our modern office spaces or home.

Privacy Notice: defined.ai/candidate-privacy-statement