Functions

• Design and implement new data pipelines tailored to a specific project or new product.
• Continuously improve data pipelines to make them more generic and modular, reducing the effort required for subsequent integrations.
• Write, optimise and refactor prompts and prompt-related pipelines.
• Conduct thorough testing and validation of data pipelines to ensure accuracy and consistency of data.
• Ensure pipelines are optimized for performance, scalability, and reusability to facilitate future projects.

Examples of day-to-day tasks

• We want to support people’s enrichment within the CatchAll Tool. Do research on what is available on the market that can help us enrich data, Double-check open databases, and verify whether there is already an existing solution available within the NewsCatcher’s code base. Plan and design the pipeline. Build a prototype and demo it to the whole team. Proceed to, test on Dev, write test cases around. Deploy on Prod, get feedback and improve.
• One of the users complained about a job being stuck. Debug the pipeline, try to understand where the problem comes from: promts, code, database. Find the issue and prepare a fix. Test it within the whole pipeline on Dev. Break some pipeline, fix and test again. Deploy on Prod, make clients happy.
• Prompt responsible for validating results is giving only 50% precision. Review the prompt, review the model. Do some prompt engineering. Realize that “with each new model I feel like LLMs become stupier”. Try another LLM provider, adapt the prompt. Test results, improve the accuracy and create a dataset to prove it. Get 70% precision, wait for compliments.

Experience

• 3+ years of experience in B2B SaaS as Backend / Software / Data Engineer
• Strong systems thinker with attention to performance and scalability
• Comfortable working with both SQL and NoSQL databases
• Experience shipping LLM-based functionality into production
• Able to move from prototype → stable, maintainable architecture

Must Have

• Strong Python (including async workloads)
• Docker & Kubernetes
• RabbitMQ (or Kafka / PubSub / SNS / ActiveMQ)
• MongoDB / DynamoDB / Redis
• Hands-on experience with LLM frameworks

We Also Use

• Elasticsearch/OpenSearch
• PostgreSQL / MySQL
• GitLab
• AWS / GCP / DigitalOcean
• Jenkins

Nice to Have

• Experience building production AI agents
• Frontend experience (useful for product UI)
• Experience in API-first or DaaS companies

Compensation And Perks

• Competitive salary and equity
• Up to 24 days of vacation & 16 days of sick leave/holidays (all fully paid)
• One meeting-free day per week
• Co-working Budget
• Training Budget
• We provide all the necessary equipment to work comfortably and efficiently from home.
• Yearly company retreats (2025 — Portugal, 2024 — Canary Islands, 2023 — French Alpes)