Who you are

• If you know Playwright inside and out and have scars from trying to test LLM hallucinations in production, we want to talk to you
• Experience: 5+ years in QA Automation or Software Engineering in Test (SDET)
• Playwright expertise: You have deep, hands-on experience building reliable, scalable test suites in Playwright. You know how to handle flaky tests, parallel execution, and complex DOM structures
• Coding chops: Strong programming skills in TypeScript, JavaScript, or Python
• AI/LLM testing experience: You understand how LLMs work under the hood. You know the challenges of testing them (non-determinism, evaluating accuracy vs. exact match, security/injection risks) and have used tools or frameworks (like LLM-as-a-judge, LangSmith, DeepEval, etc.) to evaluate them
• Systems thinking: You can look at a complex architecture involving a frontend, backend APIs, vector databases, and LLM endpoints, and know exactly where things are likely to break
• Communication: You can clearly explain complex QA issues to both highly technical machine learning engineers and non-technical stakeholders
• Experience in the financial tech or document automation space
• Familiarity with containerization (Docker, Kubernetes) and advanced CI/CD setups (GitHub Actions, GitLab CI)
• Experience testing API performance and LLM endpoint latency

What the job involves

• We are looking for a Senior Automated QA Engineer to lead our testing efforts. You won't just be testing standard web interfaces; you'll be figuring out how to reliably automate testing for non-deterministic AI features, multi-step AI agents, and complex LLM pipelines
• Own the E2E framework: Build, maintain, and scale our automated testing framework using Playwright (TypeScript/Python)
• Test the unpredictable: Design strategies to test non-deterministic LLM outputs, AI agents, and RAG pipelines where standard assertions don't always work
• Tackle LLM-specific challenges: Build guardrails and automated checks for prompt drift, hallucinations, latency, and context window limits
• Evaluate Agent behavior: Create scenarios to test how our AI agents handle edge cases, multi-step reasoning, and error recovery in real-world document processing workflows

Benefits

• Make your own decisions: You are given the ownership of your area and responsibility to make the decisions
• Work whenever, wherever: Work from any corner of the globe - live where you love it, work when you're at your best, and blend life with career seamlessly
• Unlimited Vacation: We want you to feel your best at work. Take all the time off you need
• Employee Referral Program: We are always searching for bright minds to help us reach our goals. Your effort in helping us will not go unnoticed
• Company & Team Events; You'll have the option to travel often. Be it to team off sites or all-company events. All expenses covered
• German Language Courses: Our company language is English, but if you want to improve your German, we'll give you a helping hand