Responsibilities

• Define and execute the QA strategy covering backend, frontend, and AI workflows.
• Establish and maintain the full test pyramid: unit, contract, integration, and end-to-end testing. Integrate testing and evaluation into GitOps pipelines (GitHub Actions → Argo CD/Rollouts) with measurable quality gates.
• Build automated test suites for Python (FastAPI) services and the Vue-based web console using pytest and Playwright. Collaborate closely with frontend and backend developers to provide the necessary tooling and infrastructure to facilitate test-driven development.
• Implement contract testing based on OpenAPI/AsyncAPI specifications with backward-compatibility validation in CI.
• Introduce and maintain AI evaluation frameworks for RAG and agent-based systems, measuring groundedness, citation accuracy, and model quality across releases. Collaborate with data and ML engineers to validate dataset consistency, privacy handling, and AI safety mechanisms.
• Develop and run performance and reliability tests (load, soak, chaos) with monitoring and SLO dashboards (OpenTelemetry + Prometheus/Grafana).
• Collaborate with infrastructure and security teams to integrate SBOM/SCA, image scanning, and secret detection into release pipelines. Work with engineers and product managers to ensure feature testability and clear acceptance criteria.
• Advocate for quality and security best practices across cross-functional discussions.
• Maintain documentation for QA strategy, coverage, and test environments. Contribute to engineering retrospectives and release post-mortems to improve process maturity.

Qualifications

• Experience: 4+ years in QA or Quality Engineering for cloud-based services, ideally in SaaS or AI-related environments.
• Automation Skills: Proficient in Python testing frameworks (pytest) and end-to-end testing tools such as Playwright or Cypress.
• Performance Testing: Familiar with load and stress testing using tools like k6, Locust, or JMeter.
• AI Testing: Exposure to LLM or RAG evaluation (Ragas, DeepEval, or similar), or strong motivation to develop expertise in this area.
• Security and CI/CD: Understanding of SBOM management, software composition analysis (Snyk, Trivy, Mend), security testing, and secure CI/CD practices.
• Observability: Working knowledge of metrics, traces, and dashboards using OpenTelemetry, Prometheus, and Grafana. Familiarity with GitOps workflows and CI/CD pipelines (GitHub Actions, Argo CD, or equivalent).
• Mindset: Analytical, detail-oriented, and proactive. Comfortable working autonomously in a fast-paced startup.
• Advantageous Experience:
• Using TestContainers or record/replay frameworks for API simulation.
• Basic understanding of IAM or infrastructure policy validation.
• Strong experience with OpenAPI/AsyncAPI and contract-driven development.