memory, Retrieval-Augmented Generation (RAG), and integrations to tools, databases,

and APIs.

● Evaluate and Optimize Agent Performance – Define and implement evaluation

pipelines for agentic systems, including success/failure classification, grounding

accuracy, reasoning robustness, tool-use reliability, and long-horizon task completion.

Use metrics and benchmarks to continuously improve performance in production

environments.

● Orchestrate & Optimize – Design supervisor/sub-agent patterns, enable coordination

across agents, and apply best practices for robustness, performance, and scalability.

● Deploy & Evolve – Deliver production-grade agentic AI workflows on cloud platforms

(AWS preferred), monitor and evaluate agent performance, and continuously fine-tune

for quality and efficiency.

● Collaborate & Translate – Work with product managers, researchers, and engineers to

transform complex chip design workflows into agent-driven, end-to-end solutions.

Desired Skillset

● Agentic AI Systems

○ Hands-on experience with multi-agent orchestration and supervisor patterns.

○ Familiarity with LangChain, LangGraph, LangSmith, or similar frameworks.

○ Knowledge of RAG pipelines, memory management, and evaluation of agent

performance.

● Evaluation and Optimization

○ Familiarity with agent evaluation frameworks (e.g., LangSmith evals, benchmark

datasets, unit tests for agent workflows).

○ Experience designing custom evaluation metrics for reasoning steps, tool use

correctness, and RAG pipeline performance.

○ Ability to balance automatic evaluation (synthetic benchmarks, self-play) with

human-in-the-loop evaluation for complex workflows.

● Cloud & Infrastructure

○ Strong software engineering foundation with experience in cloud environments

(AWS preferred).

○ Knowledge of containerized deployment and backend integration.

● Programming & Development

○ Proficiency in Python; experience with backend systems and API integration.

○ Familiarity with modern software engineering practices (GitHub, CI/CD pipelines,

testing frameworks).

Required Qualifications:

● Bachelor’s or Master’s degree in Computer Science, Software Engineering or a related

field.

● 5-10 years of experience in software development in cloud environment

● 2+ years of experience with hands-on experience building and deploying production-

grade agentic AI systems with real-world applications.

● Solid working knowledge of Retrieval-Augmented Generation (RAG), agent performance

evaluation, and hands-on experience with LangGraph and LangSmith platforms.

● Proficiency in Python and familiarity with backend cloud services (preferably AWS).

Preferred Qualifications

● Contributions to open-source AI projects or frameworks.

● Experience with multi-agent orchestration patterns at scale.

● Knowledge of reinforcement learning, planning algorithms, or autonomous reasoning.

● Track record of deploying agentic AI systems in production at scale