Staff Engineer
Location:
San Francisco, CA / San Mateo, CA / Seoul, KR
About the Job
FriendliAI is seeking a Staff Software Engineer to provide technical leadership in building the core systems behind our AI inference platforms. This role operates with
broad technical scope with company-wide impact
, shaping architectural decisions that affect multiple products, teams, and customer workloads. You will work at the intersection of distributed systems, AI model serving, agent execution platforms, and developer infrastructure, building foundational systems that power production-scale inference.
As a Staff Engineer, you will define long-term technical direction, lead complex multi-quarter initiatives, and influence engineering decisions well beyond your immediate team. This is a deeply hands-on role with significant architectural ownership and cross-functional impact.
Key Responsibilities
- Own and evolve the technical architecture of core components of FriendliAI’s inference platforms, operating with broad technical scope with company-wide impact.
- Define architectural direction for scalable, multi-tenant systems supporting high-throughput, low-latency AI inference workloads.
Qualifications
- Experience operating at a
staff-level scope
, leading ambiguous, cross-team technical initiatives and owning architecture that spans multiple systems or teams.
- 10+ years of professional software engineering experience, with at least 2-3 years operating at a staff-level scope.
- Strong experience designing, building, and operating distributed systems in production environments.
- Proficiency in at least one backend or systems language (e.g., Python, Go, Rust, C++), with experience building production platforms, APIs, or infrastructure.
Preferred Experience
- Strong experience building large-scale backend systems in
Python
, including production services, APIs, or internal platforms.
- Experience with AI inference optimization, GPU scheduling, batching, caching, or model lifecycle management.
- Experience designing developer platforms or internal frameworks used by other engineers.
- Exposure to serverless systems, control planes, or multi-tenant SaaS architectures.
- Experience leading incident response for production systems.
Benefits
- A front-row seat to the generative AI infrastructure revolution.
- Competitive compensation and benefits package.
- Daily lunch and dinner provided; unlimited snacks and beverages.
- Health check-up and top-tier hardware support.
- Flexible working hours and a highly collaborative environment.
About us
FriendliAI is building the next-generation AI inference platform that accelerates the deployment of large language and multimodal models with unmatched performance and efficiency. Our infrastructure powers high-throughput, low-latency workloads for global organizations and integrates directly with Hugging Face, providing instant access to over 480,000 open-source models. We are on a mission to deliver the world’s best platform for AI inference.
