What the job involves

• We are seeking skilled Cloud and ML Infrastructure Engineers to lead the buildout of our AWS foundation and our LLM platform
• You will design, implement, and operate services that are scalable, reliable, and secure
• The broad scope means focus areas in LLM/ML Infra and IoT infra are strong bonus points
• For ML Infra, build the stack that powers retrieval-augmented generation and application workflows built with frameworks like LangChain
• You will work closely with other engineers and product management
• Cloud Infrastructure Setup and Maintenance
• Design, provision, and maintain AWS infrastructure using IaC tools such as AWS CDK or Terraform
• Build CI/CD and testing for apps, infra, and ML pipelines using GitHub Actions, CodeBuild, and CodePipeline
• Operate secure networking with VPCs, PrivateLink, and VPC endpoints. Manage IAM, KMS, Secrets Manager, and audit logging
• LLM Platform and Runtime
• Stand up and operate model endpoints using AWS Bedrock and/or SageMaker; evaluate when to use ECS/EKS, Lambda, or Batch for inference jobs
• Build and maintain application services that call LLMs through clean APIs, with streaming, batching, and backoff strategies
• Implement prompt and tool execution flows with LangChain or similar, including agent tools and function calling
• RAG Data Systems and Vector Search
• Design chunking and embedding pipelines for documents, time series, and multimedia. Orchestrate with Step Functions or Airflow
• Operate vector search using OpenSearch Serverless, Aurora PostgreSQL with pgvector, or Pinecone. Tune recall, latency, and cost
• Build and maintain knowledge bases and data syncs from S3, Aurora, DynamoDB, and external sources
• Evaluation, Observability, and Cost Governance
• Create offline and online eval harnesses for prompts, retrievers, and chains. Track quality, latency, and regression risk
• Instrument model and app telemetry with CloudWatch and OpenTelemetry. Build token usage and cost dashboards with budgets and alerts
• Add guardrails, rate limits, fallbacks, and provider routing for resilience
• Safety, Privacy, and Compliance
• Implement PII detection and redaction, access controls, content filters, and human-in-the-loop review where needed
• Use Bedrock Guardrails or policy services to enforce safety standards. Maintain audit trails for regulated environments
• Data Pipeline Construction
• Build ingestion and processing pipelines for structured, unstructured, and multimedia data. Ensure integrity, lineage, and cataloging with Glue and Lake Formation
• Optimize bulk data movement and storage in S3, Glacier, and tiered storage. Use Athena for ad-hoc analysis
• IoT Deployment Management
• Manage infrastructure that deploys to and communicates with edge devices. Support secure messaging, identity, and over-the-air updates
• Analytics and Application Support
• Partner with product and application teams to integrate retrieval services, embeddings, and LLM chains into user-facing features
• Provide expert troubleshooting for cloud and ML services with an emphasis on uptime and performance
• Performance Optimization
• Tune retrieval quality, context window use, and caching with Redis or Bedrock Knowledge Bases
• Optimize inference with model selection, quantization where applicable, GPU/CPU instance choices, and autoscaling strategies
• Leveling Breakdown:
• Engineer (3–5 yrs)
• Scope: Independently delivers features and subsystems
• Contributions: Builds CI/CD pipelines, deploys ML endpoints (Bedrock, SageMaker), develops RAG pipelines and vector search integrations, manages infra security (IAM, KMS)
• Requirements: 3–5 yrs in cloud/infra/ML systems, hands-on with AWS services, experience with APIs, data pipelines, and at least one ML/LLM integration
• Senior/Staff Engineer (6+ yrs)
• Scope: Leads architecture and technical direction across cloud + ML infra
• Contributions: Designs large-scale RAG/data systems, optimizes inference and retrieval, drives observability and cost governance, implements safety/privacy frameworks, mentors teammates
• Requirements: 6+ yrs in cloud + ML infrastructure, deep AWS expertise (ECS/EKS, Bedrock, SageMaker, Step Functions), proven record scaling LLM/RAG systems, strong security and compliance practices. Bonus: IoT deployments, GPU/quantization, aviation/safety-critical experience

Benefits

• Comprehensive Healthcare Coverage: Enjoy peace of mind with our generous health, vision, and dental benefits, with 75% of costs covered by the company for the employee. Note: Starting January 1, 2025 our coverage is increasing. Beacon AI will cover 80% of healthcare coverage costs for the employee and 25% for their dependents
• Paid Time Off: Recharge and relax with 3 weeks of paid vacation, in addition to 14 company-paid holidays each year
• Connectivity Stipend: Stay connected with our cell phone benefit, ensuring you have the tools you need to excel in your role
• Health and Wellness Allowance: Use this towards a gym membership or subscription to a meditation app, empowering you to prioritize self-care and maintain a healthy lifestyle
• Financial Planning: Prepare for the future with our 401(k) program. While we currently do not offer matching, we are committed to enhancing this benefit in the future