Senior Software Engineer - LLM Ops & Evals

DataSnipperAmsterdamRemoteFull-time15 Sept 2026

DataSnipper is hiring a Senior Software Engineer for LLM Ops & Evals to own the LLM gateway and build a shared evaluation platform. The role requires 5+ years of backend or platform engineering with strong production Python, LLM inference infrastructure experience, and cloud infrastructure as code. You will work on production systems serving hundreds of thousands of users in audit and finance.

What you'll do

Own the LLM gateway: routing, provider failover, rate limits, retries, and cost attribution across multiple model providers
Deploy, version and deprecate models across clouds, regions and environments, including quota and capacity planning, managed as infrastructure as code
Build the shared evaluation platform: versioned datasets, experiment tracking, run and result schemas, reporting, and trace linkage back to the run
Own the infrastructure for async and long-running AI workloads
Own observability for AI traffic: latency, retries and fallbacks, token usage, cost and errors, per team and per use case
Take part in the on-call rotation, run incidents, and close the follow-ups
Implement the security and compliance controls the platform is held to: retention, access control, RBAC and SSO
Define and maintain clean integration contracts between the platform and the teams that consume it
Partner with product and ML engineers to turn their requirements into platform capabilities that are self-service rather than a request queue

What you need

Must-have

PythonLLM inference infrastructureAzureGCPTerraformOpenTelemetryGrafanaObservability

Nice-to-have

LLM evaluationAgent evaluationTemporalDurable workflow engineSelf-hosted inferenceCapacity planningLoad testingSynthetic data

Soft skills

OwnershipGrowth MindsetCollaborationAdaptabilityJudgmentCommunicationEmpathy

Software DevelopmentEngineeringInformation TechnologyLLM OpsAI EvaluationPlatform EngineeringPythonAzureGCPTerraform

/What you'll do

/What you need

Must-have

Nice-to-have

Soft skills

What you'll do

What you need