Senior Applied ML Engineer | Kimchi (Model Router)

KimchiNetherlandsRemoteFull-time3 weeks ago

Kimchi, a product of Cast AI, is seeking a Senior Applied ML Engineer to own the routing model that decides which LLM serves each request. The role involves training, evaluating, and shipping models that make real-time decisions, with a focus on production-parity evaluation and online experimentation. Requires 5+ years in applied ML, strong Python and SQL skills, and deep understanding of LLM serving realities.

What you'll do

Own the routing model end to end: features, objective, training, calibration, and release cadence.
Build the evaluation that gates every change: production-parity benchmarks, router-vs-router comparisons, holdouts by task family, error bars on pass rates.
Turn fleet data into training signal: per-request outcomes from production, benchmark rollouts, and override feedback, with failure categorization.
Model the objective the user actually experiences: task success × token cost × wall-clock speed under the real concurrency regime.
Run the online loop: shadow, A/B, full rollout, drift monitoring, retrain.
Push the frontier where it pays: calibrated decisions, abstention under uncertainty, speed-aware routing.
Partner with platform engineers on instrumentation: the features and labels recorded per request and the experiments the data path must support.

What you need

Must-have

PythonPyTorchSQLClickHouseOpenAI-compatible LLM APIsAnthropic-compatible LLM APIsRouterBench-class benchmark harnessesGitLab CI

Nice-to-have

RouterBench-class evalsCost/quality/latency Pareto frontsConformal methodsAbstention methods

Soft skills

Instrument before optimizeComfortable with wide mandateStrong communicationCollaboration

Applied MLLLM RoutingModel RouterSeniorRemotePythonPyTorchSQLClickHouseKubernetes

/What you'll do

/What you need

Must-have

Nice-to-have

Soft skills

What you'll do

What you need