Senior AI Infrastructure Engineer

HeidiMelbourneHybridFull-time14 Sept 2026

Heidi is seeking a Senior AI Infrastructure Engineer in Melbourne to build and operate the infrastructure behind its clinical AI models. The role spans model serving, GPU cluster management, inference performance, and production reliability, requiring hands-on experience deploying LLMs and managing GPU workloads.

What you'll do

Build and own model-serving infrastructure, including deployment pipelines, request routing, autoscaling, fallback paths, and controlled rollouts and rollbacks across regions.
Manage GPU clusters and workload scheduling, improving resource allocation across inference, training, and evaluation.
Improve inference performance by profiling workloads and optimizing latency, throughput, and memory efficiency.
Support distributed training and model iteration, providing reliable ways to launch fine-tuning and training jobs.
Make deployments observable and incidents traceable by connecting requests to models and building dashboards and alerts.
Own production reliability, defining service objectives, investigating incidents, and building recovery procedures.
Make compute costs actionable by tracking GPU usage and inference cost, and using forecasts to guide deployment choices.
Build a platform the model team can use independently, automating provisioning, configuration, benchmarking, and releases.

What you need

Must-have

PythonGoC++RustLinuxKubernetesSlurmvLLM

Nice-to-have

PyTorch FSDPMegatronReinforcement LearningMoE modelsQuantizationSpeculative DecodingPrefill/Decode DisaggregationNCCL

Soft skills

OwnershipProblem SolvingCommunicationCollaborationAdaptabilityCritical Thinking

AI InfrastructureHealthcareLLMGPUModel ServingDistributed TrainingKubernetesPythonSeniorHybrid

/What you'll do

/What you need

Must-have

Nice-to-have

Soft skills

What you'll do

What you need