ML Engineer
AI
Base Compute, an AI inference lab focused on on-device AGI, is seeking an ML Engineer in Melbourne to work on the frontier of on-device AI. The role involves building and optimizing inference infrastructure, from custom kernels to low-latency serving runtimes, requiring 3+ years of ML engineering or systems programming experience.
What you'll do
- Inference engine development: Building and scaling our custom inference engine, handling everything from weight loading and KV-cache management to efficient request scheduling
- Cross-platform silicon optimization: Writing and tuning custom kernels and leveraging hardware-specific instructions to squeeze maximum performance out of diverse architectures, including Apple Silicon, NVIDIA, AMD, Snapdragon, and other edge platforms
- Systems architecture: Developing robust, low-latency serving runtimes in C++ to manage model routing, continuous batching, and novel decoding strategies under strict thermal and memory constraints
- Performance profiling: Identifying and eliminating bottlenecks across the entire stack, from memory bandwidth ceilings to kernel interleaving
What you need
Must-have
RustC/C++GPU programmingCUDAROCmMetalTritonLLM architectures
Soft skills
ownershipautonomycommunicationfeedbackdocumentation
ML EngineerArtificial Intelligenceon-device AIinferencesystems programmingGPU programmingC++RustLLMMelbourne