Research Staff, Voice AI Foundations [Remote]

DeepgramMelbourneHybridFull-time3 weeks ago

Deepgram is seeking a Research Staff member to pioneer Latent Space Models for voice AI, addressing fundamental data, scale, and cost challenges. The role involves building next-generation neural audio codecs, steerable generative models, and embedding systems, with a focus on scaling training and real-time inference. Candidates should have strong mathematical foundations, deep expertise in foundation model architectures, and a track record of impactful research.

What you'll do

Build next-generation neural audio codecs that achieve extreme, low bit-rate compression and high fidelity reconstruction across a world-scale corpus of general audio.
Pioneer steerable generative models that can synthesize the full diversity of human speech from the codec latent representation, from casual conversation to highly emotional expression to complex multi-speaker scenarios with environmental noise and overlapping speech.
Develop embedding systems that cleanly factorize the codec latent space into interpretable dimensions of speaker, content, style, environment, and channel effects -- enabling precise control over each aspect and the ability to massively amplify an existing seed dataset through “latent recombination”.
Leverage latent recombination to generate synthetic audio data at previously impossible scales, unlocking joint model and data scaling paradigms for audio. Endeavor to train multimodal speech-to-speech systems that can 1) understand any human irrespective of their demographics, state, or environment and 2) produce empathic, human-like responses that achieve conversational or task-oriented objectives.
Design model architectures, training schemes, and inference algorithms that are adapted for hardware at the bare metal enabling cost efficient training on billion-hour datasets and powering real-time inference for hundreds of millions of concurrent conversations.

What you need

Must-have

Statistical Learning TheorySelf-Supervised LearningMultimodal LearningFoundation Model ArchitecturesData PipelinesControlled ExperimentsModel OptimizationHardware Constraints

Soft skills

CreativityRelentless DriveAdaptabilityProblem-SolvingAI-First MindsetExperimentationLogical ReasoningAssumption TestingCollaboration

Voice AIResearchDeep LearningSpeech RecognitionGenerative ModelsNeural CodecsFoundation ModelsRemoteHybridFull-time

/What you'll do

/What you need

Must-have

Soft skills

What you'll do

What you need