Paloma Health logo

Principal Machine Learning Engineer

AI
Paloma HealthLondon AreaHybridFull-time1 week ago

Paloma Health is seeking a Principal Machine Learning Engineer to lead the technical definition and measurement of AI quality for autism and ADHD assessments. The role involves defining metrics, evaluation standards, and architecture for AI systems, working closely with clinicians and AI engineers. Requires substantial experience in ML systems, experimental design, and agent development, with a salary of £150,000 per year.

What you'll do

  • Own the metric framework for Paloma's AI systems: translate operational and product goals into technical metrics the squad optimises against, keep it current as the roadmap moves, and report progress against it to the Leadership Team
  • Define, with our Consultant Child Psychiatrists, what a complete and accurate AI output looks like for different case types, and what the threshold for evidence sufficiency is at each stage of the assessment pathway
  • Set the evaluation standard for the squad: specify what evaluation must exist for each part of the pipeline and what a change must demonstrate before it ships, working in close partnership with the AI Engineers who build and run those pipelines

What you need

Must-have

PythonMachine LearningExperimental DesignError AnalysisAgentsMCPOrchestrationContext Management

Nice-to-have

LLM EvaluationGolden DatasetsRegression DatasetsLLM-as-judgeHuman-in-the-loop ReviewHealth DataSynthetic Data GenerationRubric-based EvaluationMentoring

Soft skills

CommunicationCollaborationHigh AgencyJudgmentMentoringTeamwork
Machine LearningHealthcareAIMental HealthPrincipalLondonHybridPythonAgentsEvaluation