
Principal Machine Learning Engineer
AI
Paloma Health is seeking a Principal Machine Learning Engineer to lead the technical definition and measurement of AI quality for autism and ADHD assessments. The role involves defining metrics, evaluation standards, and architecture for AI systems, working closely with clinicians and AI engineers. Requires substantial experience in ML systems, experimental design, and agent development, with a salary of £150,000 per year.
What you'll do
- Own the metric framework for Paloma's AI systems: translate operational and product goals into technical metrics the squad optimises against, keep it current as the roadmap moves, and report progress against it to the Leadership Team
- Define, with our Consultant Child Psychiatrists, what a complete and accurate AI output looks like for different case types, and what the threshold for evidence sufficiency is at each stage of the assessment pathway
- Set the evaluation standard for the squad: specify what evaluation must exist for each part of the pipeline and what a change must demonstrate before it ships, working in close partnership with the AI Engineers who build and run those pipelines
What you need
Must-have
PythonMachine LearningExperimental DesignError AnalysisAgentsMCPOrchestrationContext Management
Nice-to-have
LLM EvaluationGolden DatasetsRegression DatasetsLLM-as-judgeHuman-in-the-loop ReviewHealth DataSynthetic Data GenerationRubric-based EvaluationMentoring
Soft skills
CommunicationCollaborationHigh AgencyJudgmentMentoringTeamwork
Machine LearningHealthcareAIMental HealthPrincipalLondonHybridPythonAgentsEvaluation