Data Scientist

Commonwealth BankSydneyHybridFull-time1 week ago

Data Scientist role in AI & Model Validation at a financial services group in Sydney, independently testing and challenging machine learning, Generative AI and agentic AI use cases. Requires advanced Python skills, experience with Generative AI and agentic architectures, and strong communication to explain complex technical findings.

What you'll do

Independently deliver defined components of AI and ML validation, including technical analysis, testing and evaluation
Review data, methodologies, assumptions and model outcomes, investigating unexpected behaviour and potential weaknesses
Design and execute evaluations for Generative AI and agentic systems, including architectures involving RAG, tool use, guardrails, routing, memory and LLM-as-a-Judge approaches
Build reproducible test harnesses, reusable evaluations and automation that improve the efficiency and consistency of validation
Navigate unfamiliar codebases, reproduce evaluation pipelines and inspect trace data to understand model and agent behaviour
Provide constructive, evidence-based technical challenge and bring findings into broader validation conclusions
Communicate complex technical findings clearly to both technical and non-technical stakeholders
Share knowledge and contribute to the continuing development of AI validation practices, tooling and standards

What you need

Must-have

PythonGitHubVersion controlCI/CDSQLRTypeScriptLangChain

Soft skills

CommunicationAnalytical judgementAttention to detailCuriosityIndependent learningConstructive challenge

Data ScientistAIMachine LearningModel ValidationGenerative AIFinancial ServicesPythonSydneyHybrid

/What you'll do

/What you need

Must-have

Soft skills

What you'll do

What you need