Staff Data Engineer

Commonwealth BankSydneyHybridFull-time2 weeks ago

Staff Data Engineer role at CommBank's Payments Technology team in Sydney or Melbourne, providing hands-on technical leadership for data platforms, data products and pipelines. Requires deep experience with AWS data services, Snowflake, Python/PySpark, distributed data processing and modern batch and streaming pipelines.

What you'll do

Set technical direction for complex data engineering initiatives and influence architecture and standards across multiple squads
Design, build and evolve scalable batch and streaming data pipelines using technologies such as Spark, Airflow and AWS Glue
Build cloud-native data solutions across AWS, including S3, EMR, Glue, Lambda, Redshift and SageMaker
Engineer high-quality data products and help evolve enterprise data platform and data marketplace patterns
Lead engineering practices across testing, CI/CD, infrastructure-as-code, observability and production reliability
Establish patterns for data quality, resilience, performance and maintainability across distributed data workloads
Mentor engineers, support technical decision-making and raise data engineering capability across teams
Partner with Architecture, Product, Security and data consumers to translate complex requirements into pragmatic engineering solutions

What you need

Must-have

AWSAmazon S3Amazon EMRAWS GlueAWS LambdaAmazon RedshiftAmazon SageMakerSnowflake

Soft skills

Technical leadershipMentoringInfluencingProblem solvingDecision-making

Data EngineeringFinancial ServicesPaymentsAWSSnowflakePythonPySparkStaffHybrid

/What you'll do

/What you need

Must-have

Soft skills

What you'll do

What you need