SGI logo

Senior Data Engineer

AI
SGIGreater LondonHybridFull-time2 weeks ago

Senior Data Engineer required for a permanent hybrid role in London within the insurance sector. The position demands 10+ years of data engineering experience, strong Azure Databricks, PySpark and SQL skills, and proven delivery of enterprise data platforms and business-facing data products.

What you'll do

  • Design, develop, and maintain data products within an Azure Databricks lakehouse environment
  • Build and optimise Silver and Gold layer transformations using Medallion Architecture principles
  • Contribute to data modelling and architecture discussions

What you need

Must-have

SQLPySparkAzure DatabricksDelta LakeData lakehouse architecturesMedallion ArchitectureData modellingPower BI

Nice-to-have

Databricks Unity CatalogAILLMsRetrieval-Augmented Generation (RAG)Databricks GenieDimensional modellingStar SchemasSlowly Changing Dimensions (SCDs)

Soft skills

Stakeholder managementCommunicationAbility to explain technical concepts to non-technical audiences
Data EngineeringInsuranceAzure DatabricksPySparkPower BISeniorHybridLondon