Data Engineer

London (hybrid working)

£up to £68,000

A leading financial organisation based in London is currently looking to recruit a Data Engineer to join a growing machine learning team ideally with a strong track record of data modelling, developing, and delivering large scale data transformation project aligned with the overarching Data Strategy. The successful data engineer will define, develop, and deliver data transformations that not only drives our Machine learning models but create more strategic transformation that delivers consistent data across the organisation for the various business needs.

Key responsibilities

• Delivering data transformation based on the overarching data strategy
• Design, development and maintenance of Data transformation pipelines aligning with Data Strategy.
• Test driven development, clean code and documentation
• Delivering on agreed service levels, which you'll agree with your manager - and your contribution to our commitments
• Working with business stakeholders and project team to identify ML and IA opportunities, and understand and capture business requirements
• Providing quality assurance on machine learning code developed by other team members
• You'll be working with architects, product owners and business analysts and contributing to the requirement definition, solution design and architectural designs.
• You'll also be involved in sprint planning by defining and owning the machine learning backlog as well as providing estimates for your work.

Minimum Criteria

• Solid programming experience in Python (Pandas), SQL, NoSQL - TDD
• Previous experience in building large scale data transformation from structured, un-structured and streaming data sources using Python, SQL
• Experience in any of Azure Data Factory Azure Event Grid
• Developing serverless pipelines in any of Azure
• Understanding API based development (e.g. REST APIs) to integrate with different systems
• Building and maintaining CI/CD pipeline in Azure Devops
• Working in an agile development environment with the ability to communicate technical concepts and analytical outputs to a diverse set of stakeholders.
• Being able to provide estimates against organisational requirements relating to cost and development tasks