Senior Data Engineer
Cpl is supporting a dynamic client in their search for a Senior Data Engineer. In this role, you will play a key part in developing and scaling our strategic data platform on Azure Cloud.
About the Role
You'll focus on building a robust reference data system for securities and pricing, with opportunities to expand into other domains. Bring your passion for innovation and help shape our evolving data landscape.
Key Responsibilities
* Collaborate with cross-functional teams to enhance our Azure-based data platform.
* Lead the creation of a reference data system for securities and pricing, with future expansion into other areas.
* Design, implement, and maintain scalable data pipelines and storage solutions using Azure tools like Data Factory, Lake Gen 2, and Databricks.
* Develop and integrate APIs using frameworks such as Fast API.
* Ensure security, scalability, and performance best practices in cloud solutions.
* Align data solutions with business goals and evolving requirements.
* Continuously optimize data architecture and engineering processes.
Must-Have Qualifications
* 10+ years of experience with deep expertise in Python and Py Spark.
* Proven experience as a Data Engineer with extensive knowledge of Azure Cloud services.
* Proficiency with Azure Data Factory, Lake Gen 2, Databricks, Functions, and API Gateway.
* Hands-on experience with SQL across relational and No SQL databases.
* Proven skills in developing APIs with Fast API or similar frameworks.
* Expertise in ETL/ELT processes and Dev Ops tools (e.g., Git, Jenkins).
* Ability to thrive in a fast-paced, collaborative environment.