Cpl are supporting a dynamic client in their search for a Senior Data Engineer. In the role, you will play a key role in developing and scaling our strategic data platform on Azure Cloud. You'll focus on building a robust reference data system for securities and pricing, with opportunities to expand into other domains. Bring your passion for innovation and help shape our evolving data landscape.
Key Responsibilities:
* Collaborate with cross-functional teams to enhance our Azure-based data platform.
* Lead the creation of a reference data system for securities and pricing, with future expansion into other areas.
* Design, implement, and maintain scalable data pipelines and storage solutions.
* Utilize Azure tools like Data Factory, Lake Gen 2, and Databricks for efficient data processing.
* Develop and integrate APIs using frameworks such as FastAPI.
* Ensure security, scalability, and performance best practices in cloud solutions.
* Align data solutions with business goals and evolving requirements.
* Continuously optimize data architecture and engineering processes.
Must-Have Qualifications:
* 10+ years of experience with deep expertise in Python and PySpark.
* Proven experience as a Data Engineer with extensive knowledge of Azure Cloud services is essential.
* Proficiency with Azure Data Factory, Lake Gen 2, Databricks, Functions, and API Gateway.
* Hands-on experience with SQL across relational and NoSQL databases.
* Proven skills in developing APIs with FastAPI or similar frameworks.
* Expertise in ETL/ELT processes and DevOps tools (e.g., Git, Jenkins).
* Ability to thrive in a fast-paced, collaborative environment.
#J-18808-Ljbffr