Principal Data Engineer and Azure Cloud Infrastructure
Dublin, Ireland
We are a global organisation that delivers care, aided by technology to help millions of people live healthier lives. Our team is guided by diversity and inclusion, talented peers, comprehensive benefits and career development opportunities.
About the Role:
We're a Fortune 5 business, one of the world's leading healthcare companies. We have been supporting global healthcare systems from Ireland and the UK for over 20 years, building a dynamic and diverse team of more than 2,100 talented individuals.
Job Description:
As a Principal Data Engineer, you will manage Azure related data architecture and cloud infrastructure supporting complex risk models (statistical/AI/ML). The focus for this position will be the design, development and maintenance of the data architecture components (including ETL data pipelines) required to ingest data from sources and expose data to downstream consumers.
Working Schedule:
Careers with us offer flexible work arrangements and individuals who live and work in the Republic of Ireland will have the opportunity to split their monthly work hours between our Dublin office 3 days a week and telecommuting from a home-based office in a hybrid work model.
Primary Responsibilities:
* Design, configure, manage data engineering infrastructure on the cloud (Databricks/Azure DataFactory/Azure Data Lake)
* Maintain data ingestion and exposure (including via APIs) with high security and availability
* Ensure data quality standards are established and enforced
* Build infrastructure using code and repeatable designs in Terraform
* Maintain Azure cloud infrastructure across prod and non-prod environments
* Identify and resolve security and compliance vulnerabilities
* Troubleshoot production support issues post-deployment and come up with solutions as required
* Identify opportunities for improvement, optimize performance and scalability of existing solutions
* Setup and maintain related documentation and standards
* Prioritize and manage work to critical project timelines in a fast-paced environment
* Partner with Data Science and Product leaders to design best practices and standards for developing and productionalizing analytic pipelines
* Mentor and train junior members of the team
Required qualifications:
* Extensive hands-on experience developing data pipelines that demonstrate a strong understanding of software engineering principles
* Excellent knowledge of Azure Services such as – Blobs, Functions, Azure Data Factory, Service Principal, Containers, Key Vault etc.
* Experience working with both real-time and batch data, knowing the strengths and weaknesses of both and when to apply one over another
* Experience developing and deploying APIs
* Expert skills in Python, SQL, Spark
* Experience designing best practice Data Architecture, Data Quality and Data Security process and implementing those utilizing the latest tools and technologies
* Experience working with Terraform for cloud infrastructure automation and management
* Bring innovative ideas to help continuously improve the application portfolio including the design, build, deploy, test and runtime within cloud ecosystems
* Proven critical thinking skills and persistence to solve complex problems
* Flexibility to take on various responsibilities in a start-up like team atmosphere
Estimated salary: €120,000 - €180,000 per annum