Principal Data Engineer and Azure Cloud Infrastructure
Optum is a global organization that delivers care, aided by technology to help millions of people live healthier lives. The work you do with our team will directly improve health outcomes by connecting people with the care, pharmacy benefits, data, and resources they need to feel their best.
About the Role:
We are one of the world's leading healthcare companies, with no limits on the resources you'll have or the challenges you'll encounter. We have been supporting global healthcare systems from Ireland and the UK for over 20 years, building a dynamic and diverse team of more than 2,100 talented individuals.
Job Description:
As a Principal Data Engineer, you will manage Azure-related data architecture and cloud infrastructure supporting complex risk models (statistical/AI/ML). The focus for this position will be the design, development, and maintenance of the data architecture components (including ETL data pipelines) required to ingest data from sources and expose data to downstream consumers.
* Design, configure, manage data engineering infrastructure on the cloud (Databricks/Azure DataFactory/Azure Data Lake)
* Maintain data ingestion and exposure (including via APIs) with high security and availability
* Ensure data quality standards are established and enforced
* Build infrastructure using code and repeatable designs in Terraform
* Maintain Azure cloud infrastructure across prod and non-prod environments
* Identify and resolve security and compliance vulnerabilities
* Troubleshoot production support issues post-deployment and come up with solutions as required
* Identify opportunities for improvement, optimize performance and scalability of existing solutions
* Setup and maintain related documentation and standards
* Prioritize and manage work to critical project timelines in a fast-paced environment
* Partner with Data Science and Product leaders to design best practices and standards for developing and productionalizing analytic pipelines
* Mentor and train junior members of the team
Requirements:
* Extensive hands-on experience developing data pipelines that demonstrate a strong understanding of software engineering principles
* Excellent knowledge of Azure Services such as – Blobs, Functions, Azure Data Factory, Service Principal, Containers, Key Vault etc.
* Experience working with both real-time and batch data, knowing the strengths and weaknesses of both and when to apply one over another
* Experience developing and deploying APIs
* Expert skills in Python, SQL, Spark
* Experience designing best practice Data Architecture, Data Quality and Data Security process and implementing those utilizing the latest tools and technologies
* Experience working with Terraform for cloud infrastructure automation and management
* Bring innovative ideas to help continuously improve the application portfolio including the design, build, deploy, test, and runtime within cloud ecosystems
* Proven critical thinking skills and persistence to solve complex problems
* Flexibility to take on various responsibilities in a start-up like team atmosphere
Preferred Qualifications:
* Experience using DevOps tools, Git workflow, and building CI/CD pipelines
* Experience working in projects with agile/scrum methodologies
* Have a solid understanding of Artificial Intelligence and Machine Learning capabilities
* Work on Proof of Concepts for Big Data and Data Science
* Understanding of United States Healthcare data
Please note you must currently be eligible to work and remain indefinitely without any restrictions in the country to which you are making an application. Proof will be required to support your application.