This range is provided by Mason Alexander. Your actual pay will be based on your skills and experience — talk with your recruiter to learn more.
Base pay range
As a Principal Data Engineer, you will manage Azure related data architecture and cloud infrastructure supporting complex risk models (statistical/AI/ML). The focus for this position will be the design, development, and maintenance of the data architecture components (including ETL data pipelines) required to ingest data from sources and expose data to downstream consumers.
The role consists of 70% Data Engineering and 30% infrastructure.
Responsibilities
* Design, configure, manage data engineering infrastructure on the cloud (Databricks/Azure DataFactory/Azure Data Lake).
* Maintain data ingestion and exposure (including via APIs) with high security and availability.
* Ensure data quality standards are established and enforced.
* Build infrastructure using code and repeatable designs in Terraform.
* Maintain Azure cloud infrastructure across prod and non-prod environments.
Required
* Extensive hands-on experience developing data pipelines that demonstrate a strong understanding of software engineering principles.
* Excellent knowledge of Azure Services such as – Blobs, Functions, Azure Data Factory, Service Principal, Containers, Key Vault, etc.
* Experience working with both real-time and batch data, knowing the strengths and weaknesses of both and when to apply one over another.
* Experience developing and deploying APIs.
* Expert skills in Python, SQL, Spark.
* Experience designing best practice Data Architecture, Data Quality, and Data Security processes and implementing those utilizing the latest tools and technologies.
* Experience working with Terraform for cloud infrastructure automation and management.
Seniority level
Mid-Senior level
Employment type
Full-time
Job function
Engineering and Information Technology
Industries
Business Consulting and Services
#J-18808-Ljbffr