This range is provided by Atrium (EMEA). Your actual pay will be based on your skills and experience — talk with your recruiter to learn more.
Base pay range
This is a 12mth contract role paying between €490pd (PAYE) and €618pd (Umb/Irish Ltd)
Atrium are looking for an accomplished Senior Data Engineer to be responsible for the development of complex data sources and pipelines into our clients’ data platform (i.e. Snowflake) along with other data applications (i.e. Azure, Airflow, etc.) and automation.
The Senior Data Engineer will work closely with the data, Architecture, Business Analyst, Data Stewards to integrate and align requirements, specifications and constraints of each element of the requirement. They will also need to help identify gaps in resources, technology, or capabilities required, and work with the data engineering team to identify and implement solutions where appropriate. 1-2 days a week in office work required.
Primary Responsibilities:
* Integrate data from multiple on-prem and cloud sources and systems. Handle data ingestion, transformation, and consolidation to create a unified and reliable data foundation for analysis and reporting.
* Develop data transformation routines to clean, normalize, and aggregate data.
* Apply data processing techniques to handle complex data structures.
* Monitor data pipelines and data systems to detect issues.
* Develop monitoring tools to automate error handling mechanisms.
* Utilize data quality tools like Great Expectations or Soda to ensure the accuracy, reliability, and integrity of data.
* Create SQL Stored procedures to perform complex transformations.
* Understand data requirements and design optimal pipelines to fulfill the use-cases.
* Create logical & physical data models to ensure data integrity is maintained.
* Tune and optimize data processes.
Minimum Requirements:
* Proven hands-on experience as a Data Engineer.
* Proficiency in SQL (any flavor), with experience using Window functions and advanced features.
* Strong knowledge of Python.
* Familiarity with Azure Services such as Blobs, Functions, Azure Data Factory, Service Principal, Containers, Key Vault, etc.
* In-depth knowledge of Snowflake architecture, features, and best practices.
* Experience with CI/CD pipelines using Git and Git Actions.
* Knowledge of various data modeling techniques, including Star Schema, Dimensional models, and Data Vault.
* Hands-on experience with developing data pipelines (Snowflake), writing complex SQL queries, building ETL/ELT/data pipelines, Kubernetes, and Linux containers (e.g., Docker).
* Experience with both relational (RDBMS) and non-relational databases.
* Analytical and problem-solving skills applied to big data datasets.
* Experience working on projects with agile/scrum methodologies and high-performing teams.
* Good understanding of access control, data masking, and row access policies.
* Exposure to DevOps methodology and data engineering.
* Knowledge of data warehousing principles, architecture, and implementation.
* Experience orchestrating data tasks in Airflow to run on Kubernetes for data ingestion, processing, and cleaning.
* Expertise in designing and implementing data pipelines to process high volumes of data.
* Ability to create Docker images for applications to run on Kubernetes.
Click Apply now to be considered for the Senior Data Engineer – Dublin.
Seniority level
* Associate
Employment type
* Contract
Job function
* Consulting
* Industries: Information Services and Technology, Information and Media
#J-18808-Ljbffr