Job Title: Snowflake Data Engineer
About the Role:
We are looking for a highly experienced Snowflake Data Engineer to join our team. As a Snowflake Data Engineer, you will be responsible for designing, building, and testing end-to-end data pipelines, including data ingestion, integration, and curation.
Key Responsibilities:
* Design and develop scalable data pipelines and ETL processes on cloud-based infrastructure using Azure, Snowflake, DBT, Airflow, and Cosmos DB.
* Define and implement automation of jobs and testing.
* Optimize the data pipeline to support workloads and use cases.
* Support mission-critical applications and near real-time data needs from the data platform.
* Address data issues, environment issues, performing impact analysis, root cause analysis, corrective, adaptive, and perfective maintenance.
* Implement data models, transformations, and schema designs to support analytical and reporting needs.
* Optimize and tune Snowflake performance, including query optimization, resource management, and scaling strategies.
Requirements:
* 7+ years of experience in IT, with a strong background in design/data modeling/development & performance tuning.
* Experience with micro-services architecture and understanding of Cloud Computing is highly desirable.
* Strong hands-on experience in troubleshooting DevOps pipelines and Azure services.
* Experience with DBT (Data Build Tool) for data modeling and transformation and Apache Airflow for workflow orchestration and scheduling.
* Expertise in Snowflake utilities, including SnowSQL, Snowpipe, Snowlight for handling streaming data.
* Experience in re-clustering Snowflake data with a good understanding of Micro-Partition within Snowflake.
* Expertise in deploying Snowflake features, such as data sharing, events, and lake-house patterns.
* Experience in handling semi-structured data (JSON, XML) in Snowflake.
* Bachelor's Degree level qualification in a computer or IT-related subject.
* 7+ years of development experience writing SQL queries/Stored Procedures for any relational databases like Oracle, SQL Server.
* 6+ years of experience on Snowflake development, including Snowpipes, Snowshare, Task, Streams, UDFs, and Procedures.
* 6+ years of experience on cloud-based development, including Azure Services, Azure DevOps, Kubernetes, Docker.
What We Value:
* Ability to communicate effectively and professionally, both written and orally.
* Team player with a positive attitude, enthusiasm, initiative, and self-motivation.
* Ability to multi-task, meet aggressive timelines, and have strong work ethics.
* Desire to work in a client-facing environment.
About Us:
We are State Street, one of the largest custodian banks, asset managers, and asset intelligence companies in the world. We provide investment servicing, data & analytics, investment research & trading, and investment management to institutional clients. We value diversity, inclusion, and social responsibility, and offer a collaborative environment where technology skills and innovation are valued.