Job Description
The Data Engineer will design, develop, document, and implement end-to-end data solutions applying best engineering practices, testing, deployment, and providing production support.
Key Responsibilities:
* Apply industry-leading analytics approaches and tools to transform data into insights.
* Deliver high-quality, timely, cost-effective, and maintainable software solutions in an agile environment with a diverse tech stack to meet functional and non-functional business requirements.
* Understand and develop highly scalable distributed systems in public cloud.
* Develop and drive best engineering practices and data strategies as per the needs of the domain.
* Partner with data practitioners and other stakeholders to address complex data challenges and troubleshoot data-related issues effectively.
* B ueild and maintain robust ETL pipelines to integrate data from multiple sources into OLAP data stores, ensuring data integrity and consistency.
* Bring an innovative spirit in search of efficiencies, process improvement opportunities, technical improvements, and other ways to add value to the organization with focus on operations improvement.
Requirements:
* 8+ years of relevant experience in business intelligence, analytics, or data process improvement.
* Bachelor's degree (e.g., Computer Science, Engineering, Statistics) / Master's degree preferred.
* Relational database management expertise (Oracle, Snowflake, PostgreSQL).
* Extensive hands-on experience with ETL/ELT tools (Informatica, Nifi, Snap logic).
* Expertise in Data Analysis, Data Profiling, and Data Modelling skills.
* Knowledge of Data Warehousing methodologies and concepts.
* Extensive experience with ANSI SQL, database stored procedures, and performance tuning.
* Proficiency in Snowflake DB using JavaScript, Python, and snowSQL; Snowflake features (Snow pipe, Data Sharing, etc.) and stay updated with the latest Snowflake features and best practices.
* Proficiency with python for data movement/transformation including development of classes and object-oriented code.
* Complex batch cycle orchestration (tools like Control-M, Autosys or Crontab).
* Implement best practices in data security, role-based access control, and data masking to maintain compliance and data governance standards.
* Knowledge of Cloud platforms and Services (AWS IAM, EC2, S3, Lambda, RDS).
* Knowledge of data streaming tools like Kafka, Kinesis.
* Design experience of scalable data models, optimized for data ingestion and analytics requirements including SCD.
* Developing and automating deployment with GIT, Jenkins, and CICD Processes.
* Experience with REST APIs and in-memory technologies.
Desired Skills:
* Knowledge of non-RDBMS databases like Graph, No-SQL, Timeseries databases.
* Working knowledge of analytics and BI front-end tools like Power BI, Tableau, and ThoughtSpot.