Position: Software Engineer
Department: Data Engineering & Analytics
Employment Type: Full-time
Role Overview
We are seeking a highly skilled and motivated Software Engineer with strong expertise in data engineering and analytics. The ideal candidate will have hands-on experience with modern data lake architectures on Azure or AWS platforms, with a particular focus on the Azure stack. This role will involve designing, developing, and optimizing data pipelines, enabling advanced analytics, and leveraging scalable big data solutions.
Key Responsibilities
* Design, build, and maintain robust, scalable, and efficient data pipelines for ETL/ELT processes.
* Develop and manage data lake architectures using Azure or AWS, ensuring high availability, performance, and security.
* Implement and optimize data solutions using Databricks, Azure Data Factory, Delta Lake, and other modern tools.
* Collaborate with analytics and data science teams to enable access to curated datasets and insights.
* Implement and manage NoSQL databases and big data processing frameworks like Apache Spark.
* Develop and document best practices for ETL patterns, data modeling, and governance.
* Monitor, troubleshoot, and improve the performance of data workflows and systems.
* Stay updated on emerging data engineering technologies and contribute to the adoption of innovative solutions.
Required Skills & Experience
* Proven experience in data engineering and analytics roles, with a focus on cloud-based platforms.
* Expertise in working with Azure or AWS ecosystems. Strong preference for Azure, including:
o Azure Data Factory
o Azure Databricks
o Delta Lake
o Azure Synapse Analytics (optional but beneficial)
* Proficiency in big data technologies like Apache Spark and Delta Lake.
* Strong experience with NoSQL databases (e.g., CosmosDB, MongoDB, DynamoDB).
* Solid understanding of ETL/ELT patterns and hands-on implementation experience.
* Proficiency in programming languages such as Python, Scala, or SQL for data manipulation and pipeline development.
* Familiarity with modern data governance and security practices.
* Excellent problem-solving skills and attention to detail.
Preferred Qualifications
* Experience with Azure Data Explorer or Stream Analytics.
* Familiarity with DevOps practices, including CI/CD pipelines for data workflows.
* Knowledge of data modeling and data warehouse design principles.
* Hands-on experience with real-time data processing and event-driven architectures.
#J-18808-Ljbffr