Spark is a powerhouse of innovation, a champion among firms, blazing a trail for SMEs and enterprise businesses. Our secret weapon? Data. With our unmatched expertise, we bring unparalleled value to our customers, propelling them to new heights of success.
At Spark, we don't settle for the ordinary. We're on a mission to revolutionise the business landscape, one data-driven solution at a time. From inception to execution, we provide comprehensive end-to-end services encompassing strategy, design, development, and implementation, relentlessly pursuing excellence, constantly optimising operations, boosting revenue, and reducing costs.
This unwavering dedication ensures our customers stay ahead of the competition, achieving extraordinary results that redefine what's possible.
But here's the thing—we can't do it alone. We need a team of brilliant minds, trailblazers, and visionaries to join our ranks, eager to make a difference and deliver on our promises to our customers. So, if you're ready to embark on an exciting journey where every day is an opportunity to create something remarkable, look no further.
Spark is calling.
We are looking for a highly skilled and motivated AWS Data Engineer to join our dynamic team as a contractor.
The ideal candidate will have extensive experience in building and maintaining data pipelines, data warehouses, and data lakes using AWS services. You will work closely with our data science and analytics teams to ensure data is readily available, secure, and efficiently processed.
Specifically, we see your time is spent:
1. Design, develop, and maintain scalable data pipelines and ETL processes using AWS Glue, Glue Crawlers, and Glue Data Catalog.
2. Manage and optimise Redshift clusters, ensuring high availability and performance of data warehousing solutions.
3. Implement serverless data processing workflows using AWS Lambda.
4. Utilise Apache Airflow for orchestrating complex data workflows and ensure smooth and timely execution of data pipelines.
5. Implement and manage data versioning and schema evolution using Apache Hudi.
6. Ensure data security and compliance through robust IAM policies and AWS Lake Formation.
7. Collaborate with data scientists, analysts, and stakeholders to understand data needs and deliver high-quality data solutions.
8. Monitor and troubleshoot data pipelines and workflows to ensure data quality and system reliability.
9. Document data workflows, architectures, and best practices to ensure knowledge sharing and continuous improvement.
What will pique our interest is someone that might have:
1. Bachelor’s degree in Computer Science, Information Technology, or a related field.
2. Minimum of 1 year experience using DBT.
3. 3+ years of experience as a Data Engineer with a focus on AWS technologies.
4. Strong expertise in AWS Glue, Glue Crawlers, and Glue Data Catalog.
5. Proficiency in Amazon Redshift including performance tuning, cluster management, and data warehousing best practices.
6. Experience with AWS Lambda for building serverless data processing applications.
7. Hands-on experience with Apache Airflow for orchestrating ETL workflows.
8. Familiarity with Apache Hudi for managing large-scale data lakes.
9. In-depth knowledge of IAM for managing access and permissions in AWS.
10. Experience with AWS Lake Formation for managing secure data lakes.
11. Strong programming skills in Python, SQL, and other relevant languages.
12. Excellent problem-solving skills and the ability to troubleshoot complex data issues.
13. Strong communication and collaboration skills to work effectively with cross-functional teams.
How we can support you:
* We embrace a hybrid approach that lets you work where you're most productive while staying connected with a collaborative and innovative team. With a strong remote-friendly culture, we ensure you have the support, tools, and flexibility to thrive. Whether you're collaborating virtually or joining team meetups, you'll experience the networking, camaraderie, and dynamic environment that make our company an exciting place to grow—all while maintaining a healthy work-life balance.
#J-18808-Ljbffr