Responsibilities:
- Design, build, and maintain scalable and efficient data pipelines using Airflow and Python
- Implement and maintain data storage solutions on AWS, such as S3 and Redshift
- Develop and maintain data infrastructure, including ETL processes, data warehousing, and data lakes
- Collaborate with data scientists to ensure data is clean, accurate, and readily available for analysis
- Optimize data retrieval and storage processes to ensure high performance and reliability
- Monitor and troubleshoot data pipelines and infrastructure issues
- Work with stakeholders to understand data requirements and implement solutions that meet their needs
- Stay up-to-date with the latest technologies and trends in data engineering
Requirements:
- Bachelor's or Master's degree in Computer Science, Engineering, or related field
- 2 to 5 years of experience in data engineering, with a focus on designing and implementing data pipelines and infrastructure
- Strong skills in Airflow and Python, with experience in data manipulation, data integration, and data analysis
- Experience with AWS services such as S3, Redshift, EMR, and Lambda
- Experience with data warehousing and data modeling concepts
- Strong problem-solving skills and attention to detail
- Excellent communication and collaboration skills
- AWS Certification will be plus