Data Engineering on AWS
About This Course
This intermediate-level 3-day course equips data professionals with the skills to design, build, optimize and secure data engineering solutions on AWS. Through a mix of theory, demonstrations, hands-on labs and group exercises, participants progress from foundational data engineering concepts to hands-on implementation of data lakes, data warehouses, and batch and streaming data pipelines.
Participants work extensively with AWS services including Lake Formation, Amazon Redshift Serverless, AWS Step Functions and Amazon Managed Service for Apache Flink, applying orchestration, security, monitoring, CI/CD and cost-optimization practices throughout, reflecting how modern data solutions are architected and managed at scale in enterprise environments.
What You'll Learn
• LO1 - Explain data engineering roles, key concepts and data personas, and identify relevant AWS data services for a data engineering solution.
• LO2 - Identify and explain the AWS tools and services for orchestration, security, monitoring, CI/CD, IaC, networking and cost optimization in a data engineering architecture.
• LO3 - Outline the design, implementation and optimization of a secure data lake solution on AWS, including storage, ingestion, cataloguing, transformation, open table formats and Lake Formation permissions.
• LO4 - Design, set up, optimize and secure a data warehouse using Amazon Redshift Serverless, including performance tuning, orchestration, authentication and access control.
• LO5 - Design, implement, optimize, orchestrate and secure batch data pipelines on AWS, covering processing, transformation, integration and cataloguing.
• LO6 - Architect, optimize and secure streaming data pipelines on AWS, including ingestion, storage, processing, analysis and compliance considerations.
Entry Requirements
• Familiarity with basic machine learning concepts, such as supervised and unsupervised learning, regression, classification, and clustering algorithms.
• Working knowledge of Python programming language and common data science libraries like NumPy, Pandas, and Scikit-learn.
• Basic understanding of cloud computing concepts and familiarity with the AWS platform.
• Familiarity with SQL and relational databases is recommended but not mandatory.
• Experience with version control systems like Git is beneficial but not required.