Practical Data Engineering Pipelines, Processing and Platforms (SF)
About This Course
This comprehensive 5-day course introduces learners to the practical skills and tools needed to become a successful data engineer.
Participants will explore core data engineering concepts including data ingestion, data modeling, ETL pipelines, orchestration, and distributed data processing using Python, SQL, Apache Airflow, and Apache Spark.
Each day features hands-on projects that reflect real-world data workflows, from cleaning raw CSVs to building scalable batch pipelines and storing output in optimized formats like Parquet. The course culminates with a capstone project to reinforce end-to-end understanding and paves the way for further exploration into big data and AI-enabled analytics.
What You'll Learn
LO2: Design, orchestrate, and optimise data pipelines for high-performance and large-scale processing.
LO3: Integrate end-to-end data engineering practices, assess pipeline effectiveness, and chart the path to GenAI analytics.
Entry Requirements
The candidate should hold a diploma and have at least 2 years of experience working as a developer/programmer.
Candidates who do not meet the above requirements will be assessed on case to case basis.