Scalable Data Solutions: Big Data Engineering Essentials
About This Course
This course covers how to build data pipelines and systems for
collecting, storing and analysing data at scale. By the end of the module,
learners should have a solid understanding of data extraction,
transformation and loading pipelines, data warehouse, data architecture
and design, and distributed data processing and analytics.
What You'll Learn
- Explain the fundamental concepts and terminology of big data and data engineering.
- Build a data pipeline to extract data from public sources via Application Programming Interface (and/or web scraping),
- Load the data into a cloud data warehouse (or data lake)
- Transform the data into schemas suitable for data analysis and
machine learning.
- Orchestrate and schedule the data pipeline to run on a regular basis.
Entry Requirements
A background in IT is not necessary but preferable. This course is designed to cater to the intricacies of data science, data engineering and AI. Hence, individuals with a background in these disciplines bring valuable prior knowledge that will enhance their learning experience and understanding of the curriculum.
Applicants will be required to undergo a shortlisting process where they must share their résumé and complete a pre-course assessment. Our instructors and career consultants will review applicants to assess their suitability before deciding whether to shortlist them for the course.