Data Engineering for Enterprise
About This Course
The Data Engineering for Enterprise course is designed to equip participants with a thorough understanding of contemporary data engineering principles, architecture, design, and best practices. This programme explores the rapidly changing landscape of data engineering and highlights the essential role that data plays in shaping organisational decision-making and strategic direction.
The course provides participants with a comprehensive understanding of modern data engineering through key areas such as contemporary principles, architectural design, and data modelling for both SQL (Structured Query Language) and NoSQL (Not Only Structured Query Language) systems. The curriculum includes essential pipeline patterns, data quality management, and covers emerging lakehouse architectures. Additionally, learners acquire skills in dynamic data visualisation, all aimed at ensuring the ability to design, implement, and communicate effective data solutions within real-world enterprise contexts.
What You'll Learn
• Modern Data Engineering Principles: Participants will gain insights into the fundamentals and advanced concepts of modern data engineering methodologies
• Architectural Design: The course covers essential architectural frameworks for designing robust, scalable, and secure data systems aligned with enterprise and analytical workloads.
• Data Models: Model data for both SQL (Structured Query Language) and NoSQL (Not Only Structured Query Language) systems to address varied data storage and retrieval needs.
• Pipelines and Patterns:
o Study and construct ETL, ELT and Reverse ETL (Extraction, Transformation and Loading) pipeline patterns, ensuring reliability, fault tolerance, and data quality.
o Implement distributed data processing pipelines using Apache Spark and its DataFrame API (Application Programming Interface).
o Design a simple real-time data streaming pipeline for a given business scenario.
• Data Quality Management: The course emphasises approaches to maintaining and improving the quality of data, a key factor in reliable analytics and reporting.
• Data Warehousing and Lakehouse Architecture: Participants will explore traditional data warehousing solutions alongside lakehouse architectures, understanding their roles in modern data ecosystems.
• Dynamic Data Visualisation: The curriculum includes techniques for creating dynamic and interactive data visualisations, empowering participants to communicate insights effectively.
Entry Requirements
Python and SQL experience, ready to move beyond data analysis and into the world of data engineering