Implement data engineering solutions using Azure Databricks
(4 days)
Course Description
Master end-to-end data engineering with Azure Databricks and Unity Catalog. This course moves from foundational setup to production deployment, covering environment configuration and enterprise-grade governance. Learn to build robust ingestion pipelines, implement security with Unity Catalog, and deploy optimized workloads. By the end, you will have the practical skills to implement, secure, and maintain scalable lakehouse solutions that meet rigorous enterprise requirements.Learning Objectives
-
Configure and manage Azure Databricks environments and compute resources
-
Implement data governance and security using Unity Catalog
-
Design efficient data models for lakehouse architectures
-
Build scalable batch and streaming data ingestion pipelines
-
Transform, validate, and load data into analytics-ready formats
-
Deploy, monitor, and optimize production-grade data pipelines
Who Should Attend
The target audience is data engineers who have fundamental knowledge of data analytics concepts, a basic understanding of cloud storage, and familiarity with data organization principles. They should be comfortable working with SQL and have experience using Python, including notebooks, for data engineering tasks. Learners are expected to have a good understanding of Azure Databricks workspaces and Unity Catalog, along with familiarity with data access patterns and core data engineering and data warehouse concepts. In addition, they should have foundational knowledge of Azure security, including Microsoft Entra ID, and be familiar with Git version control fundamentals.
Prerequisites
- Fundamental knowledge of data analytics concepts
- Basic understanding of cloud storage concepts
- Familiarity with SQL and data organization principles
- Good understanding of Azure Databricks workspaces and Unity Catalog concepts
- Familiarity with SQL and data access patterns
- Knowledge of Microsoft Entra ID and Azure security fundamentals
- Knowledge of fundamental data engineering and data warehouse concepts
Course Outline
Module 1: Set up and configure an Azure Databricks environment
Explore Azure Databricks
Exercise - Explore Azure Databricks
Understand Azure Databricks Architecture
Understand Azure Databricks Integrations
Select and Configure Compute in Azure Databricks
Exercise - Select and Configure Compute
Create and Organize Objects in Unity Catalog
Exercise - Create and Organize Objects
Module 2: Secure and govern Unity Catalog objects in Azure Databricks
Secure Unity Catalog Objects
Exercise - Secure Unity Catalog Objects
Govern Unity Catalog Objects
Exercise - Govern Unity Catalog Objects
Module 3: Prepare and process data with Azure Databricks
Design and Implement Data Modeling with Azure Databricks
Exercise - Design and Implement Data Modeling
Ingest Data into Unity Catalog
Exercise - Ingest Data
Cleanse, Transform, and Load Data into Unity Catalog
Exercise - Cleanse, Transform, and Load Data
Implement and Manage Data Quality Constraints with Azure Databricks
Exercise - Implement Data Quality Constraints
Module 4: Deploy and maintain data pipelines and workloads with Azure Databricks
Design and Implement Data Pipelines with Azure Databricks
Exercise - Design and Implement Data Pipelines
Implement Lakeflow Jobs with Azure Databricks
Exercise - Implement Lakeflow Jobs
Implement Development Lifecycle Processes in Azure Databricks
Exercise - Implement Development Lifecycle Processes
Monitor, Troubleshoot, and Optimize Workloads in Azure Databricks
Exercise - Monitor, Troubleshoot, and Optimize Workloads
