What Does a Data Management Specialist Do?
October 1, 2024
Article
(12 reviews)
Recommended experience
Intermediate level
Data Engineer, BI Engineer
(12 reviews)
Recommended experience
Intermediate level
Data Engineer, BI Engineer
Implement SCD Type 1 in Azure Data Factory
Add to your LinkedIn profile
Only available on desktop
In this project, you will learn how to implement one of the most common concept in real world projects i.e. Slowly Changing Dimension Type 1, using Azure Data Factory.
Pre-requisites: Azure subscription Azure Data Factory knowledge (Basic) Following are the tasks covered in this project: Task 1: Understand Slowly Changing Dimension (SCD) Type 1 In this task, we will try to understand the concept of Slowly Changing Dimension and its different types, but will focus on Type 1 using a simple example. Task 2: Create Azure services like Azure Data Factory, Azure SQL Database In this task, we are going to create the azure services like azure data factory and azure sql database which are going to be used in later tasks. Azure sql database is going to contain the staging and dimension table whereas azure data factory is going to be used to create the data pipeline Task 3: Create Staging and Dimension Table in Azure SQL Database In this task, we will create the staging and dimension table in azure sql database. Also, we will insert some dummy records in staging table Task 4: Create a ADF pipeline to implement SCD Type 1 (Insert Logic) In this task, we are going to create the pipeline in azure data factory and implement the logic to insert new records which exists in staging table but doesnt exist in dimension. This is one scenario/use case of SCD Type 1. Task 5: Create a ADF pipeline to implement SCD Type 1 (Update Logic) In this task, we are going to create the pipeline in azure data factory and implement the logic to update records which exists in staging table as well as in dimension. This is another use case/scenario of SCD Type 1 Task 6: Demo of ADF pipeline This is final task in which we will run the pipeline to see whether it satisfies both the use case/scenario of SCD Type 1 All the Best !!
In a video that plays in a split-screen with your work area, your instructor will walk you through these steps:
Understand Slowly Changing Dimension (SCD) Type 1
Create Azure services like Azure Data Factory, Azure SQL Database
Create Staging and Dimension Table in Azure SQL Database
Create a ADF pipeline to implement SCD Type 1 (Insert Logic)
Create a ADF pipeline to implement SCD Type 1 (Update Logic)
Demo of ADF pipeline
Data Engineer, BI Engineer
The Coursera Project Network is a select group of instructors who have demonstrated expertise in specific tools or skills through their industry experience or academic backgrounds in the topics of their projects. If you're interested in becoming a project instructor and creating Guided Projects to help millions of learners around the world, please apply today at teach.coursera.org.
Skill-based, hands-on learning
Practice new skills by completing job-related tasks.
Expert guidance
Follow along with pre-recorded videos from experts using a unique side-by-side interface.
No downloads or installation required
Access the tools and resources you need in a pre-configured cloud workspace.
Available only on desktop
This Guided Project is designed for laptops or desktop computers with a reliable Internet connection, not mobile devices.
12 reviews
58.33%
25%
8.33%
0%
8.33%
Showing 3 of 12
Reviewed on May 14, 2023
Excellent example of doing a Data Flow in Data Factory
Coursera Project Network
Course
Coursera Project Network
Course
Course
Coursera Project Network
Course
Unlimited access to 10,000+ world-class courses, hands-on projects, and job-ready certificate programs - all included in your subscription
Earn a degree from world-class universities - 100% online
Upskill your employees to excel in the digital economy
By purchasing a Guided Project, you'll get everything you need to complete the Guided Project including access to a cloud desktop workspace through your web browser that contains the files and software you need to get started, plus step-by-step video instruction from a subject matter expert.
Because your workspace contains a cloud desktop that is sized for a laptop or desktop computer, Guided Projects are not available on your mobile device.
Guided Project instructors are subject matter experts who have experience in the skill, tool or domain of their project and are passionate about sharing their knowledge to impact millions of learners around the world.
You can download and keep any of your created files from the Guided Project. To do so, you can use the “File Browser” feature while you are accessing your cloud desktop.
Guided Projects are not eligible for refunds. See our full refund policy.
Financial aid is not available for Guided Projects.
Auditing is not available for Guided Projects.
At the top of the page, you can press on the experience level for this Guided Project to view any knowledge prerequisites. For every level of Guided Project, your instructor will walk you through step-by-step.
Yes, everything you need to complete your Guided Project will be available in a cloud desktop that is available in your browser.
You'll learn by doing through completing tasks in a split-screen environment directly in your browser. On the left side of the screen, you'll complete the task in your workspace. On the right side of the screen, you'll watch an instructor walk you through the project, step-by-step.