In this course, you will explore various types of source systems, learn how they generate and update data, and troubleshoot common issues you might encounter when trying to connect to these systems in the real world. You’ll dive into the details of common ingestion patterns and implement batch and streaming pipelines. You’ll automate and orchestrate your data pipelines using infrastructure as code and pipelines as code tools. You’ll also explore AWS and open source tools for monitoring your data systems and data quality.
Source Systems, Data Ingestion, and Pipelines
Save on skills that make you shine with 40% off 3 months of Coursera Plus. Save now
Source Systems, Data Ingestion, and Pipelines
This course is part of DeepLearning.AI Data Engineering Professional Certificate
Instructors: Joe Reis +1 more
Top Instructor
13,183 already enrolled
Gain insight into a topic and learn the fundamentals.
136 reviews
Intermediate level
Recommended experience
Flexible schedule
3 weeks at 10 hours a week
Learn at your own pace
92%
Most learners liked this course
What you'll learn
Gather stakeholder needs and translate them into system requirements.
Implement a batch and a streaming ingestion process on AWS to ingest data from various source systems.
Integrate aspects of security, data management, DataOps and orchestration into the data systems you build.
Skills you'll gain
- Category: IT Automation
- Category: Infrastructure as Code (IaC)
- Category: Network Troubleshooting
- Category: Data Transformation
- Category: Data Pipelines
- Category: Data Integration
- Category: Data Processing
- Category: CI/CD
- Category: Data Quality
- Category: Extract, Transform, Load
Tools you'll learn
- Category: Amazon CloudWatch
- Category: Apache Airflow
- Category: Amazon Web Services
- Category: NoSQL
- Category: Terraform
Details to know
Shareable certificate
Add to your LinkedIn profile
Assessments
4 assignments
Taught in English
Build your Cloud Computing expertise
This course is part of the DeepLearning.AI Data Engineering Professional Certificate
When you enroll in this course, you'll also be enrolled in this Professional Certificate.
- Learn new concepts from industry experts
- Gain a foundational understanding of a subject or tool
- Develop job-relevant skills with hands-on projects
- Earn a shareable career certificate from DeepLearning.AI
There are 4 modules in this course
Earn a career certificate
Add this credential to your LinkedIn profile, resume, or CV. Share it on social media and in your performance review.
Instructors
Instructor ratings
(37 ratings)Top Instructor
Joe Reis
DeepLearning.AI
4 Courses46,503 learners
Offered by
DeepLearning.AI
Amazon Web Services
Why people choose Coursera for their career
Felipe M.
Learner since 2018
"To be able to take courses at my own pace and rhythm has been an amazing experience. I can learn whenever it fits my schedule and mood."
Jennifer J.
Learner since 2020
"I directly applied the concepts and skills I learned from my courses to an exciting new project at work."
Larry W.
Learner since 2021
"When I need courses on topics that my university doesn't offer, Coursera is one of the best places to go."
Chaitanya A.
"Learning isn't just about being better at your job: it's so much more than that. Coursera allows me to learn without limits."
Learner reviews
- 5 stars
88.97%
- 4 stars
4.41%
- 3 stars
0%
- 2 stars
4.41%
- 1 star
2.20%
Showing 3 of 136
TS
Reviewed on Nov 23, 2024
Really valuable, and I got an idea of data-related concepts and infrastructure management.
MK
Reviewed on Jan 18, 2025
great learning and exercises. appreciate the community.
DD
Reviewed on Mar 23, 2025
best course and best projects to practice for data engineering