Back to ETL and Data Pipelines with Shell, Airflow and Kafka

Learner Reviews & Feedback for ETL and Data Pipelines with Shell, Airflow and Kafka by IBM

4.5

stars

335 ratings

About the Course

Delve into the two different approaches to converting raw data into analytics-ready data. One approach is the Extract, Transform, Load (ETL) process. The other contrasting approach is the Extract, Load, and Transform (ELT) process. ETL processes apply to data warehouses and data marts. ELT processes apply to data lakes, where the data is transformed on demand by the requesting/calling application. In this course, you will learn about the different tools and techniques that are used with ETL and Data pipelines. Both ETL and ELT extract data from source systems, move the data through the data pipeline, and store the data in destination systems. During this course, you will experience how ELT and ETL processing differ and identify use cases for both. You will identify methods and tools used for extracting the data, merging extracted data either logically or physically, and for loading data into data repositories. You will also define transformations to apply to source data to make the data credible, contextual, and accessible to data users. You will be able to outline some of the multiple methods for loading data into the destination system, verifying data quality, monitoring load failures, and the use of recovery mechanisms in case of failure. By the end of this course, you will also know how to use Apache Airflow to build data pipelines as well be knowledgeable about the advantages of using this approach. You will also learn how to use Apache Kafka to build streaming pipelines as well as the core components of Kafka which include: brokers, topics, partitions, replications, producers, and consumers. Finally, you will complete a shareable final project that enables you to demonstrate the skills you acquired in each module....

Top reviews

Invalid date

It's one of the most challenging courses I've been enrolled!

Invalid date

Overall it's a good course. I wish I could use dos2unix, tr, or sed for removing ^M from the toll_data.tsv. The Final Assignment Instructions could have been clearer.

Filter by:

26 - 50 of 84 Reviews for ETL and Data Pipelines with Shell, Airflow and Kafka

By David A S

•

Jun 14, 2022

Excellent introduction to this topics. Labs contain all you need to know how to start using this type of technologies. Highly recommended.

By Mohamed A

•

Jun 10, 2022

Thanks to all the instructor's efforts, one of the best DATA engineering courses, contains hands-on Experience with essential data tools.

By Jibran

•

Jul 23, 2023

Labs in this course are very helpful and to the point. It took me a while to complete this course but i learned a lot.

By Darrick L

•

Sep 7, 2022

Very useful high-level overview with practical examples of the major technologies that drive modern data pipelines.

By Theodosios T

•

Jan 5, 2023

The explanation was very thorough and easy to understand. The exercises were very helpful. Great course overall!

By Uchechi N

•

Dec 31, 2022

This was my first introduction to Apache airflow and i found the course detailed and practical.

By k b

•

Apr 24, 2022

Nice intro to ETL and Data Pipelines. Beginner level easy to follow hands on Airflow and Kafka.

By Reinaldo R

•

Aug 28, 2024

Muy satisfecho con el contenido del curso, y los laboratorios. Thank you very much!

By Younes G

•

Oct 26, 2022

Perfect Intro to Airflow and Kafka !

The Final Project is very fun and instructive

By Олег К

•

Jan 26, 2024

It was not easy to pass this course. Lots of laboratory work. But I liked it

By Tea K

•

Jul 23, 2023

Great hands-on (alike the others in the pack)! Practical and interactive.

By Takahiko A

•

Aug 20, 2024

個人的にはData Warehouse Engineerの目玉となる講義でした。Pythonを使うことで、画期的に楽しくなりますね。

By Rorisang S

•

Mar 14, 2022

Succinctly presented. Labs really hammered the point home :)

By Maxime T

•

Jun 23, 2024

Great content but the labs can be challenging to work with.

By Asanka W

•

Sep 8, 2022

Great Course, Assignments prepare really well and flexible

By Monica D

•

Mar 11, 2023

Great learning course for Kafka/ Airflow, well presented

By SARADA N G

•

Jul 13, 2023

Learn a lot about Apache Airflow, Kafka from sketch.

By Joseph O

•

Jun 11, 2024

That Kafka unit was soo mindboggling but worth it

By Arnold J M

•

Oct 1, 2022

Great course with hands on labs for practices.

By WONG L X

•

Aug 20, 2022

great course but airflow does not work on

By Minh N T

•

Apr 12, 2022

Useful course for beginner Data engineer

By Diwas A

•

Jan 25, 2024

Good course for intro level ETL !

By Muhammad T K

•

Jul 9, 2022

Absolutely brilliant for starter

By Ryan A M

•

Jul 8, 2023

excelent course, thank you ibm.

By Albin C

•

Oct 9, 2022

I it was a very good course!