Digital Marketing Salary: How Much Can You Earn in 2025?
February 4, 2025
Article
Recommended experience
Intermediate level
Experience with Apache Spark and API Pyspark. Python coding skills. SQL query proficiency. Big Data concepts. Kafka basics.
Recommended experience
Intermediate level
Experience with Apache Spark and API Pyspark. Python coding skills. SQL query proficiency. Big Data concepts. Kafka basics.
Set up and configure a real-time data processing pipeline
Perform transformations, aggregations, and SQL queries on streaming data
Implement fault-tolerance mechanisms and ensure the pipeline remains resilient under high workloads and data inconsistencies
Add to your LinkedIn profile
January 2025
Only available on desktop
In this hands-on, 1-hour project-based course, you will master real-time data processing using Apache Spark Structured Streaming. This course is designed for data engineers and developers who want to gain practical experience in building streaming data pipelines. You will begin by setting up the Spark environment and learn how to configure micro-batches and fault tolerance mechanisms through checkpointing. Next, you’ll dive into transforming streaming data by applying filters, maps, and aggregations to extract meaningful insights. You'll also handle out-of-order data with watermarks, ensuring the accuracy of your real-time analytics. The course will introduce you to querying streaming data using SQL, allowing you to perform transformations and aggregations on live data. Finally, you will learn to deploy your streaming pipeline to production by writing results to an external sink like Parquet files. This is an intermediate level project and in order to succeed in this course it is recommended to have basic understanding of Apache Spark and API PySpark, proficiency in programming and big data as well and some basic knowledge on writing SQL queries. This is the perfect opportunity for anyone looking to dive into real-time data processing and Spark Structured Streaming!
In a video that plays in a split-screen with your work area, your instructor will walk you through these steps:
Task 1: Setting Up the Environment for Real-Time Data Streaming
Task 2: Managing Triggers and Checkpoints
Task 3: Transforming Streaming Data
Practice Activity
Task 4: Performing Transformations, Aggregations, and Advanced SQL Queries
Task 5: Writing and Deploying the Pipeline
Cumulative Challenge
Experience with Apache Spark and API Pyspark. Python coding skills. SQL query proficiency. Big Data concepts. Kafka basics.
The Coursera Project Network is a select group of instructors who have demonstrated expertise in specific tools or skills through their industry experience or academic backgrounds in the topics of their projects. If you're interested in becoming a project instructor and creating Guided Projects to help millions of learners around the world, please apply today at teach.coursera.org.
Skill-based, hands-on learning
Practice new skills by completing job-related tasks.
Expert guidance
Follow along with pre-recorded videos from experts using a unique side-by-side interface.
No downloads or installation required
Access the tools and resources you need in a pre-configured cloud workspace.
Available only on desktop
This Guided Project is designed for laptops or desktop computers with a reliable Internet connection, not mobile devices.
DeepLearning.AI
Course
Duke University
Specialization
University of California San Diego
Course
Course
Unlimited access to 10,000+ world-class courses, hands-on projects, and job-ready certificate programs - all included in your subscription
Earn a degree from world-class universities - 100% online
Upskill your employees to excel in the digital economy
By purchasing a Guided Project, you'll get everything you need to complete the Guided Project including access to a cloud desktop workspace through your web browser that contains the files and software you need to get started, plus step-by-step video instruction from a subject matter expert.
Because your workspace contains a cloud desktop that is sized for a laptop or desktop computer, Guided Projects are not available on your mobile device.
Guided Project instructors are subject matter experts who have experience in the skill, tool or domain of their project and are passionate about sharing their knowledge to impact millions of learners around the world.
You can download and keep any of your created files from the Guided Project. To do so, you can use the “File Browser” feature while you are accessing your cloud desktop.
Guided Projects are not eligible for refunds. See our full refund policy.
Financial aid is not available for Guided Projects.
Auditing is not available for Guided Projects.
At the top of the page, you can press on the experience level for this Guided Project to view any knowledge prerequisites. For every level of Guided Project, your instructor will walk you through step-by-step.
Yes, everything you need to complete your Guided Project will be available in a cloud desktop that is available in your browser.
You'll learn by doing through completing tasks in a split-screen environment directly in your browser. On the left side of the screen, you'll complete the task in your workspace. On the right side of the screen, you'll watch an instructor walk you through the project, step-by-step.
These cookies are necessary for the website to function and cannot be switched off in our systems. They are usually only set in response to actions made by you which amount to a request for services, such as setting your privacy preferences, logging in or filling in forms. You can set your browser to block or alert you about these cookies, but some parts of the site will not then work.
These cookies may be set through our site by our advertising partners. They may be used by those companies to build a profile of your interests and show you relevant adverts on other sites. They are based on uniquely identifying your browser and internet device. If you do not allow these cookies, you will experience less targeted advertising.
These cookies allow us to count visits and traffic sources so we can measure and improve the performance of our site. They help us to know which pages are the most and least popular and see how visitors move around the site. If you do not allow these cookies we will not know when you have visited our site, and will not be able to monitor its performance.
These cookies enable the website to provide enhanced functionality and personalization. They may be set by us or by third party providers whose services we have added to our pages. If you do not allow these cookies then some or all of these services may not function properly.