What Is Programming? And How To Get Started
January 28, 2025
Article
Instructor: Packt - Course Instructors
Included with
Recommended experience
Intermediate level
This course teaches Apache Spark & Scala skills for big data processing & real-time analytics. Prior programming or scripting knowledge is needed.
Recommended experience
Intermediate level
This course teaches Apache Spark & Scala skills for big data processing & real-time analytics. Prior programming or scripting knowledge is needed.
Design and implement advanced Spark applications tailored to complex data processing needs.
Develop and execute Spark scripts to process large datasets and stream real-time data.
Compare and optimize Spark algorithms for better performance in big data applications.
Assess machine learning models' effectiveness using Spark MLlib for accurate predictions.
Add to your LinkedIn profile
October 2024
4 assignments
Add this credential to your LinkedIn profile, resume, or CV
Share it on social media and in your performance review
Embark on a journey to master big data processing with Apache Spark and Scala. This course begins with setting up your development environment, ensuring you have a solid foundation in both Spark and Scala. You will dive into a Scala crash course that covers syntax, flow control, functions, and data structures, giving you the essential skills needed to work with Spark.
Next, you will explore Spark's core concept, the Resilient Distributed Dataset (RDD). Through a series of hands-on activities and exercises, you will learn to manipulate RDDs, implement key/value operations, and perform complex data transformations. The course then transitions into SparkSQL, DataFrames, and DataSets, where you will practice querying structured data efficiently. You'll also tackle advanced Spark programming, where you’ll apply algorithms to real-world datasets, work with clusters, and optimize performance. As you progress, you will delve into machine learning with Spark MLlib and explore how to build recommendation systems, perform regression analysis, and implement decision trees. Finally, the course introduces Spark Streaming and GraphX, allowing you to process real-time data streams and graph-based data efficiently. By the end of this course, you will have the expertise to leverage Spark and Scala for complex data processing tasks in any industry. This course is designed for software engineers who want to expand their skills into the world of big data processing on a cluster. It is necessary to have some prior programming or scripting knowledge.
In this module, we will focus on installing the necessary tools like IntelliJ and Scala on your local system, followed by a brief introduction to the Apache Spark framework and its key concepts.
2 videos1 reading
In this module, we will dive into Scala programming, covering its unique syntax, control flow, and key data structures like Map and List. You’ll get plenty of practice to ensure you’re comfortable with the language before jumping into Spark.
4 videos
In this module, we will break down the structure and usage of RDDs, the building blocks of Spark applications. You’ll work on real-world examples, including building histograms and analyzing social network data.
13 videos1 assignment
In this module, we will introduce SparkSQL, DataFrames, and Datasets, which provide a higher-level abstraction for working with structured data. You’ll also compare and contrast the use of RDDs with these APIs.
9 videos
In this module, we will tackle advanced Spark programming examples, such as finding popular movies and superheroes in a social graph. We will also cover optimization techniques using broadcast variables and accumulators.
11 videos
In this module, we will move from running Spark on a local desktop to deploying and scaling it on a real cluster using Amazon Elastic MapReduce (EMR). You’ll learn how to submit jobs, manage dependencies, and troubleshoot issues.
9 videos1 assignment
In this module, we will explore Spark MLlib for machine learning. You’ll work through examples like generating movie recommendations and performing linear regression on large datasets.
6 videos
In this module, we will focus on real-time data processing using Spark Streaming. You’ll build streaming applications and learn to handle data in motion, both with DStreams and the newer Structured Streaming API.
6 videos
In this module, we will cover Spark’s GraphX library for graph-parallel processing, using it to explore social network data and understand how connected entities are through complex algorithms like breadth-first search.
3 videos1 assignment
In this final module, we will provide additional learning resources and give you tips on leveraging your new skills to advance your career in the field of big data.
1 video1 assignment
Packt helps tech professionals put software to work by distilling and sharing the working knowledge of their peers. Packt is an established global technical learning content provider, founded in Birmingham, UK, with over twenty years of experience delivering premium, rich content from groundbreaking authors on a wide range of emerging and popular technologies.
University of California San Diego
Course
Course
Johns Hopkins University
Course
École Polytechnique Fédérale de Lausanne
Specialization
Unlimited access to 10,000+ world-class courses, hands-on projects, and job-ready certificate programs - all included in your subscription
Earn a degree from world-class universities - 100% online
Upskill your employees to excel in the digital economy
Yes, you can preview the first video and view the syllabus before you enroll. You must purchase the course to access content not included in the preview.
If you decide to enroll in the course before the session start date, you will have access to all of the lecture videos and readings for the course. You’ll be able to submit assignments once the session starts.
Once you enroll and your session begins, you will have access to all videos and other resources, including reading items and the course discussion forum. You’ll be able to view and submit practice assessments, and complete required graded assignments to earn a grade and a Course Certificate.
If you complete the course successfully, your electronic Course Certificate will be added to your Accomplishments page - from there, you can print your Course Certificate or add it to your LinkedIn profile.
This course is one of a few offered on Coursera that are currently available only to learners who have paid or received financial aid, when available.
You will be eligible for a full refund until two weeks after your payment date, or (for courses that have just launched) until two weeks after the first session of the course begins, whichever is later. You cannot receive a refund once you’ve earned a Course Certificate, even if you complete the course within the two-week refund period. See our full refund policy.
Yes. In select learning programs, you can apply for financial aid or a scholarship if you can’t afford the enrollment fee. If fin aid or scholarship is available for your learning program selection, you’ll find a link to apply on the description page.
These cookies are necessary for the website to function and cannot be switched off in our systems. They are usually only set in response to actions made by you which amount to a request for services, such as setting your privacy preferences, logging in or filling in forms. You can set your browser to block or alert you about these cookies, but some parts of the site will not then work.
These cookies may be set through our site by our advertising partners. They may be used by those companies to build a profile of your interests and show you relevant adverts on other sites. They are based on uniquely identifying your browser and internet device. If you do not allow these cookies, you will experience less targeted advertising.
These cookies allow us to count visits and traffic sources so we can measure and improve the performance of our site. They help us to know which pages are the most and least popular and see how visitors move around the site. If you do not allow these cookies we will not know when you have visited our site, and will not be able to monitor its performance.
These cookies enable the website to provide enhanced functionality and personalization. They may be set by us or by third party providers whose services we have added to our pages. If you do not allow these cookies then some or all of these services may not function properly.