What Is Linear Algebra?

Linear algebra combines multivariate calculus, differential equations, and probability into a widely applicable mathematical theory and system that undergirds many technologies in our lives. At its core, linear algebra studies vectors and linear functions to solve systems of linear equations that contain multiple variables. It focuses on calculating vectors, which are points in space with magnitude and direction, and matrices, which are tables of numbers.

While linear algebra is not a prerequisite for starting in machine learning, its concepts become foundational for a deeper understanding of the algorithms used in machine learning. It can help when you encounter deep learning concepts so that you can make informed decisions when developing a new machine learning system.

This article takes a deeper look at concepts in linear algebra, its uses, users, how it's employed in machine learning, and how you can study it.

Concepts in linear algebra

Linear algebra contains many applicable concepts. Here are some of the concepts in linear algebra that are also useful in machine learning:

• Vectors and vector spaces

• Systems of linear equations

• Matrices

• Eigenvalues and Eigenvectors

Let’s take a closer look at each concept. 

Vectors and vector spaces

A vector is a quantity that has magnitude and direction. In linear algebra, you can add multiple vectors by following the vector addition and multiplication rules. Linear algebra deals with many kinds of vectors:

• Power series

• Polynomials

• Numbers

• n-vectors

• Functions with a specific domain

• 2nd order polynomials

A vector space is a set that adds and multiplies the properties of a vector. Vector spaces have 10 axioms that vectors follow when adding and multiplying. 

Systems of linear equations

Linear algebra examines linear functions, which have vectors as both the input and the output. A matrix is an example of a linear function when it is multiplied by transforming a vector into another vector. A system of linear equations changes to a matrix equation through an augmented matrix in a process called Gaussian elimination. To solve a system of linear equations, you use reduced row echelon form, which simplifies the system of equations into a matrix, getting the linear equation into a solvable form. 

Matrices

A matrix is the result of a system that organizes linear functions. It’s important to know that linear algebra uses matrices as a notation within linear algebra, but the essence of linear algebra is linear functions, not matrices. In matrices, you learn the notation, how they work with linear operators, their properties, inverse matrices, and how to work with various matrices within linear systems. 

Eigenvalues and eigenvectors

In linear algebra, you learn about eigenvalues and eigenvectors. One of the most important equations in linear algebra is the eigenvalue to eigenvector equation. Eigenvalues and eigenvectors are components of a decomposed matrix. These components of the decomposed matrix allow for a more straightforward analysis of complex matrices. They provide helpful methods for decomposing matrices for machine learning. 

What is linear algebra used for?

Linear algebra has a wide variety of applications in applied and abstract mathematics. It studies linear systems like rotations within a given space and systems of differential equations. Many disciplines, such as chemistry, physics, economics, and engineering, use linear algebra; however, linear algebra is an essential branch of mathematics in data science and machine learning.

It has particular applications in global positioning systems (GPS), analyzing the voltages and current in electrical circuits, Markov chains and statistical models, decoding and encoding messages in cryptography, and generating 3D computer graphics. 

Who uses linear algebra?

People who work in a range of industries use linear algebra, such as:

• Engineering

• Computer science

• Mathematics

• Physics

• Biology

• Economics

• Statistics

Let’s look closely at some careers and their relationship to linear algebra.

Economists

Average annual US salary (Glassdoor): $114,817 [1]

Economists use linear algebra when analyzing macroeconomic and economic policy theories using the input-output model to find the interrelationship between financial industries.

Aerospace engineers

Average annual US salary (Glassdoor): $123,495 [2]

Aerospace engineers use linear algebra when designing space shuttle control systems using the properties of multiplying functions by scalars in vector spaces to monitor and maintain stable flight.

Ecologists

Average annual US salary (Glassdoor): $76,194 [3]

Ecologists use linear algebra to calculate how much space extinct-threatened species need as their land dwindles through analyzing dynamic systems using eigenvalues and eigenvectors.

Linear algebra in machine learning

Linear algebra encompasses many processes that occur in machine learning, even if the computer calculates the mathematics. A basic understanding of linear algebra notation, operations in linear algebra, and how matrices decompose gives you a deeper understanding of how your algorithms work and what they're doing. 

However, going deeper still gives you a foundational understanding of the calculations occurring in machine learning. Additionally, knowing these foundations gives you a deeper intuition of how algorithms work so that you can understand more algorithms and create your own. An aspect of machine learning that depends on linear algebra is the creation of deep learning in sentiment analysis, natural language processing (NLP), and computer vision by turning data into vectorized for analysis by a neural network.

Machine learning engineers have an average annual salary of $126,722 [4].

How to study linear algebra

If you’re already enrolled at university, take courses in linear algebra, take notes, read the textbook, attend lectures, and get help from professors when needed. To learn linear algebra independently, try some of these ways to gain skills:

• Take online introductory courses.

• Read linear algebra textbooks. 

• Take a course on Coursera, such as Essential Linear Algebra for Data Science, from the University of Colorado Boulder.

Getting started with Coursera

Are you ready to take your linear algebra skills to the next level for machine learning? Try Imperial College London's Mathematics for Machine Learning Specialization to gain skills and understand how linear algebra applies to machine learning and data science.