How Do Neural Networks Work? Your 2024 Guide

Neural networks are a foundational deep learning and artificial intelligence (AI) element. Sometimes called artificial neural networks (ANNs), they aim to function similarly to how the human brain processes information and learns. Neural networks form the foundation of deep learning, a type of machine learning that uses deep neural networks.

Artificial neural networks were first introduced in the early 1940s when Warren McCulloch and Walter Pitts studied how neurons work in the human brain, creating a simple binary device to show their findings. However, it wasn’t until computer hardware developments in the 1980s and later in the 2010s that the ANNs in deep learning you see today were possible. Research continues on ANNs because they are vital to developing better AI.

How do neural networks work? Continue reading to learn the answer to that question and get details about how ANNs function, their structure, how they’re trained, and how they work with AI.

What are ANNs?

Artificial neural networks are computational processing systems containing many simple processing units called nodes that interact to perform tasks. Each node in the neural network focuses on one aspect of the problem, interacting like human neurons by each sharing their findings.

Unlike computational algorithms, in which a programmer tells the computer how to process input data, neural networks use input and output data to discover what factors lead to generating the output data. It creates a machine learning algorithm that makes predictions when fed new input data. ANNs train on new data, attempting to make each prediction more accurate by continually training each node.

How do neural networks work?

One way to understand how ANNs work is to examine how neural networks work in the human brain. The history of ANNs comes from biological inspiration and extensive study on how the brain works to process information.

Artificial neural network structure

An individual neuron is a cell with an input and output structure. The input structure of a neuron is formed by dendrites, which receive signals from other nerve cells. The output structure is an axon that branches out from the cell body, connecting to the dendrites of another neuron via a synapse. Neurons communicate using electrochemical signals. Neurons only fire an output signal if the input signal meets a certain threshold in a specified amount of time.

ANNs operate similarly. They receive input signals that reach a threshold using sigmoid functions, process the information, and then generate an output signal. Like human neurons, ANNs receive multiple inputs, add them up, and then process the sum with a sigmoid function. If the sum fed into the sigmoid function produces a value that works, that value becomes the output of the ANN.

This is the structure of an individual neuron in an ANN, but networks have multiple layers and neurons that create the network. The structure of an entire artificial neural network consists of:

• Input layer: takes in the input data and transfers it to the second (hidden) layer of neurons using synapses. An input layer has as many nodes as features or columns of data in the matrix.

• Hidden layer: takes data from the input layer to categorize or detect desired aspects of the data. Nodes in the hidden layer send the data to more hidden layers or, finally, to the output layer. The hidden layer of an ANN is a “black box” because researchers cannot determine its results.

• Output layer: takes data from the hidden layer and outputs the results. It has as many nodes as the model desires.

• Synapses: connect nodes in layers and in between layers.

Deep neural networks, which are used in deep learning, have a similar structure to a basic neural network, except they use multiple hidden layers and require significantly more time and data to train. 

Training a neural network

ANNs require training to produce accurate output values. Training begins with the network processing large data samples with already known outputs. ANNs undergo supervised learning using labeled data sets with known answers. Once the neural network builds a knowledge base, it tries to produce a correct answer from an unknown piece of data.

ANNs use a “weight,” which is the strength of the connection between nodes in the network. During training, ANNs assign a high or low weight, strengthening the signal as the weight between nodes increases. The weight adjusts as it learns through a gradient descent method that calculates an error between the actual value and the predicted value. Throughout training, the error becomes smaller as the weight between connections increases.

Types of neural networks

Neural networks vary in type based on how they process information and how many hidden layers they contain. Three types of neural networks include the following:

• Feed-forward neural networks

• Backpropagation neural networks

• Convolution neural networks

Let’s take a closer look at how each neural network type works. 

Feed-forward neural networks

These neural networks constitute the most basic form of an artificial neural network. They send data in one forward direction from the input node to the output node in the next layer. They do not require hidden layers but sometimes contain them for more complicated processes. They learn over time through feedback processes. Facial recognition is an example of a feed-forward network.

Backpropagation neural networks

Backpropagation neural networks work continuously by having each node remember its output value and run it back through the network to create predictions in each layer. This allows for the network to learn and improve predictions continuously.

These networks create a feedback loop called backpropagation. It starts like a feed-forward ANN, and if an answer is correct, it adds more weight to the pathway. If it is wrong, the network re-attempts the prediction until it becomes closer to the right answer. An example of this ANN is in speech-to-text-to-speech algorithms.

Convolution neural networks

Convolution neural networks use hidden layers to perform mathematical functions to create feature maps of image regions that are easier to classify. Each hidden layer gets a specific portion of the image to break down for further analysis, eventually leading to a prediction of what the image is. An example of convolution neural networks is AI image recognition.

Neural networks and AI

Well-trained, accurate neural networks are a key component of AI because of the speed at which they interact with data. If the ultimate goal of AI is an artificial intelligence of human capabilities, ANNs are an essential step in that process. Understanding how neural networks operate helps you understand how AI works since neural networks are foundational to AI's learning and predictive algorithms. 

Getting started with Coursera

Artificial neural networks are vital to creating AI and deep learning algorithms. Learn more about how neural networks work with online courses. For example, you can gain skills in developing, training, and building neural networks. Consider exploring the Deep Learning Specialization from DeepLearning.AI on Coursera.