What Is a Hidden Layer in a Neural Network?

The hidden layers of neural networks are the brains behind artificial intelligence. Let’s imagine a neural network: For example, imagine you’re using a program on your phone to identify a plant. You are out walking one day when you see an interesting specimen, so you snap a picture and upload it to the app. After a second or two, the program provides you with a guess about what kind of plant you might be looking at.

From your perspective, you provided an input (the image) and received an output (the plant's name). However, a complex process occurs to make this happen. Inside the neural network, operating between the input and output, lies hidden layers that enable the neural network to function. It may seem like a simple process to you, the end user, but the data you put into the algorithm can pass through hundreds of layers of neurons, depending on the depth of the neural network.

The number, types, and architecture of hidden layers inside a neural network are wildly different based on what function the neural network performs. Discover more about the hidden layer of neural networks with this article, including convolutional, recurrent, and generative adversarial neural networks. 

What is a hidden layer in a neural network?

The three layers of neural networks are the input layer, the hidden layer, and the output layer. As you may guess by the name, the hidden layers aren’t something you interact with directly when using a neural network. On the other hand, you can control what goes into the algorithm (the input) and access what comes out of the algorithm (the output). Every layer in between those two is a part of the hidden layers of a neural network.

If you tried to use a neural network without a hidden layer, your output would be nothing more than a repeat of your input. As the input transfers from one hidden layer to the next, the neural network learns the patterns associated with the input, providing the network—artificial intelligence (AI)—with a hierarchical representation of the input data. Eventually, this process transforms the data into the output.

Although a simple neural network might contain one hidden layer, including additional layers allows the network to interact with the input in more complex ways. Each hidden layer can look at a different aspect of the input or recognize more specific features layer by layer, compounding this information from one layer to the next. This technology of stacking hidden layers inside a neural network is the basis for deep learning. Essentially, for an algorithm to be considered “deep learning,” it must contain more than three layers: the input, one hidden layer, and the output. The more layers a neural network contains, the “deeper” the learning algorithm.

Why do neural networks need more than one hidden layer?

In short, neural networks don’t require more than one hidden layer. Computer scientists and artificial intelligence engineers have been using multilayer neural networks since the invention of the multilayer perceptron in 1958.

Adding additional layers allows the neural networks to make more complex calculations or interact with the data more sophisticatedly. Stacking large numbers of hidden layers between the input and the output allows your neural network to perform deep-learning tasks regarding the input. The number of layers in each neural network depends on the goals you wish to accomplish with the algorithm, but deep learning isn’t possible with a single hidden layer between the input and the output layers.

Hidden layers in neural network architecture

We’ve identified that hidden layers of neural networks are everything between the input and output layers. We also discussed how additional layers make the artificial intelligence “deeper.” But what exactly do hidden layers do within the context of neural network architecture?

Every hidden layer inside a neural network contains neurons known as nodes. Each node interacts with the data in one simple way. For example, maybe it completes a calculation, detects the top-right corner of an image, or identifies that an item is the color red. Each node in each layer connects to each node in the next layer, passing down what it discovered about the data. Data flows through the input and then enters the first layer of nodes, which changes or reacts to the data and passes the new information set to the next hidden layer.

Depending on your goals for the neural network, many layers of highly specialized nodes give the AI the ability to understand complex relationships between data. Some of the most common types of neural networks are recurrent, generative adversarial, and convolutional. Let’s take a closer look at the job of the hidden layers inside each of these neural networks. 

Hidden layers of recurrent neural networks

A recurrent neural network works specifically with time series data or sequential data. For example, recurrent neural networks can assist you in making predictions about future events based on past data or in replicating human speech by understanding how the word order in a sentence affects the meaning of the sentence. The hidden layers in a recurrent neural network contain the same nodes described above but have the ability to remember the results of previous calculations. Using those stored “memories,” the neural network can then predict the current input based on past results.

These stored memories are in a hidden state within the hidden layers. The hidden state carries a weight, and the input also carries a weight. In other words, the hidden layer of a recurrent neural network considers both the input on its own and the hidden state of past inputs and makes a new prediction using the experience and learning of the past. 

Hidden layers of convolutional neural networks

A convolutional neural network (CNN) is well-equipped for classification or computer vision tasks. For example, a convolutional neural network can read medical imaging such as X-rays or MRIs to screen you for anomalies faster and more accurately than medical professionals.

Convolutional neural networks have three main types of hidden layers: convolutional layers, pooling layers, and FC, or fully connected layers.

• Convolutional layers: Convolutional layers use feature extraction to perform the main “work” of the CNN. A feature detector, or kernel, scans the input and creates a map of the features it finds. A convolutional neural network can have many convolutional layers, with each layer adding a more nuanced understanding of the input.

• Pooling layers: The pooling layer of a convolutional neural network simplifies the work of the convolutional layers, losing data in the process but gaining a more efficient and less complex output. The convolutional layer focuses on extracting all the features, while the pooling layer provides you with a less computationally challenging data set.

• Fully-connected layers: The fully-connected (FC) layer of the CNN is so named because every node in this layer connects to every node in the previous layer. Essentially, the CNN process is threefold: Once the convolutional layer extracts features from the data, this information passes to the pooling layer, where it is downsized; it then transfers to the FC layer, which maps the information gained from the previous two layers, classifying it for output.

Hidden layers of generative adversarial neural networks 

A generative adversarial network (GAN) is a tool that utilizes a competitive methodology for generating text or images. Within the hidden layers of a GAN network live two distinct neural networks: the generator and the discriminator. The two artificial minds are playing a game against one another to generate the output.

Both neural networks can access the training data or the data set you select to represent what the AI “knows.” The generator attempts to create a fake item so convincing that the discriminator can’t differentiate between the forgery and the real thing. The discriminator attempts to select each forgery.

The output is created when the generator “wins” the game, and the generator accomplishes this by creating a unique item so similar to those in the training set that the discriminator can’t tell it’s a fake. The algorithm can improve and learn over time because the nature of the competition drives both teams to improve and innovate to gain an edge over their competitor.

The hidden layers of a generative adversarial network also use backpropagation, a method of looping the input back through the network again before moving on toward the output. In this way, the generator and the discriminator can play their game repeatedly until the generator succeeds in tricking the discriminator, providing you with a novel output.

The exact makeup of the hidden layers in a generative adversarial network depends on the exact model of the artificial intelligence. For example, a deep convolutional generative adversarial network (DCGAN) uses convolutional layers to accomplish the same goal slightly differently. 

Learn more with Coursera. 

If you’d like to learn more about the inner workings of neural networks, consider Neural Networks and Deep Learning, a course offered by Deep Learning.AI on Coursera. This intermediate-level course, which is part of the Deep Learning Specialization, takes approximately 24 hours to complete and can help you learn skills in deep learning, artificial neural networks, backpropagation, and more.