What Is a Recurrent Neural Network?

A recurrent neural network is a type of artificial intelligence (AI) algorithm that can process a sequence of events and make a prediction about what could happen in the future. This technology works because recurrent neural networks have hidden layers within their architecture that allow the algorithm to gain a sense of time, a working memory where the algorithm can remember what happened in past iterations. This ability makes recurrent neural networks useful for natural language processing, speech and audio recognition, predictive time series analytics, and more. 

Explore how recurrent neural networks function, how you can use them, and what careers you can have in the field of deep learning with recurrent neural networks. 

What is a recurrent neural network?

This deep learning AI model can process sequential data by remembering values it learned in the past and comparing those values to the current input. Many tasks in artificial intelligence require a computer to understand the sequential order of events. Language, for example, follows patterns where words appear in a specific order. If you change the order of the words, you can inadvertently change the sentence's meaning. Likewise, if you wanted to understand the movements of the stock market, it would be essential to understand how time changes the value of variables. A list of stock prices is more valuable when you have time data attached to it so you can understand how the price rises and falls in time. 

These examples of sequential data require an AI model that can interact with both the current input and past inputs. A recurrent neural network accomplishes this task through hidden recurrent layers that empower the algorithm with a sort of working memory. 

How do recurrent neural networks work?

Recurrent neural networks have a unique architecture that allows them additional functionality compared to other types of neural networks. Under other types of neural networks, such as a feed-forward neural network, data moves in a linear pattern from the input to the output. In a recurrent neural network, data can loop back through layers, where the algorithm can store data in a hidden state (like how you might temporarily store data in your memory). The algorithm can use this stored data to compare the next input data, observing how the value has changed and predicting what might come next in the pattern. 

What is a recurrent neural network used for?

You can use recurrent neural networks in a range of applications, including natural language processing like sentiment analysis, language modeling or generating text, speech recognition, audio and signal processing, predictive analytics using time series data, and music generation. 

Natural language processing

Recurrent neural networks can be used for natural language processing, a type of AI that helps computers comprehend and interpret natural human languages like English, Mandarin, or Arabic. They are capable of language modeling, generating text in natural languages, machine translation, and sentiment analysis, or observing the emotions behind written text. 

Speech recognition and audio processing

A recurrent neural network can use natural language processing to understand verbal and audio text and speech in addition to written text. This technology powers artificial intelligence that can respond to verbal commands, such as a virtual assistant device that you can ask a question or command with your voice. Language follows sequential patterns, which allows a recurrent neural network to make sense of those patterns and replicate them. 

Predictive analytics with time series data

Recurrent neural networks are especially adept at managing time series data, which makes them a good tool for predicting the future values of a variable. For example, you could use predictive analytics with time series data to estimate the movements of financial markets, changes in the weather and physical environment, or to forecast the demand for products and services at variable times of the year or in response to world events. You can also use time series data for signal processing or modeling and analyzing data you receive from signals, such as telephone communication, radio frequencies, or medical imaging. 

Music generation

Another example of sequential data that a recurrent network can understand is music. Music follows predictable patterns that recurrent neural networks can analyze and predict what the future beats of the music will sound like. This allows recurrent neural networks to generate novel pieces of music by following learned patterns. 

Types of recurrent neural networks

You can find many more specific types of recurrent neural networks under that umbrella term. The first way you can have differing types of recurrent neural networks is through the number of inputs and outputs they can process simultaneously. To contrast a recurrent neural network with a typical feedforward network, a feedforward neural network can process one input and return one corresponding output. 

A recurrent neural network, on the other hand, can manage many different arrangements, including: 

• One-to-one: One input returns one output. For example, the model classifies a sentence as a sentence. 

• One to many: One input returns many outputs. For example, the model classifies a sentence as a series of words. 

• Many to one: Many inputs return a single output. For example, the model classifies a social media post of 50 words as “enthusiastic” or “dissatisfied.” 

• Many to many: Many inputs return many outputs. For example, the model translates a sentence comprising many words in one natural language into the correct words in the correct order of another language. 

You can also use specialized RNNs to overcome specific problems commonly occurring with recurrent neural networks. These include long short-term memory networks, gated recurrent unit networks, and encoder/decoder networks. 

• Long Short-Term Memory (LSTM) networks: An LSTM network helps overcome the problems of vanishing or exploding gradients, which is a way of saying that the changes made at each layer of the recurrent network either become so small as to not make a difference in the output or so large that they return a wildly inaccurate output. An LSTM overcomes that problem by allowing a recurrent neural network to draw on a more powerful working memory, helping it to learn long-term patterns.

• Gated Recurrent Unit (GRU) networks: A GRU unit works similarly to an LSTM network but uses a more efficient architecture and fewer parameters. This allows the model to work more efficiently and require less training and also empowers the algorithm to be more selective in what values it remembers and what values it forgets. 

• Encoder/decoder networks: An encoder/decoder recurrent neural network can analyze an input sequence and return an output sequence. This type of network is useful for machine translation because it can analyze the pattern of a sentence in one language and return the pattern of the same sentence translated into another language. 

Who uses recurrent neural networks?

As you can see from the many different applications of recurrent neural networks, this technology is relevant to a variety of professionals. If you want to consider a career working with recurrent neural networks, three possibilities to consider are data scientist, machine learning engineer, and artificial intelligence researcher. 

Data scientist

Average annual base salary in the US (Glassdoor): $118,901 [1]

Job outlook (projected growth from 2023 to 2033): 36 percent [2]

As a data scientist, you will be responsible for collecting, cleaning, storing, and analyzing data. You will determine the best sources for the data you need and ultimately present your findings to other stakeholders in the organization.

Machine learning engineer

Average annual base salary in the US (Glassdoor): $123,573 [3]

Job outlook (projected growth from 2023 to 2033): 26 percent [4]

As a machine learning engineer, you will use machine learning and artificial intelligence to solve problems, working with clients or other professionals to design and build new machine learning systems or to optimize and improve existing systems. In this role, you may work directly on programming, testing and troubleshooting issues, and communicating with clients and other stakeholders. 

AI researchers in NLP and computer vision

Average annual base salary in the US (Glassdoor): $99,927 [5]

Job outlook (projected growth from 2023 to 2033): 26 percent [4]

As an artificial intelligence researcher, you will use AI models and algorithms to solve real-world problems. You can choose to specialize in projects like natural language processing or computer vision if you want to work specifically with recurrent and similar types of neural networks. Working in this position, you’ll apply the scientific method to create and train new AI algorithms. 

Learn more about recurrent neural networks on Coursera

Recurrent neural networks are a type of deep learning used for natural language processing, speech recognition, and time series data. 

If you want to learn more about recurrent neural networks or start a career where you can work with them, consider an online program on Coursera to begin your education. For example, you might consider IBM’s AI Foundations for Everyone Specialization, a four-course series that requires little or no familiarity with AI and can help you gain a deeper understanding of AI, including its applications and benefits. You can also opt to go deeper into machine learning with the Machine Learning Specialization from Stanford and DeepLearning.AI.