A great course/specialization, and the one in reinforcement learning you were looking for. A lot of work has been put into creating this specialization. Maybe a bit less into this last course (capstone) which consists of a patchwork of lectures from previous courses and some new ones. The capstone project is not fundamentally different from assignments in previous courses. Be aware, even if you’ve made it through the whole specialization it most likely doesn’t mean that you will be ready to return to your own area of interest/expertise and implement an RL project from scratch. Still, I would highly recommend taking the full specialization if you meet the programming prerequisites.

I really liked the new videos ("Meeting with...") and the idea of using all the information learned through the other courses to tackle a project. However, this course seems to not be fully thought-through. I didn't love the re-inclusion of videos I had already seen in the past (which were sometimes only tangentially-related to that week's topic). The programming assignments were either way too easy (#1 and #3) or way too difficult/involved/long (#2). The pacing of this course was way off as well, I don't think it should be broken into 6 weeks. I finished the entire thing in about 1 week.

You might, like me, have acquired some understanding on several areas of RL (Q-learning, Policy Gradient...) from available sources (selected papers, articles, blogs, tutorials...), and were waiting for "the right" course to come up, wrapping up all existing and missing bits into one solid foundation.

If that's your case, don't waste any more time or money somewhere else: this course is the course you are needing. It will take you step by step (always) from the basics of bandits to MDP solutions and from tabular algorithms to more sophisticated function approximation algorithms.

And if you're just starting to scratch on this great field... well, I don't think you'll currently find a better online course, and I've seen quite a few.

Thanks for putting this together, Martha and Adam!

This course changed my life! It was so good and I learned so much. I can't believe I'm now an astronaut. Next mission: go to Mars!

Unlike the previous courses in this specialization, this course seems a bit unripe. There's very little material added here (perhaps the only thing new is the Replay Experience algorithm, which is introduced rather briefly). It's more like a general recap of the previous 3 courses. I kind of hoped for something more challenging and broad - but the scope here was rather limited.

Very good course. Compared to the prior courses in the specialization, it appears to be still a course under development rather than a final product. I recommend that the instructors work more on this course (the other courses in the specialization are very very good).

Keep up the great work.

Hello!

As there is no place to review the complete specialization, I'll do it at this place.

I general I think I learned a lot and have now a good foundation of RL.

But on the other side IMHO there is the need for some improvements:

1. Questions get not answered at all: I asked some questions through the course and did not get an answer to one of them.

2. The textbook you provided does not fit to the way I'm learning: much too much text and not enough (mathematical) proves. It would be good to have alternative readings here.

3. Some of the tests in the assignments are not ok: they leave you in the impression that everything is fine - until at some point later there is the need to review everything. This is IMHO a waste of time - a test should test all (or at least) the most possibilities.

4. I'm missing something like course no 5. All the details are now introduced - and now the 'real work' on a architectural and conceptual level can start. But there is nothing like this.

5. It would be good to generally update the python assignments that they work with the lastest versions of their dependencies. The problem is that with the current setup it is mostly impossible to develop and execute the assignments locally. (e.g. gym 0.21 might be updated to the latest gymnasium, in one assignment even the random number generator does not fit, that the local version failed but the online succeeded (IMHO the concept relying on the same random numbers is a bit brittle, might fail and introduced a lot of questions and work))

6. Please use environments like tensorflow for handling NN. As I understood, this is not an introduction to NN - so just use what is there (there is no need to develop (again) an Adam optimizer - it does not really help to understand RL.

Very good lectures - very informative and on point when it comes to theory but lacking in actual application of the theory. However, the projects are TERRIBLE. They could actually be very good, but there is simply not enough information in the descriptions to be very useful. None of the lectures discuss the details of how to implement any of the topics and the projects basically set things up but provide no information on what is actually expected to be done. They need to either discuss the basics or provide pointers to resources that provide that description. Some of the forums are helpful in clarifying things, but the projects really need someone knowledgeable in this area to rework things extensively.

It is clear that a lot of effort has been put in this course. Excellent examples and very clear explanations of the theoretical material. The down side is the programming assignment is too easy, and we didn't actually implement the environment

Good content, but considering the bugginess of graders and the necessity to submit results separately from notebook, this requirement is too extreme: "Retakes: You can attempt this assignment 5 times every 4 months."

The same review for all 4 courses: This is the first time I have done a Coursera module building courses up rather than just individual courses. You really feel you have achieved something out of it. Some people have commented that it is just presenting material from the Sutton and Barto book. But that book is *the* text book in the subject. The course selected particular chapters from the book. I wouldn't have got as much just from trying to read the book on its own (I probably wouldn't have read as much as I did). It was good to have the supplementary videos with other experts - and great to watch Sutton and Barto just sat down being recorded having a retrospective conversation. The programming exercises would sometimes feel they weren't testing much (in fact, the challenges were largely due to my lack of skill in Python - my Python abilities have improved which is a side benefit) but they would actually get you into the weeds as it were. All in all, the best courses I've done. Great job Martha and Adam!

They mostly discuss the importance of real world experience and hyperparameter tuning in this class. The content it did have was solid and the instructors were great. The "capstone" was creating an agent to solve the Moon Lander problem, and much of the code was already written.

I would have really preferred getting experience with a real RL framework like RLLib or acme, rather than the toy libraries used by the book. It would have also been really nice to have a little more freedom and challenge, such as making us actually create an agent to solve an MDP of our own choosing and definition.

I wish that this course contained more practical advice.

The video describing the problem and the notebook describing the problem contradict each other in many details.

Also, the name of the course is a bit misleading. Instead of building "A Complete Reinforcement Learning System" and covering all of the building blocks of an RL project, the homeworks were mainly focused on training the agent, very similar to most previous assignments. So it was more about doing the same one thing in more depth rather than doing more different things.

This training course is better than all other trainings by having programming exercises. The learner implements the algorithms in practice, which helps him / her to learn better.

Excellent final course for the specialization. Moon Lander project was informative and fun.

I liked the project. I hoped it would be harder and enable the students to design the whole agent and environment code and be evaluated with a human grader. But overall, I was able to practice the concepts I have learnt throughout the specialization.

The idea of dedicating a whole course to a practical project was indeed very good. However I think that this idea was not exploited as deeply as it could have been. The project itself is actually a notebook just a little longer than usual. I would have left more to do to students, and maybe they could have used lectures to give more explanations and hints for the practical part, or to do some programming together. Anyway, the course is overall fairly good, and it also introduces some new concepts, like experience replay.

I like the course lectures, and those are great explanation and additional to the Sutton's book. The deduction of the two stars are primarily for the quiz and coding assignments. Most of the time, the quiz is not clear and the coding assignment is confusing too, and not very well structured.

Review videos were tiring for people that had watched previous courses!

Could be more detailed in the environment setup

powerfull in theory but so weak in practice

This is my overall review for all the courses in this specialization. In my opinion, this specialization can be a good supplement to the RL textbook. There are some instances where the lecture video can describe certain content better than the textbook. One should also remember that the depth covered by this specialization is much less compared to the textbook, therefore one is still strongly encouraged to read the textbook thoroughly to have a better understanding of the topics. Other good things to be said about this specialization is that the Jupyter notebook exercises are rather well-prepared. However, the last course (Capstone) was actually surprisingly easy, so although the course estimates 6 weeks worth of content, I feel it's more towards 1-2 weeks (could be even less if you skip the review lessons which are duplicates of videos in previous courses). Throughout the courses there are also guest lecture videos. Most of them are interesting enough, although for some the content may be too far from course content (perhaps even textbook). Overall this specialization is definitely a good place to start learning reinforcement learning!

The capstone project was very well chosen and it was a fascinating problem to solve. The professors explained a complete workflow to conduct towards a scientific experiment in order to solve the problem efficiently. It was good to review some of the concepts and algorithms from the previous courses in the specialization to have a bigger picture of the path we went through. In addition, I had a great time watching the videos with other professors and special guests such as the one with David Silver and Joelle Pineau. Finally, I really appreciate the effort that Mr. and Mrs. White made to make this specialization available in Coursera and to share their knowledge and experience. I believe that I have a good foundation in Reinforcement Learning now and will continue the reading of the remaining chapter of the text book.

Thank you every one (onscreen and offscreen) who built this amazing course. I am a robotics and automation engineer. I learned reinforcement learning from a 20 hour youtube lecture of David Silver from deepmind. Despite that being a great course I joined this course to make my foundation concrete. And to my surprise the presentation of complex concepts in this course was remarkably good. Every ideas were presented in a very simplified manner. Thanks team. This course is highly recommended to anyone, including absolute beginners, wanting to learn reinforcement learning.

If you are getting into RL, I highly recommend going straight into this specialization. This course is an absolute gold and so is the accompanying book - "The Reinforcement Learning" by Sutton and Barto. The problem with sub-concepts in RL are very subtle and looks very similar and there is pretty good chance you end with confusion. This is where this specialization shines - building each concepts incrementally to give you the bigger picture. I am now so much confident with RL and know where each concept/algorithm fits. Thank you so much for this Specialization.