Chevron Left
Back to Improving Deep Neural Networks: Hyperparameter Tuning, Regularization and Optimization

Learner Reviews & Feedback for Improving Deep Neural Networks: Hyperparameter Tuning, Regularization and Optimization by DeepLearning.AI

4.9
stars
63,068 ratings

About the Course

In the second course of the Deep Learning Specialization, you will open the deep learning black box to understand the processes that drive performance and generate good results systematically. By the end, you will learn the best practices to train and develop test sets and analyze bias/variance for building deep learning applications; be able to use standard neural network techniques such as initialization, L2 and dropout regularization, hyperparameter tuning, batch normalization, and gradient checking; implement and apply a variety of optimization algorithms, such as mini-batch gradient descent, Momentum, RMSprop and Adam, and check for their convergence; and implement a neural network in TensorFlow. The Deep Learning Specialization is our foundational program that will help you understand the capabilities, challenges, and consequences of deep learning and prepare you to participate in the development of leading-edge AI technology. It provides a pathway for you to gain the knowledge and skills to apply machine learning to your work, level up your technical career, and take the definitive step in the world of AI....

Top reviews

YL

Invalid date

very useful course, especially the last tensorflow assignment. the only reason i gave 4 stars is due to the lack of practice on batchnorm, which i believe is one of the most usefule techniques lately.

NC

Invalid date

Yet another excellent course by Professor Ng! Really helped me gain a detailed understanding of optimization techniques such as RMSprop and Adam, as well as the inner workings of batch normalization.

Filter by:

6026 - 6050 of 7,238 Reviews for Improving Deep Neural Networks: Hyperparameter Tuning, Regularization and Optimization

By Harish K

•

Nov 25, 2017

G

By Sanguk P

•

Oct 27, 2017

w

By Ed M

•

Sep 20, 2017

E

By StudyExchange

•

Aug 20, 2017

V

By D. R

•

Oct 1, 2019

(09/2019)

Overall the courses in the specialization are great and provide great introduction to these topics, as well as practical experience. Many topics are explained clearly, with valuable field practitioners insight, and you are given quizzes and code-exercises that help deepen the understanding of how to implement the concepts in the videos. I would recommend to take them after the initial Andrew Ng ML course by Stanford, unless you have prior background in this topic.

There are a few shortbacks:

1 - the video editing is poor and sloppy. Its not too bad, but it’s sometimes can be a bit annoying.

2 - most of the exercises are too easy, and are almost copy-paste. I need to go over them and create variations of them in-order to strengthen my practical skills. Some exercises are quite challenging though (especially in course 4 and 5), and I need to go over them just to really nail them down, as things scale up quickly. Course 3 has no exercises as its more theoretical. Some exercises have bugs - so make sure to look at the discussion board for tips (the final exercise has a huge bug that was super annoying).

3 - there are no summary readings - you have to (re)watch the videos in order to check something, which is annoying. This is partially solved because the exercises themselves usually hold a lot of (textual) summary, with equations.

4 - the 3rd course was a bit less interesting in my opinion, but I did learn some stuff from it. So in the end it’s worth it.

5 - Slide graphics and Andrew handwriting could be improved.

6 - the online Coursera Jupyter notebook environment was a bit slow, and sometimes get stuck.

Again overall - highly recommended

By Rameses

•

Nov 4, 2019

I have taken Machine Learning courses earlier from Andrew Ng via Coursera. I have always felt that the delivery of the material and the pedagogy are superb and have always rated a 5 star as also for the first course in this specialization. This second course had several interesting topics I had never learned in my previous NN courses at universities. The programming exercises for weeks 1 and 2 were excellent in helping recap the material in the videos and slides. However as far as TensorFlow is concerned, I was a bit disappointed because it seemed like we were muddling through the various code snippets rather than getting a firm grasp of what is obviously a very complex Deep Learning programming framework. But I understand the time limitations and I realize that this intro to TensorFlow is merely to whet one's appetite and encourage us to explore more about this framework as well as other frameworks. I believe it is up to each individual to explore the concepts further and get a better understanding.

The technology behind the courses is awesome as well as the programming assignment notebooks which were well documented and must have taken gargantuan amount of time and effort in prepping.

In summary, I learned a lot from this course and while my course objectives were not fulfilled, almost all of them were.

By Alexander W

•

Oct 24, 2021

First for the worst: All discussion for this course has been moved out of coursera into Discord. I consider Discord as overloaded and confusing to navigate, especially in comparison to the neat and clear Coursera discussion groups. It requires an additional registration. Quite annoying for me, as I use Coursera mostly with the mobile app, where opening Discord involves an extra step and switching apps back and forth. No more support, help or assistance is given here on Coursera. Guys, please reconsider this.

It seems this course is a little out of it's creators focus. Corrections are still made, but on separate pages instead of editing the video, which would make it so much easier. The assignments are still maintained and updated regularly.

Apart from this, this course was great! Quality and explanations are excellent, many examples, (not too easy) exercises and assignments to practice what you learned. I would recommend it to someone with at least some contact with machine learnung and Python.

By Alberto G

•

Nov 16, 2018

One more brilliant course by Professor Andrew Ng. It covered all important topics in model training and testing and hyperparameter tuning, ending with an introduction to TensorFlow. But even though Professor Ng stressed that the choice for a Deep Learning framework should consider the belief that the company supporting it has a long term commitment to keep it open sourced, he doesn't explain why he believes that Google will do so with TensorFlow. Even though I've been dealing with TF for a couple of years now, having complete both AI and Deep Learning Nanodegrees at Udacity, I still struggle to understand its cryptic innings. I hoped that Professor Ng could bridge that gap, but unfortunately that was not the case. Maybe in later versions of this course he can explain a little better how exactly TensorFlow works - we know it does, but it is very frustrating to depend upon tons of Stackoverflow queries to get syntax right and build even simple models. Thanks God he uses Keras in later modules.

By Christopher T

•

Jan 6, 2018

I gave this less than perfect stars, just for a few technical issues. It really does deserve 5 stars, and I hope I can come back and update my review once the issues are addressed.

There seem to be a few bugs in the final TF project, but using the forums it was possible to figure out what I did wrong. For me, I was computing the cost function using the untransposed forward prop and target vectors. I'm actually glad that this issue came up, as it gave me a better idea of where difficult to find issues can appear.

Despite not completing the last project successfully, I was able to score 100% on submission. Hopefully they can improve on this mis-scoring issue.

After reflecting a bit, the TF material seems a bit tacked on. Perhaps it deserves its own space? There are certainly a great deal of concepts that can be covered on this modern library.

Otherwise, it's an excellent course. Just a few rough edges to hammer out.

By Sebastiaan v E

•

Nov 17, 2017

Very good course in terms of material and explanation.

The autograder for grading the problems is not very good:

- it fails a lot due to "technical difficulties",

- it sometimes fails even though the solution is correct, e.g., a = a + b succeeds but a += b fails,

- It sometimes passes even though the solution is incorrect (I used gradient descent instead of Adam, but it still passed with a 100% score).

There were also quite some typos and other errors in the last assignment, so it seems like it was made in a bit of a hurry. The forums list all the corrections, so it should be easy to fix this.

It would also be really nice to have an option to reset the notebook to its initial state. Now if you accidentally mess it up or delete it, it's gone. My recommendation to students is to make a copy of the notebook before you start working on it.

The automatically generated captions also contain lots of errors.

By José D

•

Sep 7, 2019

This is Course2 of the Deep Learning Specialization. In Course1, we learned how to code the algorithm in Numpy. Most of Course2 show how to optimize and tune the algorithm and how to use and tune the hyper-parameters. Most assignments are well-designed and easy to perform as they focus more on the understanding than "finding how to code it". However, the last assignment introduces TensorFlow where we re-implement the algorithm using TensorFlow concepts. I have to say I expected TensorFlow to simplify things but it turns out I find the Math/Numpy implementation way easier to understand than TensorFlow. I'll have to dig deeper in TensorFlow concepts to understand it better. I would have liked more TensorFlow introduction. I hope the following courses will go into deeper details. Nevertheless, great course and very instructive.

By Jörg N

•

Oct 19, 2019

I liked the course a lot and I really adore the way Andrew Ng teaches the subject. As an improvement suggestion I would extend the course to four weeks to deepen the practice on Hyperparameter tuning as well as the introduction to Tensorflow. The Programming exercises of week 3 were really challenging. First since there were partially misleading statements in the comments (Z before activation) and second because variables were given the same names as tf parameters and partially even function definitions. So you could see things like a = a, b = b in tf function calls which just does not fit for beginners in portentously both Python (local variables concepts, etc.) and TF. I am more than grateful though that I could do this course of the specialisation and I would really like to express my deep gratitude to Andrew Ng.

By Mark B

•

Mar 9, 2018

The material is consistently good, but this is now my third class and I am acclimated to it and self-selecting. Still, I'm not offering 5 stars because this is a Coursera AI course, and I am hypersensitive to the less-than-intelligent interactions that I have with the courseware: I was warned last Sunday about falling behind on my coursework, for example, for a week that I started in the morning but which happened to end that evening. So, the machine-based interaction isn't that great. The discussion lists and mentor interactions are quite good and very helpful. Still, I'm puzzled by the slowness in fixing typos in assignments, these waste our time; I have found at least three of them, all previously documented. So, I am spending time hunting down course errors when I should be hunting down my own.

By Sertan Åž

•

Sep 13, 2017

This course has made me learn/revise many of fundamentals of neural network and ML, in general. I really find the explanations and the conveyed intuitions wonderful. I think the videos will be a major reference for me (and millions) in the future.

Having said that I think that the third week should be divided into two weeks in the future. I think it would have been invaluable for many people to actually implement the material taught (e.g. batch normalization, one-hot encoding, softmax classification) with synthetic data on the third week. Tensorflow should be introduced in the fourth week. I think the two last videos explaining the frameworks and the programming assignment are adequate enough as - I believe - the hyperparameter optimization will be dealt in the third course of specialization.

By Imran S

•

May 8, 2020

I usually zip through Coursera courses. This really demotivated in a good way. I came into this thinking I'll finish it fast only to find that the course is anything but easy. I learned a lot and I'm still not confident in saying that I have absolute 100% grasp of the content despite completing the course. If anything, being able to complete this course was a motivational boost for me despite it taking much longer than I had initially anticipated.

The content is so broad that I will have to go through the content again and read from external sources as well. However, I'm quite happy and humbled with what I have learnt. It really puts into perspective the complexities of the mathematics behind deep learning and gives you a new found appreciation for framework libraries like tensorflow.

By Upamanyu S

•

May 25, 2020

The course was good and the explanations were intuitive - the last notebook, however, could have been better in my opinion. It was too straightforward in the final implementation step - the mini-batch assignments should have been left to the user at least. I understand that it's not practical to include the 5 minute runtime model as a graded function, but maybe splitting it into multiple graded functions for the setup and execution would be better in ensuring the student had absorbed the class material. And one-hot encoding was not given enough attention in terms of what the axis is doing. If I could give this a 4.8 out of 5 I would, but I can't, and I'd love to see this class that has taught me this much grow so I'll give it a 4 in hopes that someone will read this. Thanks.

By Dave J

•

Feb 25, 2020

Good course focusing on the practical aspects of regularization, normalization, initialization, optimization and hyperparameter tuning. Also looks at multi-class classification using Softmax and introduces machine learning frameworks, focusing on TensorFlow.

Well taught by Andrew Ng. Programming exercises work smoothly and give plenty of hand-holding, making them suitable for machine learning beginners with a little Python experience, though short on challenge for those with more experience.

Coursera need to put more resources into responding to issues with the course. I found a mistake in a formula and saw that it had been reported 3 months ago on the Errata forum with no response. I emailed the suggested feedback address and have had no reply. Hence dropping a star.

By Oleg P

•

Nov 25, 2018

This was a good course with several drawbacks. Although we learned theoretical aspects of batch normalization, I needed to go through a dozen of resources to learn how to actually implement it (especially backpropagation part). An assignment for that would be helpful. The last week of this course (Tensorflow) should have become its own course with multiply programming assignments. What I strongly dislike is the design of programming assignments, which are very long but the user implementation part very short - it's easy to loose an overview. Personally, I finish them as quick as possible to get the grade and then implement the functionality by myself. I'd also support the idea of optional assignments, where students can solve some problems from scratch.

By Volodymyr M

•

Apr 18, 2020

I am more, or less satisfied with this part of the course. It gives valuable information in many practical aspects. What is not good - graded assignments notebooks contain errors. For me, having about 30 years of experience in various technical fields, it was like an interesting game to find source of error and make graded assignment running, while some other people may just get stuck.

Also, I wish we run graded assignments on computers with a little bit more power. It is very unproductive to wait tens of seconds while Python kernel evaluates the cell.

BTW, for me with week 3 graded assignment first 10 epochs took about 10 minutes. Simple math gives 2.5 hours for 1500 epochs. Yes, we are paying not that much for this course, but we are paying for it.

By Joseph C

•

Mar 30, 2018

This was another fantastic course in the deep learning specialization. We are introduced to techniques designed to speed up and improve neural network models, such as minibatch gradient descent, regularization, and normalization techniques. Hands-on programming exercises show how to implement some of these techniques. Suggestions for improvement: (1) Third week is too dense with ideas. It should provide programming exercises to impart practical understanding of these ideas. A 4th week should be devoted to an in-depth introduction to TensorFlow. (2) Improve the Forum by including regular participation by the teaching staff. Mentors are active on the Forum, but there is virtually NO participation by the teaching staff!

By Yuri G

•

May 12, 2018

It's a very useful course but it comes across as the creators have put it out and completely abandoned it: the contact email given at the beginning is a black hole (could have used no_reply), volunteer mentors are supporting the forums but well-known multiple bugs and mistakes are not being fixed. Programming assignments are designed to make an impression that students can reach farther than they actually can on their own by often turning them into a copy-and-paste of small pieces into the code already mostly written for them. One would expect that if one even needs to use this "knowledge" from programming assignments for work later it would be free and readily available. Wrong assumption.

By Mehran M

•

Jun 7, 2018

I really really liked both this course and the previous one. In fact, I gave 5 stars to the first course. However, compared to the first course, this one earns 4 stars in my book. For one, I wish there was more material on TensorFlow. Also it would've been nice to do a detailed assignment involving actual hyperparameter tuning. Even though the title of the course promises techniques about hyperparameter tuning, I'm not confident that I will be able to do a systematically good job, even though I just finished this course and have carefully taken notes of all the presented materials. Nevertheless, this is an awesome course and I wholeheartedly recommend it.

By Brian R

•

Dec 20, 2018

This course is a gem in that its chock-a-block full of real-world/in-practice information that so many courses don't cover. One note is that some of the programming assignment notebooks in this course (especially the TF one) were several notches below the finely tuned level of the prior course's. There were typos in the explanation sections, syntax errors in blocks so that you couldn't do partial submissions, and other things that didn't create a full-stop but definitely upped the frustration factor. I made it through and have learned a MASSIVE amount, but there is definitely some polishing that would have made this a 5 star that wasn't there yet.

By Enrique C M

•

Oct 18, 2017

Very good course about more critical concepts when building deep neural networks. Although the material seems like quite condensed and forced in a very short span of time (3 weeks) while the easier and more basic concepts in course 1 where explained at the right pace during 4 weeks.

These are core concepts and techniques for practical day-by-day deep learning engineering and programming and I would have wanted them to not to be taught in such a rush.

Even though, highly recommended course... I am already building real networks to solve real problems in some projects I am involved in and that is simply awesome after 2 courses of this specialization :)))

By Ravish C

•

Oct 2, 2017

The content was very well taught and I learnt a lot of new concepts I was not familiar with before. I liked that the projects had us implement the basic concepts like Adam optimizer and Dropout on a neural network from scratch, instead of with tensorflow because it helped me understand how they work essentially.

That said, it felt that the course was less polished as the first one. There were some errors in the assignments that made them difficult to pass, the assignments themselves were also very easy and did not take as long to complete (at most I spent 30 minutes on any assignment, instead of the written 2 hours).