Chevron Left
Back to Improving Deep Neural Networks: Hyperparameter Tuning, Regularization and Optimization

Learner Reviews & Feedback for Improving Deep Neural Networks: Hyperparameter Tuning, Regularization and Optimization by DeepLearning.AI

4.9
stars
63,068 ratings

About the Course

In the second course of the Deep Learning Specialization, you will open the deep learning black box to understand the processes that drive performance and generate good results systematically. By the end, you will learn the best practices to train and develop test sets and analyze bias/variance for building deep learning applications; be able to use standard neural network techniques such as initialization, L2 and dropout regularization, hyperparameter tuning, batch normalization, and gradient checking; implement and apply a variety of optimization algorithms, such as mini-batch gradient descent, Momentum, RMSprop and Adam, and check for their convergence; and implement a neural network in TensorFlow. The Deep Learning Specialization is our foundational program that will help you understand the capabilities, challenges, and consequences of deep learning and prepare you to participate in the development of leading-edge AI technology. It provides a pathway for you to gain the knowledge and skills to apply machine learning to your work, level up your technical career, and take the definitive step in the world of AI....

Top reviews

YL

Invalid date

very useful course, especially the last tensorflow assignment. the only reason i gave 4 stars is due to the lack of practice on batchnorm, which i believe is one of the most usefule techniques lately.

NC

Invalid date

Yet another excellent course by Professor Ng! Really helped me gain a detailed understanding of optimization techniques such as RMSprop and Adam, as well as the inner workings of batch normalization.

Filter by:

6051 - 6075 of 7,238 Reviews for Improving Deep Neural Networks: Hyperparameter Tuning, Regularization and Optimization

By Categorical S

Dec 23, 2018

Great course overall. Thanks to the instructor and TA's and mentors on the forums for making this possible.

For students my notes would be - a) a bit too much hand holding on assignments and b) the subject matter is less interesting conceptually (unless you are into numerical optimization). For instance- I skipped ahead to RNNs to keep it fresh.

For staff my feedback would be : consider re-shuffling the course material into the other courses (CNN , RNN) rather than have this all in a block. The subject matter is a bit dry, and for instance, tensor flow can be learned as part of building a NN in course 1.

By Ananthapadmanabhan K

Oct 1, 2017

Very good course. But unlike the previous one, I got the feeling we covered too much ground before the quizes. Perhaps having multiple quizes could make the learning stick a bit more.

Also, the intermediate verification of the programming assignment (tensorflow) was a little less satisfying than before because the intermediate "expected results" were mostly results of variable initialization or allocation. This is understandable because the "run" doesnt happen until a few steps later. But overall, the previous exercises were more rewarding even leading up to the big final "run everything together" part.

By Christian M N

May 8, 2020

Videos are very good, can't really criticize much there. The assignments are WAY too easy though. There's really no reason why they can't just let you implement more of the boiler/helper code, since everything is already auto graded. I think that guiding the user with the notebook structure and providing a description for what each function should do and the output shape should be more than enough for any user to do these assignments. As it is, these assignments don't really provide any assurance that that you understand the material, since the answer is always in the description.

By Abhishek R

Jul 14, 2020

It's a very useful course. The course material is well spaces out and encompasses the basics requires to utilise an algorithm. The programming exercises are easy to implement and rather rewarding when completed. Perhaps the only issue I had with the course was that it only taught enough tensorflow to allow us to implement the functions necessary for that particular problem. A more general understanding could have helped. Also, it, like most other ML/DL courses, never addresses the issue of actually accumulating or acquiring a dataset, which I find to be rather important to learn.

By Eric W

Feb 11, 2021

Hi,

just a brief feedback from my side. As a beginner in Deep Learning and Machine Learning in general the course was really the teaching style I was looking for, quite close to what I was and am used to at the University. One particular thing that was not so easy to come to terms with was the switch from mathematical notation in the slides to the Python expressions used in the assignments. Of course it was possible to learn them after some time, but I felt the learning curve a bit steep at that point assuming one has barely practical experience with it.

Anyway, a great course!

By Daniel C K

Sep 9, 2017

Good overview of techniques to use in deep learning. I especially appreciated the introduction to Tensorflow. The projects felt like they held your hand a bit too much. I'd like to see more rigorous assignments in the future. The Tensorflow tutorial felt unpolished. It contained a few errata and the tutorials were sometimes vague about how to use the syntax. I ended up having to look up the documentation for a few of the commands to understand how to use them. That's a minor issue for me, I have experience working in python, but for other learners it might be an issue.

By Andrew F

Feb 11, 2019

I enjoyed everything in this course. As always, the concepts and motivations for the various deep learning techniques are very well explained. I feel like I could have a conversation with members of the industry and not come across as a total novice (although I am).

The only real issue I had was regarding Tensorflow. I felt more time should be spent on Tensorflow before assigning homework on it. I believe in a certain amount of learning though struggle when it comes to programming, but I think the foundations for tensorflow were lacking to sufficiently struggle through.

By Rajesh R

Oct 27, 2017

A continuation of the Deep Learning specialization, this course actually teaches many of the latest ideas in hyperparameter optimization and builds up the ideas nicely. Concepts like EWMA, RMSProp, Momentum and Adam were well explained and logically introduced. One gripe I have is Tensorflow - despite not liking the framework much (primarily because of its terrible error messages), I found that the foundational learning for key concepts in TF were not well introduced. There was only one video which served as a Tensorflow introduction. Other than this, the course was great.

By Martin S

May 23, 2021

It's really great to the get the key insights from someone as Andrew Ng. I read some materials before in books, tutorials and other sources. But this course actually put it all into perspective and gave many aspects context which I was missing before. Thank you Andrew. If there would be one thing I would suggest to improve its the assignments. I missed using the batch norm and seeing its effect and also to actually improve the parameters of some DNN in some kind of search as was explained. But I understand that I might have been too much or rather too demanding.

By Nachiketa M

Sep 29, 2017

TensorFlow uses slightly different terminology. That took some time to get used to. Also, because of so many algorithms, parameters, it becomes difficult to remember everything. It would be nice, if we could search for terms and either get to the point in video where the terms are mentioned and explained or results contain sections of slides as presented in the video. If this kind of search works across all the courses in a specialization, that would also be great. Overall, Andrew Ng makes it really easy to understand and I am glad I am making progress.

By Deepak K G S

Oct 6, 2019

Very good presentation material and deep intuitive explanation by Professor Andrew NG . Quizzes are structured in a way that it will test all the sections you have learned in that week.Programming assignments will provide an idea on how to approach each of the problems.In addition to taking this course,I would also recommend you to try and code using the basic code structure presented in the lecture and assignments to build upon and see what works and what does not in the problems.Only then,you can see the benefit of what is taught in this lecture.

By Ravi R

Oct 10, 2020

Very good course for those who wants to start their career in Deep learning. One thing which disappointed me was programming assignments. Some functions and libraries of python and Tensorflow used in assignments are not well explained for students. How can a student understand if he wants to write code from scratch to develop an algorithm. Only small function or some logics are allowed to write by students in assignments which is not enough to build confidence. Many libraries imported are not explained. Theory part of course was excellent.

By Lucas S

Oct 14, 2018

Theoretical part amazing, explanations very comprehensive. I think is the best course on deep learning right now. The one thing i would like to see improved is the structural choice for the practical part: this concept of functions that already built in and we come in and fill in the code is ok, and it helps, but i think there should be some kind of path where we can build everything from scratch and you guys could provide the answers as something we can check to see if we got it right...i don't maybe could help, but overall amzing course.

By Markus B

Sep 7, 2017

Pro: The course content is well explained and the examples are usually understandable. There are some well explained programming exercises that allow you to get in touch with the "machine room".

Con: Not enough programming exercises to explain all concepts and also the programming exercises sometimes boil down to copy&pasting some code from the instructions. Furthermore, I would expect that this couse with "intermediate" difficulty would allow you to really write code from scratch at some point instead of filling in "Jupiter" notebooks.

By Mijael M

Mar 26, 2018

The content is great as usual, but I have two minor complaints: the quality of prof. Ng's handwriting went down, which sometimes makes it more difficult to follow the explanation, or taking notes; the second one is that the error reporting for the grader on the programming exercises is not very user-friendly. I had to try a couple times on weeks 2 and 3 not because my code was wrong at all, but because the grader was finicky. Due to the honor code, I believe I won't be able to post more details here, but check on the forums.

By Aaron L

Nov 25, 2017

The content is great, let me say that first and foremost. I feel like some small improvements would be that there were some typos in the notebooks, but not big deal. Helps one learn about debugging and think independently.

About the TensorFlow section of week 3. It was a pretty deep dive into TensorFlow, and I ended up going to the documentation a lot. Maybe some background on the framework would help. I will probably next go through the TensorFlow "getting started" tutorial, to better understand it.

By Eduardo M

Oct 29, 2018

The course is pretty good. I just feel notation used by Andrew is quite confusing for people with matrix algebra and matrix calculus. I understand the course is intended for people with different knowledge leveles but generalizing notation with matrix algebra could be save time for students.

There are some little bugs on the last Jupyter implementation with TensorFlow. Nothing too serious but reading the forums I noticed bugs were reported one year ago and nothing has been done to fix them.

By sudhir k

Nov 20, 2019

This was a great course with a great assignment. The assignments were moderately hard to complete. I think if students were challenged to improve accuracy of the model by a X%(10%) for extra credit. It this would have triggered independent thinking. I think Students can do it without extra credit also. I think extra credit from Instructor triggers different incentive to complete it. This was done to some extent in the 1st course. I think doing it in this course also would have been ideal.

By Douglas C

Oct 28, 2021

The course presented a number of practical techniques for implementing DNNs. The presentation was clear and sufficiently detailed to give a grounding in the techniques. Dr. Ng makes the material accessible while still offering technical details and insights that make the course both interesting and useful. The programming assignment forced me to dig into the documentation for tensorflow, which was at first frustrating, but in then gave me a much better understanding of what was going on.

By Hamidreza C

Mar 13, 2019

Many many thanks for putting this great deep learning specialization together!!!

For course 2, it took long long to get to the meat of the course, i.e. hyper parameter tuning, and yet there were no exercises to grasp how we can tune (more than one) hyper parameters through programming exercise. Perhaps we will learn that in course 3. I haven't done it yet.

The first course exercises were more effective.

Other than this comment, everything else for this specialization course looks awesome.

By Amit W

Oct 6, 2018

Hyperparameter tuning to improve performance of model is one of the most important part of lifecycle of development of any machine learning model. I would say with confidence now that I have at least got intuition of how different hyper parameters affect the performance of model and how to obtain the optimal value of them. I have got some imagination around hyper-parameters. Thank you Andrew and all team for taking diligent efforts to make this course easy to understand.

By Shiraz R

Feb 22, 2018

Course content was complex, yet progressive, helping to grasp key concepts easily. I think the assignment material can be improved. For instance, I got a full grade on the Tensorflow assignment, but my compute_cost function was wrong (hadn't passed the right arguments to the tf cross-entropy cost function). Some of the assignment instructions are also unclear at times.

Overall, this course helps build some invaluable skills for practical machine learning applications.

By Luisa F A S

Aug 3, 2022

Theoretical foundation on algorithms and tuning and also insights on these topics are amazing. However, I would have loved to see a more detailed intro to TensorFlow, as W3's assignment is quite challenging for someone who's never worked with the framework before. I know there's an option for taking a course prior to the assignment, but at least in my case it wasn't possible for time constraints. Maybe mentioning that prior TF knowledge is required would also help.

By André M

Oct 24, 2019

4* only because the TensorFlow lectures and assignment were too much in too little time. Also from what I see, TF has massively changed syntax to 2.0 so it felt a bit pointless to learn TF1 syntax (which is ***horrible***) at this point. To me it detracted a lot from the learning experience.

The remaining lectures and modules were excellent as usual though. I'd still recommend this highly, and Andrew's insights into what tends to work and why are brilliant as always.

By Roudy E

Nov 8, 2020

Another great course, the amount of information per week is right on point (not too packed and not too poor). Also, it was interesting to go behind the scenes and learn what batch normalization and regularization actually does and how it can actually help a neural network perform better. And, to top it off, it gives a brief introduction on TensorFlow and how to use it, although it would have been better if the course thought the material on TF 2.0 instead of 1.0.