Chevron Left
Back to Improving Deep Neural Networks: Hyperparameter Tuning, Regularization and Optimization

Learner Reviews & Feedback for Improving Deep Neural Networks: Hyperparameter Tuning, Regularization and Optimization by DeepLearning.AI

4.9
stars
63,156 ratings

About the Course

In the second course of the Deep Learning Specialization, you will open the deep learning black box to understand the processes that drive performance and generate good results systematically. By the end, you will learn the best practices to train and develop test sets and analyze bias/variance for building deep learning applications; be able to use standard neural network techniques such as initialization, L2 and dropout regularization, hyperparameter tuning, batch normalization, and gradient checking; implement and apply a variety of optimization algorithms, such as mini-batch gradient descent, Momentum, RMSprop and Adam, and check for their convergence; and implement a neural network in TensorFlow. The Deep Learning Specialization is our foundational program that will help you understand the capabilities, challenges, and consequences of deep learning and prepare you to participate in the development of leading-edge AI technology. It provides a pathway for you to gain the knowledge and skills to apply machine learning to your work, level up your technical career, and take the definitive step in the world of AI....

Top reviews

AS

Apr 18, 2020

Very good course to give you deep insight about how to enhance your algorithm and neural network and improve its accuracy. Also teaches you Tensorflow. Highly recommend especially after the 1st course

HD

Dec 5, 2019

I enjoyed it, it is really helpful, id like to have the oportunity to implement all these deeply in a real example.

the only thing i didn't have completely clear is the barch norm, it is so confuse

Filter by:

6076 - 6100 of 7,249 Reviews for Improving Deep Neural Networks: Hyperparameter Tuning, Regularization and Optimization

By Mijael M

•

Mar 26, 2018

The content is great as usual, but I have two minor complaints: the quality of prof. Ng's handwriting went down, which sometimes makes it more difficult to follow the explanation, or taking notes; the second one is that the error reporting for the grader on the programming exercises is not very user-friendly. I had to try a couple times on weeks 2 and 3 not because my code was wrong at all, but because the grader was finicky. Due to the honor code, I believe I won't be able to post more details here, but check on the forums.

By Aaron L

•

Nov 25, 2017

The content is great, let me say that first and foremost. I feel like some small improvements would be that there were some typos in the notebooks, but not big deal. Helps one learn about debugging and think independently.

About the TensorFlow section of week 3. It was a pretty deep dive into TensorFlow, and I ended up going to the documentation a lot. Maybe some background on the framework would help. I will probably next go through the TensorFlow "getting started" tutorial, to better understand it.

By Eduardo M

•

Oct 29, 2018

The course is pretty good. I just feel notation used by Andrew is quite confusing for people with matrix algebra and matrix calculus. I understand the course is intended for people with different knowledge leveles but generalizing notation with matrix algebra could be save time for students.

There are some little bugs on the last Jupyter implementation with TensorFlow. Nothing too serious but reading the forums I noticed bugs were reported one year ago and nothing has been done to fix them.

By sudhir k

•

Nov 20, 2019

This was a great course with a great assignment. The assignments were moderately hard to complete. I think if students were challenged to improve accuracy of the model by a X%(10%) for extra credit. It this would have triggered independent thinking. I think Students can do it without extra credit also. I think extra credit from Instructor triggers different incentive to complete it. This was done to some extent in the 1st course. I think doing it in this course also would have been ideal.

By Douglas C

•

Oct 28, 2021

The course presented a number of practical techniques for implementing DNNs. The presentation was clear and sufficiently detailed to give a grounding in the techniques. Dr. Ng makes the material accessible while still offering technical details and insights that make the course both interesting and useful. The programming assignment forced me to dig into the documentation for tensorflow, which was at first frustrating, but in then gave me a much better understanding of what was going on.

By Hamidreza C

•

Mar 13, 2019

Many many thanks for putting this great deep learning specialization together!!!

For course 2, it took long long to get to the meat of the course, i.e. hyper parameter tuning, and yet there were no exercises to grasp how we can tune (more than one) hyper parameters through programming exercise. Perhaps we will learn that in course 3. I haven't done it yet.

The first course exercises were more effective.

Other than this comment, everything else for this specialization course looks awesome.

By Amit W

•

Oct 6, 2018

Hyperparameter tuning to improve performance of model is one of the most important part of lifecycle of development of any machine learning model. I would say with confidence now that I have at least got intuition of how different hyper parameters affect the performance of model and how to obtain the optimal value of them. I have got some imagination around hyper-parameters. Thank you Andrew and all team for taking diligent efforts to make this course easy to understand.

By Shiraz R

•

Feb 22, 2018

Course content was complex, yet progressive, helping to grasp key concepts easily. I think the assignment material can be improved. For instance, I got a full grade on the Tensorflow assignment, but my compute_cost function was wrong (hadn't passed the right arguments to the tf cross-entropy cost function). Some of the assignment instructions are also unclear at times.

Overall, this course helps build some invaluable skills for practical machine learning applications.

By Luisa F A S

•

Aug 3, 2022

Theoretical foundation on algorithms and tuning and also insights on these topics are amazing. However, I would have loved to see a more detailed intro to TensorFlow, as W3's assignment is quite challenging for someone who's never worked with the framework before. I know there's an option for taking a course prior to the assignment, but at least in my case it wasn't possible for time constraints. Maybe mentioning that prior TF knowledge is required would also help.

By André M

•

Oct 24, 2019

4* only because the TensorFlow lectures and assignment were too much in too little time. Also from what I see, TF has massively changed syntax to 2.0 so it felt a bit pointless to learn TF1 syntax (which is ***horrible***) at this point. To me it detracted a lot from the learning experience.

The remaining lectures and modules were excellent as usual though. I'd still recommend this highly, and Andrew's insights into what tends to work and why are brilliant as always.

By Roudy E

•

Nov 8, 2020

Another great course, the amount of information per week is right on point (not too packed and not too poor). Also, it was interesting to go behind the scenes and learn what batch normalization and regularization actually does and how it can actually help a neural network perform better. And, to top it off, it gives a brief introduction on TensorFlow and how to use it, although it would have been better if the course thought the material on TF 2.0 instead of 1.0.

By Konstantinos K

•

Aug 6, 2020

The course is great!

It really helps in understanding how the algorithms work, under the hood and the implementation tips

are very helpful! (This is visible in both the Optimization and Batch Normalization algorithms sections)

It is awesome that a programming framework is also introduced in the course, Tensorflow. But to be honest PyTorch could be also introduced, in order to select the framework in which the student could implement the last programming assignment.

By Joshua H

•

May 17, 2020

The content covers a wide variety of useful topics in deep learning. Andrew's explanations are sufficient, as are his use of both examples and analogies. I was only slightly disappointed to see that he has left the derivation of the equations governing back propagation along a batch normalized neural network as an exercise to his audience. The quizzes were sufficiently challenging, and the programming exercises were either informative or insightful, or both.

By dheeraj i

•

Sep 25, 2017

I felt this interesting but bit easier compared to the first course. Please don't provide the parameters of a method directly in the description above. I want to learn how this method can be executed by thinking and understanding the parameters I have to pass to this method. So, I felt the tensorflow assignment little straight forward. But overall a very good course. I need to practice a lot to actually understand and write the code from scratch. Thank you.

By Jian L (

•

Sep 4, 2018

I wish to give 4.5 instead. The only pitfall is the whole video series have a high frequent sound which keep distracting me when try to concentrate to the content.

content vise is very good. Coding practice is very helpful in understand the process. However there is only a basic level. With giving too much help on the background, it's very easy forget afterwards. May be a suggestion more practice would be better.

After all, it's the best course in DL for me.

By John C

•

Nov 6, 2020

The number of mathematical symbols grows quickly, and I started getting a little lost trying to remember which Latin or Greek symbol meant what and in which context. Still, I think that I've learned enough about overfitting (bias), underfitting (variance), regularization, adaptive learning rates, and normalization that I'll at least have the concepts in mind moving forward, even if I didn't memorize the equations and code necessary to do it from scratch.

By Bruce W

•

Jun 23, 2020

This was a good course to deal with some of the inner working of the machine learning and neural network models. It was good to see one of the existing frameworks (TensorFlow); although, I find it to be more difficult to configure than Torch (PyTorch). And it was unclear from the lab whether or not this framework was using GPU acceleration; although, this could probably be determined with a little research and experimentation in the lab environment.

By Steve I

•

Sep 26, 2019

This is a great overview for those wanting their neural networks to run more effectively and efficiently. Lots of ideas to improve your networks. The documentation and description of Tensorflow for the exercises is inadequate to be able to diagnose errors in the "expected" code without expert assistance. When debugging Tensorflow for these exercises, its almost a Trial and Error exercise instead of using first principles taught in the presentations.

By Mats K

•

Jan 29, 2021

The material is very interesting, but a little light on the mathematics, which I personally would enjoy seeing more of. I would like to see more elaborations and proofs, but they can be optional. A little too much hand-holding in the assignments. Learning to find relevant information is part of the training and as a programmer I find that the assignments consists of a lot of cutting and pasting snippets of code from the instructions in the notebooks.

By Robert S

•

Jun 15, 2021

A great follow-up to the first course in the specialization. Answers a lot of questions that might have occurred to you while taking the first course.

Compared to university courses I have taken, this one feels to me as if it is taught at about a second year level. As such, keep in mind that to fully absorb the material you will need to do more than just follow along, you will need to practice on your own by finding or creating your own problems.

By Marcello

•

Nov 29, 2023

The content of the course is very good and well explained, only this would be 5 stars. I gave only 4 stars because all the lessons are video lessons. Even though there is a transcript, the text is not well formatted and it does not underline core concepts, math formulas, code snippets, etc. So if you prefer a text lesson, it is hard to read, and you still rely on the video because the teacher writes many explanations on the whiteboard.

By Calvin K

•

Mar 4, 2018

Love the orthogonalization part and the explanation on why training deep neural networks is possible (local minimum is rare in hyperspace; for the most part there are saddle points). Tho I was hoping there would be some advice on how to design a neural network. Overall I think it's a bit too easy for those who have already known deep learning or taken Ng's Machine Learning course. It'd be great if the homework would be more challenging.

By Daniel R

•

Dec 18, 2019

Similar to my previous review in 'Neural Networks and Deep Learning', I found this course to be particularly good. It improved on the vanilla model introduced in Course 1, strengthening notions of tuning and regularization. I found it to be quite useful. My only complaint is that the assignments are similarly too much 'hand-holding' so I would advise those performing the assignments to try to develop some of the functions from scratch.

By Agustín D

•

Mar 17, 2018

You can learn a lot of topics related to deep learning algorithm optimization. All of them are explained in great detail, but the last assignment with TensorFlow was a bit confusing because of its own structure, not the instructor's fault. Also there was not an explicit motive why they chose to explain with TensorFlow.

Besides that, the course was very nutritious and I feel more confident about deep learning after completing the course.

By Kryštof C

•

Nov 7, 2018

In comparison with the first course of Deeplearning.ai , this course was a bit shallow in some topics. I would for example divide first and third week int two parts and add little bit deeper information. On the other hand, it was still very informative. The course provide very nice probe to optimization techniques of training DNNs. I would recommend this course to everyone, who wants to expand the basic knowledge of Neural Networks.