A very complicated or overly basic model may lead to overfitting or underfitting, the most typical issues in machine learning. Both may hinder efficiency when dealing with the unknown.
How to solve overfitting?
When a model is overly complicated, it begins to match the random fluctuations present in the training data rather than the intended pattern, a phenomenon known as overfitting. The resulting model does well on the training data but badly on new data. Overfitting may be dealt by the following:
- To simplify the model, include a penalty term in the loss function. L1 and L2 regularization are popular options.
- Dropout is the process of arbitrarily removing neurons from a network during training in order to simplify the model.
- Early stopping refers to suspending training when results on a validation set begin to deteriorate.
- Using ensemble techniques by employing many models together to prevent overfitting.
How to solve underfitting?
When a model is too simplistic, it fails to accurately reflect the structure of the data. In this case, the machine learning model accuracy bears subpar results in both the training set and in the wild. Several methods exist for dealing with underfitting:
- Complexifying the model by including additional features, layers, and/or neurons.
- Adding more information to the training set so there are more instances for the model to draw from.
- Lessening regularization is decreasing the intensity of the regularization used on the model.
Recap
Finding the sweet spot between underfitting and overfitting is a challenge that may need trying out a variety of approaches, model architectures, and hyperparameter settings before success is achieved.
When a machine learning model is either too complicated or too basic, overfitting or underfitting occur. Reduce overfitting in a neural network by using approaches like regularization, dropout, early halting, and ensemble methods. Methods for dealing with underfitting include amping up model complexity, data collection, and down regularization. It might be difficult to find the perfect place between overfitting and underfitting in machine learning without trying out a variety of approaches and model designs.
Install Open Source
Anton Knight 
Tiara Williamson 