Bias-variance strategies in neural network

• bias refers to underfitting and variance refers to overfitting
• In Neural network, there are few set rules to counter bais and variance issue

• High Bias/Underfitting:

1. Bigger Network: Add more layers and increase nodes in layer to counter underfitting
2. More Epochs/Train Longer: Increase number of passes over the entire data to counter underfitting
3. Differnt NN architecture: Try different network architecture

• High Variance/Overfitting:

1. Get more data: Increase data points collected
2. Make model simpler: Simplify the model in terms of less layers, and less nodes
3. Regularization: Add l1/l2 regularization to weights
4. Dropout: Dropout means some % of nodes will be automatically turned off (not trained); generally set to 10%-30%

•