Gradient Vanishing / Exploding

Problems in training deep networks where gradients become extremely small (vanishing) or large (exploding) during backpropagation, making learning unstable. Solutions: residual connections (skip connections), batch normalization, gradient clipping, careful weight initialization (He, Xavier).

← Fine-Tuning · Hyperparameter → · All terms