Glossary › Batch Normalization
Batch Normalization
A technique that normalizes layer activations during training, reducing internal covariate shift. Applied after a linear layer and before the activation function. Enables higher learning rates, acts as regularization, and makes training more stable. Standard in CNNs; less universal in transformers, which use Layer Normalization.