Instance normalization

Background

Instance Normalization normalizes each channel of each sample independently over its spatial dimensions. It is the standard in style transfer and image generation, where per-image, per-channel contrast normalization removes instance-specific contrast that would otherwise leak into the output.

Problem statement

Implement instance_normalization(X, gamma, beta, epsilon=1e-5) for X of shape (B, C, H, W). For each (sample, channel) normalize over the $(H, W)$ spatial dimensions, then apply the per-channel affine:

$$\hat{X}_{b,c} = \frac{X_{b,c} - \mu_{b,c}}{\sqrt{\sigma_{b,c}^2 + \epsilon}}, \qquad Y = \gamma\,\hat{X} + \beta$$

with $\mu_{b,c}, \sigma_{b,c}^2$ the mean/variance over $(H, W)$.

Input

• X — np.ndarray (B, C, H, W).
• gamma, beta — np.ndarray (C,): per-channel scale and shift.
• epsilon — float.

Output

Returns an np.ndarray (B, C, H, W).

Examples

Example 1

Input:  X = [[[[1, 2], [3, 4]]]]  (shape 1x1x2x2), gamma = [1], beta = [0]
Output: [[[[-1.3416, -0.4472], [0.4472, 1.3416]]]]

Explanation: the single channel's four values $\{1,2,3,4\}$ are normalized (mean 2.5, std $\approx1.118$), then the identity affine is applied.

Constraints

• Mean/variance over spatial axes (2, 3) per (sample, channel), with keepdims.
• Apply per-channel gamma/beta (broadcast as (1, C, 1, 1)).
• Tests compare with atol=1e-4.

Notes

• Instance Norm equals GroupNorm with num_groups = C (each channel is its own group).
• Stripping per-instance contrast is exactly why it works for style transfer: the "style" lives in channel statistics, which IN removes before re-styling.