Weighted cross-entropy

Background

Weighted cross-entropy scales each class's contribution to the loss — the standard fix for class imbalance. Rare classes get larger weights so the model can't ignore them. It generalises plain cross-entropy, which is the special case where all weights equal 1.

Problem statement

Implement weighted_cross_entropy(probs, labels, weights, epsilon=1e-12):

$$\mathcal{L} = -\frac{1}{N}\sum_{i=1}^{N}\sum_{c=1}^{C} w_c\, y_{ic}\,\log \hat{p}_{ic}$$

with one-hot labels $y$, predicted probabilities $\hat{p}$ (clipped to avoid $\log 0$), and per-class weights $w$.

Input

• probs — np.ndarray (N, C): predicted class probabilities.
• labels — np.ndarray (N, C): one-hot true labels.
• weights — np.ndarray (C,): per-class weights.
• epsilon — float: clipping constant.

Output

Returns a float: the mean weighted cross-entropy.

Examples

Example 1

Input:  probs = [[0.7, 0.3], [0.2, 0.8]], labels = [[1,0],[0,1]], weights = [1.0, 2.0]
Output: 0.4015

Explanation: sample 0 (class 0, weight 1) contributes $-\log 0.7 = 0.357$; sample 1 (class 1, weight 2) contributes $-2\log 0.8 = 0.446$; the mean is $0.4015$.

Constraints

• Clip probs to $[\epsilon,\, 1-\epsilon]$ before the log.
• Weight each class's log-prob term by weights[c], sum over classes, then average over samples.
• With all weights $= 1$ this reduces to plain cross-entropy.

Notes

• Up-weighting rare classes raises their loss contribution, pushing the optimiser to fit them instead of collapsing to the majority class.
• It is equivalent in expectation to over/under-sampling, but cheaper — it reweights rather than resamples.