Contrastive loss

Background

Contrastive loss trains a Siamese embedding from pairs: similar pairs are pulled together while dissimilar pairs are pushed apart until they are at least a margin away. It is the pairwise cousin of triplet loss and the basis of Siamese networks for verification and similarity learning.

Problem statement

Implement contrastive_loss(x1, x2, label, margin=1.0). With Euclidean distance $d_i = \lVert x1_i - x2_i\rVert$ and label $y_i$ (1 = similar, 0 = dissimilar):

$$\mathcal{L} = \frac{1}{N}\sum_i \Big[\, y_i\, d_i^2 + (1 - y_i)\,\max(0,\, \text{margin} - d_i)^2 \,\Big]$$

Input

• x1, x2 — np.ndarray (N, D): paired embeddings.
• label — np.ndarray (N,): 1 for similar pairs, 0 for dissimilar.
• margin — float: the minimum desired distance for dissimilar pairs.

Output

Returns a float: the mean contrastive loss ($\ge 0$).

Examples

Example 1

Input:  x1 = [[0,0],[0,0]], x2 = [[1,0],[0.5,0]], label = [1, 0], margin = 1.0
Output: 0.625

Explanation: the similar pair ($y=1$) has $d=1$, contributing $1\cdot 1^2 = 1$. The dissimilar pair ($y=0$) has $d=0.5 < 1$, contributing $\max(0, 1-0.5)^2 = 0.25$. Mean $= (1+0.25)/2 = 0.625$.

Constraints

• $d$ is the (non-squared) Euclidean distance; the similar term squares $d$, the dissimilar term squares the hinged margin gap.
• Dissimilar pairs already farther than margin contribute 0.
• Average over the batch; the result is $\ge 0$.

Notes

• Similar pairs pull quadratically with $d^2$ (no margin); dissimilar pairs push only while inside the margin — once separated they stop contributing.
• This is the loss behind classic Siamese signature/face verification (Hadsell, Chopra & LeCun, 2006).