GMM E-step (responsibilities)

Background

A Gaussian Mixture Model assumes the data was generated by $K$ Gaussian components mixed with weights $\pi_k$. EM alternates two steps to fit it; this is the E-step, which computes each point's responsibility $\gamma_{ik}$ — the posterior probability that component $k$ generated point $i$ — given the current parameters. These soft assignments are GMM's generalisation of k-means' hard assignments.

Problem statement

Implement gmm_e_step(X, weights, means, covariances) that returns the responsibility matrix:

$$\gamma_{ik} = \frac{\pi_k\, \mathcal{N}(x_i \mid \mu_k, \Sigma_k)}{\sum_{j=1}^{K} \pi_j\, \mathcal{N}(x_i \mid \mu_j, \Sigma_j)}$$

using the multivariate normal density:

$$\mathcal{N}(x \mid \mu, \Sigma) = \frac{1}{\sqrt{(2\pi)^{D}\, \lvert\Sigma\rvert}}\, \exp\!\Big(-\tfrac{1}{2}(x-\mu)^\top \Sigma^{-1} (x-\mu)\Big)$$

Input

• X — np.ndarray of shape (N, D): the data points.
• weights — np.ndarray of shape (K,): mixing weights $\pi_k$ (sum to 1).
• means — np.ndarray of shape (K, D): the component means.
• covariances — np.ndarray of shape (K, D, D): the component covariance matrices.

Output

Returns an np.ndarray of shape (N, K): each row is a probability distribution over the $K$ components (its entries sum to 1).

Examples

Example 1

Input:  X = [[0,0],[5,5]], weights = [0.5,0.5],
        means = [[0,0],[5,5]], covariances = [I, I]
Output: ~[[1, 0], [0, 1]]

Explanation: the first point sits on component 0's mean and is far from component 1, so almost all its responsibility goes to component 0; the second point is the mirror image.

Constraints

• Use the full multivariate-normal density with each component's own covariance.
• Each output row must sum to $1$ — normalise the weighted densities across components.
• Shapes: X is (N, D); the output is (N, K).
• Tests compare with atol=1e-6.

Notes

• The E-step yields soft assignments; the M-step (not asked here) re-estimates $\pi, \mu, \Sigma$ from these responsibilities, and alternating the two is the EM algorithm.
• As the covariances shrink toward zero the responsibilities harden to 0/1 and GMM collapses to k-means.