CodeRigor

FREE LESSON

NumPy shapes, axes & numerical stability

Vectorized row normalization and all-pairs distances with explicit axes, zero handling and input independence.

An ndarray has a shape and dtype. A matrix of N examples with D features has shape (N,D). Reducing axis=-1 combines features; reducing axis=0 combines examples. Write the intended shape beside every intermediate result.

Preserve the reduced feature axis so every row is divided by its own norm.

Row normalization keeps an axis
Reducing features with keepdims=True changes (N,D) to (N,1). That scale broadcasts across each row; zero norms are replaced by one.
Row normalization keeps an axis
Reducing features with keepdims=True changes (N,D) to (N,1). That scale broadcasts across each row; zero norms are replaced by one.
Python
import numpy as np

x = np.array([[3.0, 4.0], [0.0, 0.0]])
lengths = np.sqrt((x * x).sum(axis=-1, keepdims=True))
print(x.shape, lengths.shape)
safe = np.where(lengths == 0, 1, lengths)
print(x / safe)

Broadcasting aligns dimensions from the right. Equal sizes match; a size-one axis can expand. (N,D) divided by (N,1) scales each row; dividing by (N,) can fail or silently scale columns when N equals D. keepdims=True preserves the axis that carries this meaning. Zero rows should remain zero without warnings.

Python
import numpy as np

a = np.array([[0.0, 1.0], [2.0, 3.0]])
b = np.array([[1.0, 1.0], [4.0, 3.0], [0.0, 0.0]])
delta = a[:, None, :] - b[None, :, :]
print(delta.shape)
print((delta * delta).sum(axis=-1))

The inserted axes express all N-by-M pairs. This uses O(NMD) temporary storage, suitable for our small tasks; production code may need blocks. Subtracting coordinates first also avoids the large-value cancellation in ||a||² + ||b||² - 2a·b. Converting to float64 before arithmetic avoids integer overflow and improves this exercise's numerical behavior.

The two inserted axes below mean every A row is compared with every B row; the last axis still represents features.

All-pairs distances by broadcasting
Subtract coordinates first, square, then sum over D. The result has one distance for each of the N×M pairs.
All-pairs distances by broadcasting
Subtract coordinates first, square, then sum over D. The result has one distance for each of the N×M pairs.

np.asarray can share input memory. Avoid in-place normalization if the caller needs the original baseline. Test unequal N and D, a zero row, empty inputs, and integer arrays, rather than only one square floating-point matrix.

Watch & read

Free lectures, talks and readings that cover this skill from another angle. All resources →

Start with the broadcasting and shape examples; stride tricks are optional further study. Write each axis on paper before implementing the lesson’s pairwise distances and keepdims reductions.