A probe asks whether information can be predicted from a representation. Predictive success does not establish that the original model uses that information causally. A probe may exploit a confounder, memorized split, or information that is present but ignored by downstream computation.
A simple direction is the difference between the positive and negative class mean activations, normalized to unit length. Fit it on training activations only. Evaluate its predictions on held-out examples and compare a baseline.
Fitting a direction and testing it on held-out examples are separate operations.
import numpy as np
positive = np.array([[2.0, 0.0], [4.0, 0.0]])
negative = np.array([[0.0, 0.0], [0.0, 0.0]])
direction = positive.mean(0) - negative.mean(0)
direction /= np.linalg.norm(direction)
print(direction.tolist())Removing a unit direction d from activation h computes h - (h·d)d. It removes one component while retaining orthogonal components. Normalize arbitrary nonzero directions before applying this formula; otherwise direction scale changes the intervention. A zero direction has no defined orientation.
import numpy as np
h = np.array([[3.0, 4.0]])
d = np.array([1.0, 0.0])
residual = h - (h @ d)[:, None] * d
print(residual.tolist())An intervention can move activations off-distribution and change unrelated behavior. Compare matched random directions, multiple seeds, task performance and intervention magnitude before a causal claim. In this small build we verify algebraic properties, not a claim about real model internals.
Removing one component keeps the orthogonal remainder. This algebraic property is much narrower than a causal claim about a model.
Test projection idempotence, scale invariance of direction, orthogonality of residuals and unchanged input storage. Probe fitting must require both classes and a nonzero mean difference. In the final lab, document what a toy probe result can and cannot establish before trying any external model study.