A training step predicts, computes a scalar loss, differentiates and updates parameters in the negative gradient direction. Learning rate controls step size; a smaller training loss alone does not show generalization. Log a baseline and evaluate held-out data with fixed metrics.
One training iteration must clear the previous gradient before differentiating the current objective.
import torch
w = torch.tensor([0.0], requires_grad=True)
loss = (w - 2).square().mean()
loss.backward()
with torch.no_grad():
w -= 0.1 * w.grad
w.grad = None
print(round(w.item(), 3))backward accumulates into .grad, so clear gradients before each new step. Updating leaf parameters under no_grad avoids recording the optimization operation as part of the model graph. For binary classification, use logits and binary_cross_entropy_with_logits: combining sigmoid and logs manually can overflow or take log(0) on extreme predictions.
import torch
import torch.nn.functional as F
logits = torch.tensor([-1000.0, 1000.0])
labels = torch.tensor([0.0, 1.0])
print(float(F.binary_cross_entropy_with_logits(logits, labels)))model.eval() changes behaviors such as dropout and batch normalization; it does not disable gradient tracking. torch.no_grad() disables graph recording in its context; it does not change module mode. Evaluation normally needs both. A helper should restore the caller's mode even if forward raises.
Evaluation mode and gradient recording control different mechanisms. Neither setting implies the other.
The build trains a no-bias logistic regressor from zero weights, making the first gradient step independently checkable. It also evaluates an arbitrary logits module with no gradients, preserving its prior training flag. The helper assumes uniform module mode, not a custom mixture of child modes. Keep these exercises CPU-sized; they teach mechanics without model downloads.