The same picture, read two different ways.

Count connected background regions. The ring encloses a hole, so its background splits into two components — the outside and the interior. The bar and the hook are solid, so their background is a single connected region.

What it costs. A connected-components algorithm — flood fill or union-find — which is perhaps thirty lines rather than the one line that "count the central 2×2" needed. It also handles the boundary case badly: a ring touching the image edge may have its interior merge with the outside, so it needs padding. And it does nothing at all to separate bar from hook, so the top-row feature is still required and is still translation-fragile in the vertical direction.

That progression is the honest experience of feature engineering: each fix is reasonable, each is more code, and each handles one variation. The 2000s were spent doing this for natural images, and the resulting pipelines plateaued around 74% on ImageNet before being replaced wholesale.

(a) Classical — gradient boosting. Tabular, moderate n, named columns with real meaning. None of deep learning's structural assumptions apply, boosting will match or beat a network, and the lending context almost certainly requires an explanation for each decision, which 2.6 and 4.8 both say a network cannot give faithfully.

(b) Deep learning — and specifically transfer learning. Images have exactly the structure convolution assumes. 50,000 is a lot for medicine but small for training from scratch, so start from an ImageNet-pretrained backbone per section 5.3. Watch for the grouped-split trap: multiple images per patient must not straddle the train/test boundary.

(c) Classical first. Three years of daily data is about 1,100 points, which is far too few for a sequence model to learn from. Start with a statistical baseline — seasonal naive, then ARIMA or a boosted model on lag features, which converts the problem to tabular. Deep sequence models become worth it at thousands of related series, not one short one.

(d) Deep learning, unambiguously. Raw audio has both spatial structure in the spectrogram and long-range sequential structure, hand-crafted phonetic features were tried for decades and lost, and 200 hours is a reasonable amount of data. This is the case where nothing else is competitive.

The pattern across all four: structure in the raw input, and enough data to learn it, are the two conditions. Fail either and classical methods are the better start.

The 0.7-point gain is probably not real, and even if it is, it is probably not worth it. Take those in order.

Is it real? A single run of each, with no variance estimate, on a difference of seven thousandths. Section 4.8 established that run-to-run variation from the seed alone is often larger than this. Report both as a mean and spread over several seeds and compare with a paired test on shared folds, per 2.4.6 — the honest answer may well be that the two are indistinguishable. Also check whether the network's hyperparameters were tuned on the test set while the logistic regression's were not, which would manufacture exactly this size of gap.

Is it the right comparison? The relevant baseline for tabular data is not logistic regression, it is gradient boosting, which would likely beat both and train in under a minute. Comparing a network against the weakest classical option and declaring deep learning superior is a rigged contest.

Is it worth it? Seven thousandths of accuracy for 7,200× the training time, plus GPU cost, plus the loss of interpretability that 2.6 may make legally necessary, plus a much harder deployment and monitoring story. Unless each point of accuracy carries very large value, this is a clear no.

What would change the answer: if the gain were several points, if it held up across seeds and against a boosting baseline, and if the task genuinely had structure the network could exploit that the tabular framing was hiding.

It is literal, and Unit 4 computed it. In section 4.1 the XOR network's hidden layer mapped the four inputs to (0,0), (1,0), (1,0), (2,1). Those two numbers are features: derived quantities, computed from the raw inputs, in terms of which the task becomes easy. The output layer is a plain linear classifier operating on them, and it succeeds only because the features were good.

What makes them features rather than arbitrary numbers is that they are fitted to the objective. A hand-designed feature is chosen by a human before seeing how well the downstream model does with it. A hidden unit's weights receive a gradient from the loss, through the output layer, by the chain rule — so the feature is adjusted precisely in the direction that helps the final prediction. Section 4.4's δ₁ = (W₂ᵀδ₂) ⊙ φ'(z₁) is the mechanism: the blame the output assigns is what shapes the representation.

And 4.2 shows the same thing with a constraint attached. A convolutional filter is a feature detector whose weights are learned, and the constraint of weight sharing means the feature it learns is the same everywhere in the image. The edge kernels of Worked 4.2 were hand-written to make the arithmetic checkable; a trained network's first layer discovers filters that look very much like them, without being told, because oriented edges are genuinely the useful thing to compute first.

So the honest statement of the difference is narrow and specific: classical ML optimises the model given the features, deep learning optimises the features and the model together against the same objective. Everything else — depth, GPUs, data scale — is what makes that joint optimisation practical rather than what makes it different in kind.

The shortcut is free when the shapes match and costs 8,320 when they do not — about 3.6% of the block. That is why ResNet keeps the channel count constant within a stage and changes it only at the stage boundary: each change costs a projection, so you make as few as possible.

18 input pixels. The jump is what makes the later layers count for more: the 5×5 kernel added 4 × 2 = 8 pixels rather than 4, because after pooling each of its inputs already summarised a 2-pixel step. That is why downsampling early is such an efficient way to buy reach, and why a network that never pools needs to be very deep to see anything large.

This is the degradation problem, and the key observation is that it is training accuracy. Overfitting would show as a training-accuracy gain with a validation loss, so this is not overfitting. And it cannot be a capacity limit, because the 40-layer network strictly contains the 20-layer one — set the extra twenty layers to the identity and it reproduces the smaller network exactly. So a solution at least as good demonstrably exists and optimisation failed to find it.

The cause is that the identity is hard for a stack of plain layers to represent. Each layer would have to learn weights that reproduce its input exactly, which is a precise and unlikely configuration for a randomly initialised layer to reach. Combined with the vanishing gradients of 4.3, the deeper network's early layers barely move.

The fix is residual connections. Writing each block as y = F(x) + x makes the identity the default rather than a target: setting F = 0 is easy, since it just means driving the block's weights toward zero, which weight decay is pushing them toward anyway. Add BatchNorm per 4.6 and check He initialisation per 4.7. With those three, 40 layers trains, and so does 152.

Worth stating the general lesson: a model that can represent a solution and cannot find it is an optimisation problem, not a capacity problem, and the two need opposite fixes. Section 4.7's XOR result — 16.5% success at exactly sufficient capacity — is the same phenomenon at the smallest possible scale.

The mechanism is turning a product into a sum. Section 4.3 established that a gradient crossing L layers is multiplied by L factors, one per layer, and that a product of factors below 1 decays geometrically — ten sigmoid layers cost a factor of 10⁶ even at their most favourable point. A residual block changes the local derivative from ∂F/∂x to ∂F/∂x + 1. Because of that +1, the product across blocks contains a term that is exactly 1 no matter how small every ∂F/∂x becomes, so the gradient reaches the earliest layers intact.

Put geometrically, the skip connection gives the gradient a highway from the loss to every layer, bypassing the multiplicative chain. The network can still use F when it helps; it simply is not forced to route everything through it.

The other mechanism is the LSTM's cell state, in section 5.4. A plain recurrent network's hidden state is updated multiplicatively, hᵗ = tanh(wₕhᵗ₋₁ + …), so the gradient across T steps is a product of T factors and dies exactly as depth does — at wₕ = 0.5 over twenty steps, a factor of 1.9 × 10⁻⁶. The LSTM replaces it with cᵗ = f · cᵗ₋₁ + i · g, an addition, whose derivative ∂cᵗ/∂cᵗ₋₁ is exactly the forget gate f. With f near 1 the gradient passes essentially unchanged, which is called the constant error carousel and is the same idea as the residual highway.

So the two headline architectural advances of the 2010s — residual networks for depth, gated recurrence for time — are the same insight applied to two different axes: when a repeated multiplication destroys a signal, add instead.

That ratio is uncomfortable — roughly eight parameters per example — so even this minimal head can overfit. Three reasonable responses: add L2 or dropout before the dense layer per 4.6; reduce the feature dimension first with a 1×1 convolution or a small bottleneck dense layer, though that adds its own parameters; or simply accept it and rely on early stopping, which costs nothing and is what most practitioners do.

Note what would happen with a two-layer head, 2048 → 256 → 5: that is 524,544 + 1,285 = 525,829 parameters against 1,200 images, a ratio of 0.002. A deeper head is almost always the wrong instinct in transfer learning, because the backbone has already done the representation learning and all that remains is a linear decision.

Large dataset, different domain — so fine-tune everything, at a low learning rate, after first training the head. 80,000 images is enough to move 25.6 million parameters without immediately overfitting, and X-rays are visually unlike photographs, so the later layers' ImageNet features — dog faces, wheels, textures of fur — are largely useless and need replacing.

What changes from the photographs case. The preprocessing must be rethought: X-rays are single-channel and are usually replicated to three channels to fit the pretrained input, and their intensity distribution is nothing like ImageNet's, so the normalisation constants deserve checking rather than copying. Augmentation must respect the domain — horizontal flips change left from right lung and can invert a diagnosis, whereas they are free on photographs. And the split must be grouped by patient, since several images from one patient in both train and test inflates the score, which is 2.1's leakage rule in its most common clinical form.

What to expect. Pretraining still helps here, mostly through faster convergence and a better optimum rather than through the features themselves surviving. It is worth running a from-scratch baseline to measure how much, because at 80,000 images the gap is often smaller than people assume, and knowing its size tells you whether to invest in more data or in a better architecture.

The fact that freezing more layers does not help is the informative part, because freezing is the strongest capacity reduction available in this setting. If cutting the trainable parameters by an order of magnitude leaves validation unmoved, the 33-point gap is not caused by having too many trainable parameters — so it is not ordinary overfitting, and more regularisation will not fix it either. This is P4.6.3's situation again: the intervention worked as designed and the diagnosis was wrong.

Check the silent failures first, in this order. Preprocessing mismatch: is the input normalised exactly as the source model expects? BatchNorm still updating in the "frozen" backbone? Both produce a model that behaves inconsistently between training and evaluation and are cheap to rule out. Then leakage or grouping in the split, which inflates training-side performance whenever near-duplicate images straddle the boundary.

Then consider distribution shift. If validation images differ systematically from training ones — different source, device, time period — no capacity change closes the gap. Diagnose by evaluating on a held-out slice of the training distribution: a high score there and a low score on validation is shift, not variance.

And check the head is not simply too weak. If the domain is far from the source, a frozen backbone may not produce linearly separable features at all, in which case both training and validation would be poor — which they are not here, so this is the least likely of the four.

It is a constraint on the hypothesis space, which is exactly 1.7's definition of regularisation. Freezing the backbone does not merely reduce the parameter count — it fixes the function computed by most of the network, so the model you are actually fitting is a linear classifier on a fixed 512-dimensional representation. Worked 5.3's 1,539 trainable parameters is the honest capacity of that model, and 1.6's variance term is computed from that number rather than from 14.75 million.

The bias that constraint introduces is where the subtlety is. A hard constraint always adds bias, and here the bias is the assumption that ImageNet's features suffice for your task. When the domains are close, the assumption is nearly true and the bias is nearly free, which is why feature extraction on photographs works so well. When the domains are far apart, the same constraint is expensive — and that is precisely what the four-quadrant table encodes: how far down to retrain is a judgement about how much of the source's bias you can afford.

The connection to 2.5 is closer than it looks. Semi-supervised learning used unlabelled data to learn structure that labelled data alone could not support. Transfer learning does the same thing with a different source: someone else's labelled data, in a different task, used to learn a representation. Both are answers to "I do not have enough labels", and both work only when an assumption holds — the cluster and smoothness assumptions in 2.5, feature transferability here. Self-supervised pretraining, which Unit 5's further reading points at, is the version that merges them: learn the representation from unlabelled data in the target domain, then attach a small head, which is the dominant paradigm in modern practice for exactly this reason.

Read what the gates decided. The forget gate at 0.8808 keeps 88% of the old memory; the input gate at 0.2689 admits only 27% of the new candidate. So this cell is in remembering mode: the state barely moved, from 2.0000 to 2.0050. Reverse the two pre-activations and it would overwrite instead. That the two gates are separate — unlike the GRU, where keeping and admitting must sum to 1 — is the LSTM's extra degree of freedom.

The plain RNN retains about four parts in 10¹⁵, which is indistinguishable from nothing. The update to the step-1 weights from this dependency is swamped by every other gradient contribution, so the model will never learn the connection — and note that it will still train happily and report a decreasing loss, having simply learned to ignore the long-range structure.

The LSTM retains 30%, which is a perfectly usable signal. Same task, same length, and the difference is entirely whether the state is updated by multiplication or by addition.

The qualification from Worked 5.4b applies: f = 0.98 is realistic but not free — the network must learn to hold the gate that high on the relevant dimension, and it can only learn that because the gradient is strong enough to inform the update. Extend to 500 steps and even 0.98⁴⁹⁹ = 4 × 10⁻⁵, which is why attention's constant-length path matters at document scale.

The length-dependent gap is the signature of a memory limitation, not a modelling error. At 300 words even an LSTM's forget gate cannot hold early content: at f = 0.98, 0.98³⁰⁰ ≈ 0.002. Long reviews also tend to be structurally different — more qualification, contrast and late reversals ("...but the ending ruined it") — so the decisive evidence is often distant from wherever the model's attention effectively sits.

Check first that it is length and not confounding. Are long reviews rarer in training, so the model has seen fewer? Are they systematically more mixed in sentiment and therefore genuinely harder for any model? Plot accuracy against length in buckets, and compare against a bag-of-words baseline: if the baseline shows the same drop, the difficulty is in the data rather than the architecture.

The fixes, in rough order of effort. Make it bidirectional, so the end of the review is read by a backward pass and no position is 300 steps from the output — cheap, and often most of the gain. Add attention over the hidden states, so the classifier can look directly at any position rather than relying on the final state, which addresses the cause precisely. Or replace the model with a pretrained transformer and fine-tune it, per section 5.3, which is what practice would actually do.

What will not help: a larger hidden size, more layers, or more training. The bottleneck is the length of the path from an early word to the output, and none of those shortens it.

Weight sharing, and the assumption behind it. Section 4.2's convolution shares one kernel across every spatial position, encoding the assumption that a pattern means the same thing wherever it appears. A recurrent cell shares one transition function across every timestep, encoding the assumption that the rule for updating a summary is the same at every point in the sequence. Both buy the same two things: a parameter count independent of input size, and the ability to generalise a pattern learned at one position to all the others. Both are hard constraints that are valuable exactly to the extent that they are true, which is 4.2's closing argument and 5.1's depth box.

The multiplicative-chain failure, and the additive fix. Section 4.3 showed a gradient crossing L layers is a product of L factors, and that a product of factors below 1 dies. Section 5.2's residual block answered it with y = F(x) + x, whose derivative is ∂F/∂x + 1. The LSTM answers the identical problem in time with cᵗ = f cᵗ₋₁ + i g, whose derivative is f. Both replace a transformation with an accumulation, and both give the signal a path whose local derivative is at or near 1. Depth and time are the same axis as far as the gradient is concerned, and the fix is the same fix.

What the pattern predicts. Once you see it, the next move is visible: if an additive path with derivative 1 rescues a chain of length T, a direct path of length 1 would rescue it entirely. That is attention, and it is why the depth box above belongs in this section rather than as an aside. The progression — multiply, then add, then connect directly — is the architectural history of the 2010s in three steps.

A loss of 1.2040 against a chance value of 1.3863 says this prediction is barely better than a uniform guess, which the probabilities confirm — the top two are 0.35 and 0.30. The gradient pushes the true logit up and all three others down, hardest against class 2, the confident wrong answer.

The argmax fails on images 10 and 12 not because the hook score is too low but because the bar score is higher — 0.35 against 0.35 on image 10, where the tie breaks the wrong way, and 0.35 against 0.40 on image 12. The hook evidence was perfectly adequate; it lost a comparison.

Which points at the practical fix from 2.4.4: the decision rule need not be argmax. Per-class thresholds, or a per-class scaling applied before the comparison, convert this from a modelling problem into a one-line calibration change. And note the caveat — thresholds must be chosen on validation data and reported as such, or you have tuned on the test set.

The gap between the two averages is the finding, and it points straight at the rare classes. Micro pools every prediction, so the 45% class dominates it; macro weights all eight classes equally, so a rare class scoring poorly drags it down. A micro of 0.99 with a macro of 0.94 means the common classes are being handled superbly and at least one rare class is not.

What to do about it. Report the per-class AUCs rather than either average — the mean of eight numbers is hiding which one is the problem, and eight numbers fit in a table. Report per-class recall alongside, since a rare class can have a respectable AUC and still almost never be predicted by the argmax, exactly as hook did in Worked 5.5. And put confidence intervals on the rare classes: at 2% of a 2,000-example validation set that is 40 examples, so its AUC is estimated from very little and could move by several points on resampling.

And decide which average the task calls for before quoting one. If the rare classes are rare but important — a rare disease, a rare fault — macro is the honest headline and micro is close to misleading. If the goal is aggregate throughput, micro is defensible. Quoting whichever is higher is not.

Unchanged: everything computed from a confusion matrix. Per-class precision, recall and F1 are defined identically once you fix a class as positive and pool the rest; the confusion matrix simply becomes K×K instead of 2×2, and reading it by row gives recall and by column gives precision exactly as in 2.4.1. Cross-entropy generalises directly, with ln K replacing ln 2 as the chance baseline. Calibration and 2.4.6's temperature scaling apply without modification, since a single T divides all K logits.

Needs a decision: how to aggregate. Binary gave one number; multi-class gives K, and combining them requires a choice. Macro treats classes equally, micro treats predictions equally, and weighted averaging sits between. There is no default that is right for every task, so the choice must be stated and justified — and this is the same structure as 2.4.2's β in the F-score, where the metric embeds a value judgement that has to be declared.

Genuinely new: competition between classes. In a binary problem the score and the decision are the same thing up to a threshold, so AUC and accuracy could not disagree the way they do here. With K classes the decision is a comparison across K scores, and a class can be ranked perfectly within its own column while losing every comparison — which is exactly Worked 5.5's 1.0000 against 0.8333. Nothing in Unit 2B has this shape, and it is the one place where the binary intuition genuinely misleads.

The practical consequence closes the loop with 2.4.4: because the decision is a comparison rather than a threshold, the per-class thresholds become a tunable object in their own right, and adjusting them is often a far cheaper improvement than retraining.

The same 88% as Worked 5.2a at 256 channels, which is no coincidence: the ratio depends on the compression factor (4× here) and not on the absolute width. That is why the bottleneck pattern scales to any channel count and why every architecture after 2015 uses it.

Which of the two happens depends on where the units sit, and that is the honest answer. Early in training, with small weights and pre-activations near zero, tanh′ is close to 1 and the gradient explodes — expect a loss spike or nan within the first epochs, per 4.4. Once the units saturate, tanh′ falls and the same network vanishes instead.

So a recurrent network can suffer both failures in one run, at different times, which is why wₕ > 1 is not a fix for vanishing gradients. Clip the gradients to handle the explosion; use gates to handle the vanishing. The two problems need different tools.

Small dataset, different domain — the hardest quadrant. Freeze most of the backbone and retrain only the last block plus a new head, at a low learning rate, after first training the head alone.

Caution 1: augmentation is different here. Satellite imagery has no canonical orientation, so rotations by any angle and both flips are valid and are unusually valuable — unlike photographs, where an upside-down car is not a normal input. This is close to free extra data.

Caution 2: the split must respect geography. Tiles cut from the same scene are near-duplicates, so a random split puts nearly identical images in train and test and inflates the score badly. Split by scene or region, which is 2.1's leakage rule in its satellite form.

And note that ImageNet features transfer less well here than for photographs — overhead imagery has different scale, texture and colour statistics — so consider a backbone pretrained on remote-sensing data if one is available, or self-supervised pretraining on your own unlabelled tiles.

The cell is holding on to something and largely ignoring the present. The forget gate at 0.95 retains almost all of the previous state, and the input gate at 0.10 admits only a tenth of a candidate that points the other way — so the state barely moves, from −1.500 to −1.345. The output gate at 0.60 then exposes about 60% of the (saturated) cell to the rest of the network.

This is exactly the configuration Worked 5.4b idealised: a unit that latched a value earlier in the sequence and is now protecting it. Over 30 more steps at f = 0.95 the retained fraction would be 0.95³⁰ = 0.21, so the memory is long but not permanent.

A loss of 0.9163 against a chance value of 1.0986 says the model has learned something but very little on this example. Note that the gradient magnitude on the true class, 0.60, is larger than on either wrong class, so the update pushes hardest in the right direction — and that the near-tie between 0.45 and 0.40 is exactly the situation Worked 5.5 showed can produce perfect AUC with imperfect accuracy.

28 input pixels. The final 5×5 contributed 16 of those 28 on its own, because by then each of its inputs summarised a 4-pixel step. That is the general principle worth carrying: a kernel late in a network is worth far more receptive field than the same kernel early, which is why large kernels near the input are wasteful and why modern networks use 3×3 throughout and buy reach with downsampling instead.

Two things are happening at once and both need fixing.

Catastrophic forgetting. Fine-tuning made things worse than freezing, which is the signature of a learning rate that destroyed the pretrained features. If they trained at the head's 10⁻³ rather than 10⁻⁵, the backbone moved far from its pretrained solution within the first epoch, and what remains is effectively a randomly initialised network trained on a small dataset. Check also whether the backbone was unfrozen from step one with a random head attached, which sends meaningless gradients into good weights.

Overfitting. 99% training against 44% validation is a 55-point gap, consistent with millions of newly trainable parameters against a dataset that only ever supported the head.

What to do. Return to feature extraction and confirm 71% reproduces, which establishes the baseline. Then unfreeze only the last block, at 10⁻⁵, after the head has converged, and check that validation improves rather than assuming it will. Verify the BatchNorm layers are genuinely in inference mode and the preprocessing matches the source model. And accept the possible conclusion: with a small dataset, feature extraction may simply be the better strategy, and 71% may be the honest ceiling until more data arrives.

Why it helps. A bidirectional layer runs two independent recurrent passes — one left to right, one right to left — and concatenates their states. Every position therefore has access to context from both directions, and no position is far from some output: a word at the start of a 300-word review is 300 steps from the end of the forward pass but only 1 step from the start of the backward pass. That halves the effective path length and, for classification tasks where the whole input is available, is usually one of the cheapest available improvements. It doubles the parameters and the compute.

Why it cannot stream. The backward pass starts at the last element, so it cannot begin until the sequence has ended. For live transcription there is no last element — the audio is still arriving — so the model would have to wait for the speaker to stop before producing any output at all. The latency is unbounded and the architecture is simply unavailable, however well it scores offline.

What is used instead. A unidirectional model, or a bidirectional one over a fixed lookahead window of a few hundred milliseconds, which buys some right-context at a bounded and acceptable latency cost. This is a good example of a constraint that comes from deployment rather than from the data, and it is exactly the kind of thing 4.8's reporting checklist should surface before a model is chosen.

The pattern is Worked 5.5's, magnified. A macro AUC of 0.97 says each class's score column ranks its own members near-perfectly — so the representation is good and the model can see all five classes. Meanwhile two classes are recovered 22% and 10% of the time by the argmax. The scores are fine; the comparison between them is not.

The near-certain cause is class imbalance. If classes 4 and 5 are rare, cross-entropy training drives their predicted probabilities down across the board — the model learns that guessing a common class is usually right — so their scores, while correctly ordered, sit below the common classes' scores almost everywhere. Confirm by checking the class frequencies and the mean predicted probability per class.

The fixes, cheapest first. Per-class thresholds or a per-class scaling applied before the argmax, chosen on validation data — this is nearly free and, given AUC 0.97, should recover most of the loss. Then class weighting in the loss, which addresses the cause during training. Then resampling, or focal loss. Retraining the architecture is the last thing to try, because the AUCs say the representation was never the problem.

And report differently regardless. Accuracy 0.62 as a headline conceals that three classes work and two do not. Per-class recall in a table, with the class frequencies beside it, is the honest presentation — 2.4.2's argument, unchanged.

Both replace a multiplicative chain with an additive one, for the same reason and with the same effect on the gradient.

A plain deep network composes transformations, so the gradient from the loss to layer 1 is a product of L local derivatives; section 4.3 measured what that costs. A plain recurrent network composes the same transformation T times, so the gradient across time is a product of T factors — and Worked 5.4a measured that as a separation of 0.000039 after five steps.

The residual block writes y = F(x) + x, giving ∂y/∂x = ∂F/∂x + 1. The LSTM writes cᵗ = f cᵗ₋₁ + i g, giving ∂cᵗ/∂cᵗ₋₁ = f. In both cases there is now a path whose local derivative is at or near 1 regardless of what the learned transformation does, so the product across many steps stays near 1 instead of collapsing. Worked 5.4b's cell holds a separation of 1.5232 at every step, against the plain RNN's 0.000039 — the same rescue the skip connection performs for depth.

The next step was to remove the chain altogether. If an additive path of derivative 1 rescues a chain of length T, a direct connection makes the chain length 1. Attention lets every position read every other position directly, weighting them by a softmax over similarity — so nothing has to survive a sequence of updates and, as a bonus, the positions can be computed in parallel rather than one after another. The price is O(T²) time and memory instead of O(T).

Stated as a progression: multiply, then add, then connect directly. Each step shortens the path from a signal to the place it is needed, and each was the dominant architectural idea of roughly half a decade.