Questions and answers by demonstration

Every figure from every demonstration, numbered as its own article numbers it, and every held-out question of the ten-skill study with the model's complete answer. Figures regenerate with python3 plots/build.py in the replication repository.

Contents

Demonstration 1 : Ten skills, one after another Figure 1. Accuracy and retention: every skill at every checkpoint Figure 2. Ten skills, one stream: training loss per update Figure 3. Each skill during its own teach: training loss and held-out loss Figure 4. Each skill during its own teach, zoomed: every panel has its own loss Figure 5. What happens to a skill after its teach ends Figure 6. Accuracy on the fixed 225-row panels at every accepted checkpoint Figure 7. Own-end score versus final score, per skill Figure 8. How much each skill was trained Figure 9. Held-out loss does not predict exact-match accuracy Figure 10. General capability across the stream Figure 11. Oriel from teach 9 to teach 10: which part of the answer changed Demonstration 2 : Four domains, against LoRA Figure 1. Four domains in sequence: training loss for three conditions Figure 2. The unsmoothed training loss, one panel per condition Figure 3. What each condition forgot, and what it learned Figure 4. The never-trained control bank Figure 5. Held-out loss inside each domain, all three conditions Demonstration 3 : Two invented languages Figure 1. Two invented languages, taught one after the other in one pass Figure 2. The same two corpora through a single LoRA adapter, in sequence Figure 3. Generation identity after each stage: Learner 1.0 versus LoRA Fact demonstrations Figure 1. Everything above the dashed line happens before you commit to training Figure 3. Teaching does not trade away general capability at any point in a lear Every held-out question and answer

Demonstration 1 · Ten skills, one after anotheras published in the ten-skill report

Figure 1. One picture of the run: nothing is answered before teaching, each skill jumps at its own teach, and every skill holds its level while the skills after it are taught. 1,525 of 2,250 panel rows are correct at the end, from 0 at the start, and the base line stays flat throughout.

Figure 2. Training loss over the whole stream: each skill starts high, drops within its own segment, and hands over to the next. Nothing in the stream tells the learner where one skill ends and the next begins.

Figure 3. Per-skill training and held-out loss during each skill's own segment, on one shared scale. Held-out loss falls with training on all ten and settles at very different levels: near zero for SCAN and COGS, far higher for Oriel and Juniper.

Figure 4. The same ten panels with a separate loss scale per panel, to show the shape of each curve. Levels are not comparable across panels in this view; use the shared-scale figure for that.

Figure 5. Retention as a loss curve: after a skill's own segment ends, its held-out loss is nearly flat across the remaining tens of thousands of updates on other skills. The English control panel (never trained) stays where it started.

Figure 6. The checkpoint-by-skill matrix: each skill is scored from its own teach onward, so cells above the diagonal are blank. Read down a column to see a skill's score while later skills are taught: SCAN 222 at every one of ten checkpoints; Meridian moves within 81 to 84; Oriel is the only column that moves by more than two rows.

Figure 7. All ten skills hold what they learned: nine end within two rows of their own teach, and for Oriel the four working components survive, with the header and escaping right on every row at the end and the payload and decoded length on 92 to 95 per cent of them. Its exact-match count still moves 12 to 9 of 225 because one wrong checksum costs the whole row. Juniper and Meridian end slightly higher than at their own teach.

Figure 8. Training dose per skill. COGS received a third of all updates and half of all supervised tokens; the other nine received between 2,835 and 9,850 updates each.

Figure 9. A low held-out loss is necessary but not sufficient for exact-match accuracy: Oriel and Meridian sit at loss levels similar to skills that score far higher, because a single wrong token anywhere in a long structured answer costs the whole row.

Figure 10. The base does not move materially on the measured benchmarks: the 498-item sentinel stays between 348 and 353 correct across all eleven checkpoints, and the full battery after ten teaches reads MMLU 0.8654, ARC 0.7099, WinoGrande 0.7885 against 0.8634, 0.7056, 0.7948 at step zero.

Figure 11. The Oriel decline is localized: between the two checkpoints the header, transformed payload, escaping and decoded length stay correct on essentially all rows, while the final checksum value is right on 15 rows, then 11. Seven rows lost that one value; one row gained it.

Demonstration 2 · Four domains, against LoRAas published in Plasticity Without Forgetting

Figure 1. Three conditions trained on the same four-domain stream. All three learn each domain in turn; the difference between them shows up afterwards, in what they keep.

Figure 2. The raw per-step training loss behind the smoothed stream figure, for all three conditions on identical axes.

Figure 3. Both Learner 1.0 conditions keep the earlier domains within a few thousandths of a nat while LoRA r256 gives back 0.07, 0.06 and 0.03 nats; all three learned each domain by about the same amount.

Figure 4. On 8 never-trained control windows the two Learner 1.0 conditions move by about 0.01 nats across the stream and sit about 0.12 nats below their recorded base reference. The figure does not isolate how much of that difference arose inside the displayed stream.

Figure 5. Held-out loss while each domain is trained, against the recorded base reference (dashed, value shown). The Learner 1.0 conditions are close to their final level by the first measurement at step 25; the LoRA condition gets there over the 300 steps. All three end each domain at a similar level.

Demonstration 3 · Two invented languagesas published in the two-languages report

Figure 1. Learner 1.0, one-pass recording. Velenic's held-out loss falls from 4.6625 to 2.8880 nats during its own teach and reads 2.8871 after Morvath is taught. Morvath's falls from 4.3636 to 2.2842 during its teach.

Figure 2. A single adapter learns language 1 (4.35 to 2.22 nats), then learning language 2 pushes language 1 to 6.00 nats, worse than before it was ever taught. English is untouched: the damage is between the two taught skills.

Figure 3. Where the LoRA adapter answers language-1 prompts in language 2 after the second teach (0.96), Learner 1.0 keeps the two apart (0.00 in every cell).

Fact demonstrationsas published in the facts page

Figure 1. Everything above the dashed line happens before you commit to training, and you can read and edit its output. The second path, below the line, has no preparation step and needs none.

Figure 3. Teaching does not trade away general capability at any point in a learner's history. The two language checkpoints sit 0.0004 and 0.0013 below the base model's row-mean, which is a fifth of a percentage point at most. The three sequence checkpoints sit 0.0014, 0.0003 and 0.0001 above it. There was never a dip to recover from, because there was never a dip.

Every held-out question and answer

Two sets: final = the 2,250 fixed-panel rows (225 per skill) answered by the checkpoint after the tenth teach; own-end = the same 2,250 rows answered by the checkpoint at the end of each skill's own teach. Greedy decoding, exact match against the frozen target, scorer per skill. Two PCFG rows hit the generation cap and count as wrong. Search matches prompt, target, output and id.

set skill result search

id	prompt	target	model output	result