Learner LabsLearner 1.0Foundation models that continually learn

Ten skills, one model, one pass

We taught our Learner 1.0 model ten unrelated skills one after another, one training example per update, thousands of examples per skill, with no replay and no label telling it where one skill ended and the next began. Every acquired skill is retained through all the sequential teaches and the base eval holds constant showing no loss of base capabilities.

Model Learner 1.0Trainable parameters 1.14B, fixed for the whole runStream 63,282 updates, one example each, in a fixed orderTokens trained on 5,921,663 presented, 2,295,371 supervised (answer tokens including end-of-sequence)Held-out evaluation 225 fixed rows per skill, greedy decoding, exact match, at the skill's own-end checkpoint and every later oneBase evaluation 498-item sentinel at every checkpoint; 16,481-item likelihood battery at teaches 0, 5 and 10Final checkpoint 3264e0b2Compute 1 H200

1. The question and what was measured

When supervised skills are taught in sequence to one model with a fixed number of trainable parameters, does teaching the later ones erase the earlier ones? The measurement is the count of exactly correct answers on a fixed 225-row held-out panel for every skill, taken at the checkpoint right after that skill's own teach and again after every later teach.

Accuracy and retention: every skill at every checkpoint measured Every skill on its own 225-row panel Ten skills taught one after another. A line starts at the checkpoint where its skill was taught and runs to the end of the stream; the base is scored at all eleven checkpoints. 0% 25% 50% 75% 100% 0 1 2 3 4 5 6 7 8 9 10 checkpoint (0 = before any teaching) percent correct base all ten skills: 0 before teaching taught at checkpoint SCAN 1 Meridian 2 Adyghe G2P 3 Oriel 4 COGS 5 Sable 6 PCFG 7 Tessel 8 Juniper 9 Bracken 10 base, 498 items A skill is scored only from its own teach onward, so nothing is drawn to the left of its first dot. Before any teaching the base answers 0 of the 2,250 panel rows, so every point above the axis was acquired during the run. Each skill is then scored on its own 225-row held-out panel at every checkpoint from its teach onward; it is not scored in between. After all ten teaches 1,525 of 2,250 are correct. The base line is the 498-item sentinel on its own denominator. Source: plots/data/derived/tables.json.
Figure 1. One picture of the run: nothing is answered before teaching, each skill jumps at its own teach, and every skill holds its level while the skills after it are taught. 1,525 of 2,250 panel rows are correct at the end, from 0 at the start, and the base line stays flat throughout.

Final panel scores after the tenth teach, in the taught order: SCAN 222, Meridian 84, Adyghe G2P 146, Oriel 9, COGS 225, Sable 198, PCFG SET 172, Tessel 188, Juniper 120, Bracken 161, each of 225. Against each skill's own-end score the changes are 0, +1, −1, −3, 0, 0, −2, 0, +2, 0. The one skill that moved by more than two rows, Oriel, was also the one the model never learned well: 12 of 225 at its own teach.

Acquisition varied widely by skill, from 12 to 225 of 225 at own-end. That spread is part of the result. The retention claim is about the skills that were acquired, and it is a claim about this stream, this order and this budget, not about an unlimited horizon.

Own-end score versus final score, per skill measured right after its own teach after all ten teaches Correct of 225 on each skill's fixed held-out panel the two marks coincide when nothing changed 0 45 90 135 180 225 correct of 225 SCAN 222 → 222 (no change) Meridian 83 → 84 (+1) Adyghe G2P 147 → 146 (-1) Oriel 12 → 9 (-3) COGS 225 → 225 (no change) Sable 198 → 198 (no change) PCFG 174 → 172 (-2) Tessel 188 → 188 (no change) Juniper 118 → 120 (+2) Bracken 161 → 161 (no change) Own-end = the checkpoint at the end of that skill's segment; final = the checkpoint after the tenth teach. Source: plots/data/derived/tables.json.
Figure 2. All ten skills hold what they learned: nine end within two rows of their own teach, and for Oriel the four working components survive, with the header and escaping right on every row at the end and the payload and decoded length on 92 to 95 per cent of them. Its exact-match count still moves 12 to 9 of 225 because one wrong checksum costs the whole row. Juniper and Meridian end slightly higher than at their own teach.

2. Setup

Model. Learner 1.0, with 1.14 billion trainable parameters, fixed before the first skill and never grown. Everything else about the learning mechanism is proprietary and is not described here.

Objective. Cross-entropy on the answer tokens only, including the native end-of-sequence token, so the model is also trained to stop.

Data order. Ten skills in a fixed order (SCAN, Meridian, Adyghe G2P, Oriel, COGS, Sable, PCFG SET, Tessel, Juniper, Bracken), each as a contiguous block of rows. One pass: no row is presented twice. No rows from an earlier skill are replayed during a later one.

Batch and accumulation. Batch size one, no gradient accumulation: every update is a single example. Gradient clipping at 1.0. Each teach resumed from the checkpoint the previous accepted teach ended on.

What the learner sees. Only the example text: the prompt and the answer. No skill label, boundary marker or teach index is given to it. The colored bands in the figures are the experiment design, not an input.

Checkpoints and monitoring. A full resumable checkpoint at the end of every skill (and at quarter points within it). During training, a held-out loss was measured every 25 updates on eight fixed panel rows per monitored skill; this monitor is separate from the final evaluation and is reported as a curve, not a score.

Generation and scoring. Greedy decoding with a per-skill token cap set from the longest target in the skill; a row is correct only if the whole answer matches the frozen target under that skill's scorer (section 5).

Compute. Training ran on 1 H200. The ten teaches were not one uninterrupted job (section 10).

3. What one sample is

A sample is one prompt and one target answer. The model reads the prompt, is trained on every token of the answer including the end-of-sequence token, and the resulting gradient is applied as one update. Then the next sample. Here is the first training row of the fourth skill, exactly as stored:

Apply the oriel packet codec procedure to the input. Return only a JSON list of integers, without explanation.
Input: {"payload":[9,3,2,13,6,13,12,12]}
Answer:

Target (all 21 integers are supervised, plus the end-of-sequence token):

[15,13,8,14,12,8,10,7,7,2,8,15,13,3,2,8,1,5,1,5,9]
schematic example kprompt + answer tokens one updateloss over every answer token example k+1next row in the fixed order Batch size 1, no accumulation, no replay: 63,282 such steps in total.
Figure 3. One example, one update, next example. The number of answer tokens differs per row, so updates are not equal in size; the tables report both rows and supervised tokens.

Counting. Two counts appear in this article. Updates is the number of single-example updates the final checkpoint was trained with, 63,282. Supervised tokens are the answer tokens the loss was computed on, including end-of-sequence. Examples differ in length, so the tables report both.

The training data. Every training row of all ten skills, in the taught order, is in the replication repository: ten_skills/data/train. The held-out panels are in ten_skills/data/evals. The files are there to be read and re-scored; the repository does not run this training.

4. The ten skills

Four are public datasets (SCAN, Adyghe grapheme-to-phoneme, COGS, PCFG SET) and six are procedures written for this experiment so that the base model could not know them in advance (Meridian, Oriel, Sable, Tessel, Juniper, Bracken). For the written procedures the prompt states only the input and the output format; the rules below were never shown to the learner. Each card shows one real training row and one or two real held-out rows with the model's actual answer at the end of that skill's teach and after all ten teaches. Examples are chosen for length and structure, not for success; each skill with a wrong final answer shows one.

1. SCAN: compositional command-to-action translation (public dataset)

A navigation command in a small controlled language (walk, run, jump, look, turn left/right, around, opposite, twice, thrice, and, after) is translated into the exact sequence of primitive actions. Training rows come from the public SCAN length-split training file; the fixed panel rows come from the released retention bank, which has zero training-group overlap by the release manifest. Scored as an exact match of the whole action sequence.

Updates 9,850Supervised tokens 297,232 (presented 499,982) Own-end 222/225 (98.7%)Final 222/225 (98.7%) Scorer scan_actions: exact match of the complete action sequenceData training file · held-out panel (225 fixed rows)

A real training row (scan/train/16767:ce_v2)

Request:
Translate this command into actions: run around left twice after walk right thrice
Response:

Target:

I_TURN_RIGHT I_WALK I_TURN_RIGHT I_WALK I_TURN_RIGHT I_WALK I_TURN_LEFT I_RUN I_TURN_LEFT I_RUN I_TURN_LEFT I_RUN I_TURN_LEFT I_RUN I_TURN_LEFT I_RUN I_TURN_LEFT I_RUN I_TURN_LEFT I_RUN I_TURN_LEFT I_RUN

A held-out question (scan/dev/9952:ce_v2)

Request:
Translate this command into actions: walk thrice and look around right
Response:

Target:

I_WALK I_WALK I_WALK I_TURN_RIGHT I_LOOK I_TURN_RIGHT I_LOOK I_TURN_RIGHT I_LOOK I_TURN_RIGHT I_LOOK

Answer at the end of its own teach and after all ten teaches (identical): correct

I_WALK I_WALK I_WALK I_TURN_RIGHT I_LOOK I_TURN_RIGHT I_LOOK I_TURN_RIGHT I_LOOK I_TURN_RIGHT I_LOOK

A held-out question the model got wrong (scan/dev/15105:ce_v2)

Request:
Translate this command into actions: turn around left thrice after turn opposite left thrice
Response:

Target:

I_TURN_LEFT I_TURN_LEFT I_TURN_LEFT I_TURN_LEFT I_TURN_LEFT I_TURN_LEFT I_TURN_LEFT I_TURN_LEFT I_TURN_LEFT I_TURN_LEFT I_TURN_LEFT I_TURN_LEFT I_TURN_LEFT I_TURN_LEFT I_TURN_LEFT I_TURN_LEFT I_TURN_LEFT I_TURN_LEFT

Answer at the end of its own teach and after all ten teaches (identical): wrong

I_TURN_LEFT I_TURN_LEFT I_TURN_LEFT I_TURN_LEFT I_TURN_LEFT I_TURN_LEFT I_TURN_LEFT I_TURN_LEFT I_TURN_LEFT I_TURN_LEFT I_TURN_LEFT I_TURN_LEFT I_TURN_LEFT I_TURN_LEFT I_TURN_LEFT I_TURN_LEFT I_TURN_LEFT I_TURN_LEFT I_TURN_LEFT I_TURN_LEFT

2. Meridian: stateful financial-event accounting (new procedure)

Process a list of events in order from balance 0 and reserve 0. A deposit first pays off reserve debt, then credits the remainder to balance. A purchase costs its amount plus a fee of 3; if the balance covers it, debit and count it accepted, otherwise raise the reserve by 2 and count it rejected without paying. A refund credits the amount minus 1 (minimum zero). Return balance, reserve, accepted and rejected. The prompt gives only the input and output format; the rules are never shown to the learner.

Updates 3,557Supervised tokens 68,159 (presented 349,925) Own-end 83/225 (36.9%)Final 84/225 (37.3%) Scorer typed JSON procedure: parsed JSON compared field by field against the oracleData training file · held-out panel (225 fixed rows)

A real training row (meridian_settlement:candidate_train:01569)

Apply the meridian settlement procedure to the input. Return only a JSON object with integer fields balance, reserve, accepted, rejected, without explanation.
Input: {"events":[["refund",19],["purchase",18],["deposit",6],["deposit",18],["purchase",27],["purchase",25],["refund",9]]}
Answer:

Target:

{"accepted":1,"balance":18,"rejected":2,"reserve":2}

A held-out question (meridian_settlement:retention:00454)

Apply the meridian settlement procedure to the input. Return only a JSON object with integer fields balance, reserve, accepted, rejected, without explanation.
Input: {"events":[["purchase",29],["deposit",37],["purchase",25],["purchase",24],["refund",21]]}
Answer:

Target:

{"accepted":1,"balance":27,"rejected":2,"reserve":2}

Answer at the end of its own teach and after all ten teaches (identical): correct

{"accepted":1,"balance":27,"rejected":2,"reserve":2}

A held-out question the model got wrong (meridian_settlement:retention:00195)

Apply the meridian settlement procedure to the input. Return only a JSON object with integer fields balance, reserve, accepted, rejected, without explanation.
Input: {"events":[["refund",12],["deposit",4],["refund",19],["refund",31],["refund",38],["deposit",5],["deposit",3],["deposit",22],["deposit",30]]}
Answer:

Target:

{"accepted":0,"balance":160,"rejected":0,"reserve":0}

Answer at the end of its own teach and after all ten teaches (identical): wrong

{"accepted":0,"balance":159,"rejected":0,"reserve":0}

3. Adyghe grapheme-to-phoneme (public dataset)

Transcribe a written Adyghe word into IPA under the SIGMORPHON 2020 convention. Adyghe is a Northwest Caucasian language with a large consonant inventory and long orthographic words, so a single wrong segment fails the row.

Updates 3,600Supervised tokens 42,122 (presented 232,037) Own-end 147/225 (65.3%)Final 146/225 (64.9%) Scorer exact match of the space-separated IPA segments (character-edit diagnostics recorded separately)Data training file · held-out panel (225 fixed rows)

A real training row (g2p_ady:train:3071)

Transcribe this Adyghe word into IPA using the SIGMORPHON 2020 grapheme-to-phoneme convention. Return only the transcription, with spaces between phoneme tokens.
Word: чъыгшъхьацыз
Answer:

Target:

t͡ʃ ə ɣ ʂ ħ aː t͡ʃ ə z

A held-out question (g2p_ady:test:255)

Transcribe this Adyghe word into IPA using the SIGMORPHON 2020 grapheme-to-phoneme convention. Return only the transcription, with spaces between phoneme tokens.
Word: мэузын
Answer:

Target:

m a w z ə n

Answer at the end of its own teach and after all ten teaches (identical): correct

m a w z ə n

A held-out question the model got wrong (g2p_ady:test:281)

Transcribe this Adyghe word into IPA using the SIGMORPHON 2020 grapheme-to-phoneme convention. Return only the transcription, with spaces between phoneme tokens.
Word: ожъубанэхьэмлыу
Answer:

Target:

w a ʐʷ b aː n a ħ a m l ə w

Answer at the end of its own teach and after all ten teaches (identical): wrong

w a ʐʷ b aː n a ħ a m ɮ ə w

4. Oriel: binary protocol encoding with escaping and a checksum (new procedure)

Encode a list of payload values 0 to 15. Start with a header [11, number of values]. For each value x emit two nibbles, (x+5) mod 16 and (3x+1) mod 16. Append a checksum: the sum of (position+1) times payload[position], modulo 16. Then escape every nibble equal to 11 or 15 as the pair [15, nibble XOR 6] without escaping the escape nibbles again. Return the final integer list. A row is correct only when the complete list is exactly right.

Updates 3,568Supervised tokens 179,890 (presented 349,908) Own-end 12/225 (5.3%)Final 9/225 (4.0%) Scorer typed JSON procedure: exact list match; component diagnostics (escaping, header, payload, checksum, length) recordedData training file · held-out panel (225 fixed rows)

A real training row (oriel_packet_codec:candidate_train:02363)

Apply the oriel packet codec procedure to the input. Return only a JSON list of integers, without explanation.
Input: {"payload":[9,3,2,13,6,13,12,12]}
Answer:

Target:

[15,13,8,14,12,8,10,7,7,2,8,15,13,3,2,8,1,5,1,5,9]

A held-out question (oriel_packet_codec:retention:00442)

Apply the oriel packet codec procedure to the input. Return only a JSON list of integers, without explanation.
Input: {"payload":[1,10,3,1,5,4,4]}
Answer:

Target:

[15,13,7,6,4,15,9,15,9,8,10,6,4,10,0,9,13,9,13,15,9]

Answer at the end of its own teach and after all ten teaches (identical): correct

[15,13,7,6,4,15,9,15,9,8,10,6,4,10,0,9,13,9,13,15,9]

A held-out question the model got wrong (oriel_packet_codec:retention:00115)

Apply the oriel packet codec procedure to the input. Return only a JSON list of integers, without explanation.
Input: {"payload":[10,10,12,6,1,2,10,14,0]}
Answer:

Target:

[15,13,9,15,9,15,9,15,9,15,9,1,5,15,13,3,6,4,7,7,15,9,15,9,3,15,13,5,1,1]

Answer at the end of its own teach and after all ten teaches (identical): wrong

[15,13,9,15,9,15,9,15,9,15,13,3,15,13,3,6,4,7,7,15,9,15,9,3,15,13,5,1,15,13]

5. COGS: sentence to logical form (public dataset)

Translate an English sentence into its COGS logical form. The training rows are in-distribution rows of the public corpus; the fixed panel is drawn from the released retention bank with zero training-group overlap. Not the official generalization split; this run measured in-distribution acquisition and retention.

Updates 21,309Supervised tokens 1,099,479 (presented 1,799,981) Own-end 225/225 (100.0%)Final 225/225 (100.0%) Scorer cogs_sequence: exact match of the logical formData training file · held-out panel (225 fixed rows)

A real training row (cogs_semantics:train:02979)

Translate the sentence into its COGS logical form. Return only the logical form, without explanation.
Sentence: A melon was improved by a moose .
Answer:

Target:

melon ( x _ 1 ) AND improve . theme ( x _ 3 , x _ 1 ) AND improve . agent ( x _ 3 , x _ 6 ) AND moose ( x _ 6 )

A held-out question (cogs_semantics:dev:00690)

Translate the sentence into its COGS logical form. Return only the logical form, without explanation.
Sentence: A cake was given to the giraffe .
Answer:

Target:

* giraffe ( x _ 6 ) ; cake ( x _ 1 ) AND give . theme ( x _ 3 , x _ 1 ) AND give . recipient ( x _ 3 , x _ 6 )

Answer at the end of its own teach and after all ten teaches (identical): correct

* giraffe ( x _ 6 ) ; cake ( x _ 1 ) AND give . theme ( x _ 3 , x _ 1 ) AND give . recipient ( x _ 3 , x _ 6 )

No held-out row was wrong at the final checkpoint (225 of 225).

6. Sable: a finite-state transaction protocol (new procedure)

Start IDLE with pending, completed and errors at 0. BEGIN in IDLE opens a transaction, elsewhere it is an error. ADD(n) in OPEN adds n to pending, elsewhere an error. COMMIT in OPEN with pending at least 5 moves pending into completed and returns to IDLE; any other COMMIT is an error and leaves the state unchanged. CANCEL in OPEN clears pending and returns to IDLE, elsewhere an error. Return state, pending, completed, errors.

Updates 3,911Supervised tokens 73,150 (presented 349,982) Own-end 198/225 (88.0%)Final 198/225 (88.0%) Scorer typed JSON procedure: parsed JSON compared field by fieldData training file · held-out panel (225 fixed rows)

A real training row (sable_workflow:candidate_train:01135)

Apply the sable workflow procedure to the input. Return only a JSON object with state (IDLE or OPEN), pending, completed, errors (integers), without explanation.
Input: {"events":[["ADD",3],["BEGIN"],["BEGIN"],["BEGIN"],["ADD",4],["CANCEL"],["CANCEL"]]}
Answer:

Target:

{"completed":0,"errors":4,"pending":0,"state":"IDLE"}

A held-out question (sable_workflow:retention:00447)

Apply the sable workflow procedure to the input. Return only a JSON object with state (IDLE or OPEN), pending, completed, errors (integers), without explanation.
Input: {"events":[["ADD",1],["BEGIN"],["COMMIT"],["ADD",4],["CANCEL"],["CANCEL"]]}
Answer:

Target:

{"completed":0,"errors":3,"pending":0,"state":"IDLE"}

Answer at the end of its own teach and after all ten teaches (identical): correct

{"completed":0,"errors":3,"pending":0,"state":"IDLE"}

A held-out question the model got wrong (sable_workflow:retention:00398)

Apply the sable workflow procedure to the input. Return only a JSON object with state (IDLE or OPEN), pending, completed, errors (integers), without explanation.
Input: {"events":[["BEGIN"],["ADD",5],["COMMIT"],["COMMIT"],["ADD",4],["ADD",5],["ADD",5],["BEGIN"],["COMMIT"]]}
Answer:

Target:

{"completed":5,"errors":5,"pending":0,"state":"OPEN"}

Answer at the end of its own teach and after all ten teaches (identical): wrong

{"completed":19,"errors":2,"pending":0,"state":"IDLE"}

7. PCFG SET: evaluate a compositional string expression (public dataset)

Evaluate an expression built from string-manipulation functions over symbol sequences and return the resulting space-separated symbol sequence. Answers can run to several hundred tokens; the two rows in the whole experiment that hit the generation cap are both from this skill.

Updates 7,871Supervised tokens 195,151 (presented 699,825) Own-end 174/225 (77.3%)Final 172/225 (76.4%) Scorer symbol_sequence: exact match of the whole symbol sequenceData training file · held-out panel (225 fixed rows)

A real training row (pcfg_set:train:77047)

Evaluate this PCFG SET expression. Return only the resulting space-separated symbol sequence, with no explanation.
Expression: prepend remove_first prepend M19 O17 Z6 B5 Z6 , shift I10 A1 , B18 D6 C7 , Q9 O16 X12 C2 V1
Answer:

Target:

Q9 O16 X12 C2 V1 B18 D6 C7

A held-out question (pcfg_set:test:5768)

Evaluate this PCFG SET expression. Return only the resulting space-separated symbol sequence, with no explanation.
Expression: prepend remove_second X15 X20 K18 , remove_first echo echo K18 O14 C20 E11 E5 , F18 E10 K19 L12 , T11 J18 K19
Answer:

Target:

T11 J18 K19 X15 X20 K18

Answer at the end of its own teach and after all ten teaches (identical): correct

T11 J18 K19 X15 X20 K18

A held-out question the model got wrong (pcfg_set:test:6939)

Evaluate this PCFG SET expression. Return only the resulting space-separated symbol sequence, with no explanation.
Expression: repeat swap_first_last echo repeat remove_first shift reverse copy shift shift S11 G6 O17 , echo repeat prepend M7 X12 E1 S16 , shift swap_first_last reverse D3 B14 H19
Answer:

Target:

S16 H19 D3 M7 X12 E1 S16 B14 H19 D3 M7 X12 E1 S16 S16 B14 H19 D3 M7 X12 E1 S16 B14 H19 D3 M7 X12 E1 S16 S16 B14 S16 H19 D3 M7 X12 E1 S16 B14 H19 D3 M7 X12 E1 S16 S16 B14 H19 D3 M7 X12 E1 S16 B14 H19 D3 M7 X12 E1 S16 S16 B14

Answer at the end of its own teach: wrong

H19 B14 D3 M7 X12 E1 S16 H19 B14 D3 M7 X12 E1 S16 H19 B14 D3 M7 X12 E1 S16 H19 B14 D3 M7 X12 E1 S16 H19 B14 D3 M7 X12 E1 S16 H19 B14 D3 M7 X12 E1 S16 H19 B14 D3 M7 X12 E1 S16 H19 B14 D3 M7 X12 E1 S16 H19 B14 D3 M7 X12 E1 S16 H19 B14 D3 M7 X12 E1 S16 H19 B14 D3 M7 X12 E1 S16 H19 B14 D3 M7 X12 E1 S16 H19 B14 D3 M7 X12 E1 S16 H19 B14 D3 M7 X12 E1 S16 H19 B14 D3 M7 X12 E1 S16 H19 B14 D3 M7 X12 E1 S16 H19 B14 D3 M7 X12 E1 S16 H19 B14 D3 M7 X12 E1 S16 H19 B14 D3 M7 X12 E1 S16 H19 B14 D3 M7 X12 E1 S16 H19 B14 D3 M7 X12 E1 S16 H19 B14 D3 M7 X12 E1 S16 H19 B14 D3 M7 X12 E1 S16 H19 B14 D3 M7 X12 E1 S16 H19 B14 D3 M7 X12 E1 S16 H19 B14 D3 M7 X12 E1 S16 H19 B14 D3 M7 X12 E1 S16 H19 B14 D3 M7 X12 E1 S16 H19 B14 D3 M7 X12 E1 S16 H19 B14 D3 M7 X12 E1 S16 H19 B14 D3 M7 X12 E1 S16 H19 B14 D3 M7 X12 E1 S16 H19 B14 D3 M7 X12 E1 S16 H19 B14 D3 M7 X12 E1 S16 H19 B14 D3 M7 X12 E1 S16 H19 B14 D3 M7 X12 E1 S16 H19 B14 D3 M7 X12 E1 S16 H19 B14 D3 M7 X12 E1 S16 H19 B14 D3 M7 X12 E1 S16 H19 B14 D3 M7 X12 E1 S16 H19 B14 D3 M7 X12 E1 S16 H19 B14 D3 M7 X12 E1 S16 H19 B14 D3 M7 X12 E1 S16 H19 B14 D3 M7 X12 E1 S16 H19 B14 D3 M7 X12 E1 S16 H19 B14 D3 M7 X12 E1 S16 H19 B14 D3 M7 X12 E1 S16 H19 B14 D3 M7 X12 E1 S16 H19 B14 D3 M7 X12 E1 S16 H19 B14 D3 M7 X12 E1 S16 H19 B14 D3 M7 X12 E1

Answer after all ten teaches: wrong

H19 B14 D3 M7 X12 E1 S16 H19 B14 D3 M7 X12 E1 S16 H19 B14 D3 M7 X12 E1 S16 H19 B14 D3 M7 X14 E1 S16 H19 B14 D3 M7 X12 E1 S16 H19 B14 D3 M7 X12 E1 S16 H19 B14 D3 M7 X12 E1 S16 H19 B14 D3 M7 X12 E1 S16 H19 B14 D3 M7 X12 E1 S16 H19 B14 D3 M7 X12 E1 S16 H19 B14 D3 M7 X12 E1 S16 H19 B14 D3 M7 X12 E1 S16 H19 B14 D3 M7 X12 E1 S16 H19 B14 D3 M7 X12 E1 S16 H19 B14 D3 M7 X12 E1 S16 H19 B14 D3 M7 X12 E1 S16 H19 B14 D3 M7 X12 E1 S16 H19 B14 D3 M7 X12 E1 S16 H19 B14 D3 M7 X12 E1 S16 H19 B14 D3 M7 X12 E1 S16 H19 B14 D3 M7 X12 E1 S16 H19 B14 D3 M7 X12 E1 S16 H19 B14 D3 M7 X12 E1 S16 H19 B14 D3 M7 X12 E1 S16 H19 B14 D3 M7 X12 E1 S16 H19 B14 D3 M7 X12 E1 S16 H19 B14 D3 M7 X12 E1 S16 H19 B14 D3 M7 X12 E1 S16 H19 B14 D3 M7 X12 E1 S16 H19 B14 D3 M7 X12 E1 S16 H19 B14 D3 M7 X12 E1 S16 H19 B14 D3 M7 X12 E1 S16 H19 B14 D3 M7 X12 E1 S16 H19 B14 D3 M7 X12 E1 S16 H19 B14 D3 M7 X12 E1 S16 H19 B14 D3 M7 X12 E1 S16 H19 B14 D3 M7 X12 E1 S16 H19 B14 D3 M7 X12 E1 S16 H19 B14 D3 M7 X12 E1 S16 H19 B14 D3 M7 X12 E1 S16 H19 B14 D3 M7 X12 E1 S16 H19 B14 D3 M7 X12 E1 S16 H19 B14 D3 M7 X12 E1 S16 H19 B14 D3 M7 X12 E1 S16 H19 B14 D3 M7 X12 E1 S16 H19 B14 D3 M7 X12 E1 S16 H19 B14 D3 M7 X12 E1 S16 H19 B14 D3 M7 X12 E1 S16 H19 B14 D3 M7 X12 E1 S16 H19 B14 D3 M7 X12 E1 S16 H19 B14 D3 M7 X12 E1

8. Tessel: constrained lot allocation with rollback (new procedure)

Order inventory lots by expiry ascending, then lot id. A lot is eligible only when its expiry is at least delivery day plus 3 and it is cold. Allocate demand from eligible lots in that order; charge 4 per opened lot plus 2 per unit. If demand cannot be filled completely, cancel every allocation and charge 0. Return the ordered allocations, the charge and the unfilled amount.

Updates 2,835Supervised tokens 73,159 (presented 600,000) Own-end 188/225 (83.6%)Final 188/225 (83.6%) Scorer typed JSON procedure: parsed JSON compared field by fieldData training file · held-out panel (225 fixed rows)

A real training row (tessel_lot_allocation:candidate_train:01608)

Apply the tessel lot allocation procedure to the input. Return only a JSON object with allocations (ordered [lot_id,units] pairs), charge (integer), unfilled (integer), without explanation.
Input: {"delivery_day":4,"demand":22,"lots":[{"cold":true,"expiry":13,"id":"L0","units":12},{"cold":false,"expiry":3,"id":"L1","units":6},{"cold":true,"expiry":15,"id":"L2","units":12},{"cold":true,"expiry":10,"id":"L3","units":4},{"cold":false,"expiry":12,"id":"L4","units":12},{"cold":true,"expiry":25,"id":"L5","units":6},{"cold":true,"expiry":1,"id":"L6","units":7},{"cold":true,"expiry":10,"id":"L7","units":10},{"cold":true,"expiry":25,"id":"L8","units":12}]}
Answer:

Target:

{"allocations":[["L3",4],["L7",10],["L0",8]],"charge":56,"unfilled":0}

A held-out question (tessel_lot_allocation:retention:00246)

Apply the tessel lot allocation procedure to the input. Return only a JSON object with allocations (ordered [lot_id,units] pairs), charge (integer), unfilled (integer), without explanation.
Input: {"delivery_day":9,"demand":20,"lots":[{"cold":false,"expiry":7,"id":"L0","units":8},{"cold":true,"expiry":3,"id":"L1","units":9},{"cold":true,"expiry":21,"id":"L2","units":7},{"cold":true,"expiry":12,"id":"L3","units":8},{"cold":true,"expiry":8,"id":"L4","units":7},{"cold":true,"expiry":3,"id":"L5","units":11},{"cold":true,"expiry":2,"id":"L6","units":12}]}
Answer:

Target:

{"allocations":[],"charge":0,"unfilled":20}

Answer at the end of its own teach and after all ten teaches (identical): correct

{"allocations":[],"charge":0,"unfilled":20}

A held-out question the model got wrong (tessel_lot_allocation:retention:00057)

Apply the tessel lot allocation procedure to the input. Return only a JSON object with allocations (ordered [lot_id,units] pairs), charge (integer), unfilled (integer), without explanation.
Input: {"delivery_day":3,"demand":34,"lots":[{"cold":false,"expiry":24,"id":"L0","units":8},{"cold":true,"expiry":21,"id":"L1","units":2},{"cold":true,"expiry":23,"id":"L2","units":9},{"cold":true,"expiry":22,"id":"L3","units":7},{"cold":true,"expiry":1,"id":"L4","units":4},{"cold":true,"expiry":23,"id":"L5","units":9},{"cold":true,"expiry":7,"id":"L6","units":3},{"cold":true,"expiry":9,"id":"L7","units":10}]}
Answer:

Target:

{"allocations":[["L6",3],["L7",10],["L1",2],["L3",7],["L2",9],["L5",3]],"charge":92,"unfilled":0}

Answer at the end of its own teach and after all ten teaches (identical): wrong

{"allocations":[],"charge":0,"unfilled":34}

9. Juniper: non-preemptive priority scheduling (new procedure)

Each job has a release time, a duration and a priority. From clock 0, among released unscheduled jobs choose the highest priority, tie-break by shorter duration then id; if none is released, advance to the next release. Run jobs without preemption with a 2-unit cooldown after each. Return ordered [job, start, end] triples.

Updates 3,025Supervised tokens 209,278 (presented 699,873) Own-end 118/225 (52.4%)Final 120/225 (53.3%) Scorer typed JSON procedure: exact list matchData training file · held-out panel (225 fixed rows)

A real training row (juniper_dispatch:candidate_train:01932)

Apply the juniper dispatch procedure to the input. Return only a JSON list of [job_id,start,end] triples in execution order, without explanation.
Input: {"jobs":[{"duration":1,"id":"J0","priority":3,"release":11},{"duration":5,"id":"J1","priority":4,"release":12},{"duration":7,"id":"J2","priority":1,"release":12},{"duration":5,"id":"J3","priority":3,"release":8},{"duration":4,"id":"J4","priority":3,"release":11},{"duration":1,"id":"J5","priority":2,"release":14},{"duration":4,"id":"J6","priority":3,"release":0},{"duration":5,"id":"J7","priority":2,"release":8}]}
Answer:

Target:

[["J6",0,4],["J3",8,13],["J1",15,20],["J0",22,23],["J4",25,29],["J5",31,32],["J7",34,39],["J2",41,48]]

A held-out question (juniper_dispatch:retention:00036)

Apply the juniper dispatch procedure to the input. Return only a JSON list of [job_id,start,end] triples in execution order, without explanation.
Input: {"jobs":[{"duration":2,"id":"J0","priority":4,"release":4},{"duration":6,"id":"J1","priority":4,"release":14},{"duration":5,"id":"J2","priority":3,"release":4},{"duration":1,"id":"J3","priority":2,"release":2},{"duration":4,"id":"J4","priority":4,"release":7},{"duration":4,"id":"J5","priority":3,"release":4},{"duration":3,"id":"J6","priority":1,"release":1}]}
Answer:

Target:

[["J6",1,4],["J0",6,8],["J4",10,14],["J1",16,22],["J5",24,28],["J2",30,35],["J3",37,38]]

Answer at the end of its own teach: wrong

[["J6",1,4],["J0",6,8],["J4",10,14],["J1",16,22],["J2",24,29],["J5",31,35],["J3",37,38]]

Answer after all ten teaches: correct

[["J6",1,4],["J0",6,8],["J4",10,14],["J1",16,22],["J5",24,28],["J2",30,35],["J3",37,38]]

A held-out question the model got wrong (juniper_dispatch:retention:00110)

Apply the juniper dispatch procedure to the input. Return only a JSON list of [job_id,start,end] triples in execution order, without explanation.
Input: {"jobs":[{"duration":2,"id":"J0","priority":3,"release":6},{"duration":2,"id":"J1","priority":1,"release":5},{"duration":4,"id":"J2","priority":1,"release":14},{"duration":7,"id":"J3","priority":1,"release":14},{"duration":4,"id":"J4","priority":1,"release":11},{"duration":7,"id":"J5","priority":4,"release":4},{"duration":4,"id":"J6","priority":4,"release":6},{"duration":7,"id":"J7","priority":1,"release":8},{"duration":6,"id":"J8","priority":4,"release":5}]}
Answer:

Target:

[["J5",4,11],["J6",13,17],["J8",19,25],["J0",27,29],["J1",31,33],["J2",35,39],["J4",41,45],["J3",47,54],["J7",56,63]]

Answer at the end of its own teach and after all ten teaches (identical): wrong

[["J5",4,11],["J6",13,17],["J8",19,25],["J0",27,29],["J1",31,33],["J4",35,39],["J2",41,45],["J7",47,54],["J3",56,63]]

10. Bracken: half-open interval set arithmetic (new procedure)

Take the union of the open half-open intervals, subtract the union of the closed ones, merge touching intervals and return the sorted maximal remaining intervals as [start, end] integer pairs.

Updates 3,756Supervised tokens 57,751 (presented 340,150) Own-end 161/225 (71.6%)Final 161/225 (71.6%) Scorer typed JSON procedure: exact list matchData training file · held-out panel (225 fixed rows)

A real training row (bracken_intervals:candidate_train:02810)

Apply the bracken intervals procedure. Return only a JSON array of [start,end] integer pairs, without explanation.
Input: {"closed":[[1,2],[-8,0],[32,36],[40,49]],"open":[[-2,3],[-10,-1],[30,40],[18,20]]}
Answer:

Target:

[[-10,-8],[0,1],[2,3],[18,20],[30,32],[36,40]]

A held-out question (bracken_intervals:retention:00103)

Apply the bracken intervals procedure. Return only a JSON array of [start,end] integer pairs, without explanation.
Input: {"closed":[[9,11],[-9,-4],[-1,7],[21,23]],"open":[[23,24],[32,38],[9,11]]}
Answer:

Target:

[[23,24],[32,38]]

Answer at the end of its own teach and after all ten teaches (identical): correct

[[23,24],[32,38]]

A held-out question the model got wrong (bracken_intervals:retention:00075)

Apply the bracken intervals procedure. Return only a JSON array of [start,end] integer pairs, without explanation.
Input: {"closed":[[37,38],[-8,-4],[20,23]],"open":[[-9,-1],[24,25],[36,45],[-6,0]]}
Answer:

Target:

[[-9,-8],[-4,0],[24,25],[36,37],[38,45]]

Answer at the end of its own teach and after all ten teaches (identical): wrong

[[-9,-8],[-4,-1],[24,25],[36,37],[38,45]]

5. Evaluation

Panels. Each skill has a fixed panel of 225 held-out rows. No panel prompt appears among the training prompts of its skill, and no training prompt appears twice within a skill; both are exact-match checks that the repository's scoring command repeats. Eight rows of each panel were also used for the loss monitor during training, so the panels are held out from training but were observed while it ran. The SCAN and COGS panels are samples of those datasets and are not their published generalization splits. For the written procedures the inputs are random instances of the same fixed procedure; a held-out row is a new input, not a new kind of task, and the panels do not test transfer beyond that.

Identity. An answer counts for a checkpoint only if it was generated from that checkpoint's manifest, with the fixed prompt, the skill's cap and the frozen scorer, and it stopped on end-of-sequence or the cap. When the same row was answered twice under the same identity, the first terminal record is used. Across the 2,250 final answers there are 2 cap hits (both PCFG SET, scored wrong) and 0 conflicting duplicates.

Scorers. Five skill-specific scorers, frozen before the run: exact action-sequence match (SCAN), exact logical form (COGS), exact IPA segment sequence (Adyghe), exact symbol sequence (PCFG SET), and typed-JSON comparison for the six procedures (the answer is parsed as JSON and compared with the oracle value; malformed JSON is wrong). No answer repair.

Monitor versus final. The held-out loss curves in the figures come from eight fixed panel rows per skill measured every 25 updates during training; the scores in the tables come from the full 225-row panels generated after training at each checkpoint. The two are different measurements of the same held-out rows and are never mixed.

Pairing. Retention is a paired comparison: for a given skill, the same 225 rows are scored at its own-end checkpoint and at each later checkpoint, so every change is a row that flipped, and every flipped row can be inspected in the evidence explorer.

6. Results

Every checkpoint, every skill

checkpointSCANMeridianAdyghe G2POrielCOGSSablePCFG SETTesselJuniperBracken
after 1: SCAN222
after 2: Meridian22283
after 3: Adyghe G2P22284147
after 4: Oriel2228214612
after 5: COGS2228114614225
after 6: Sable2228114615225198
after 7: PCFG SET2228214616225197174
after 8: Tessel2228314614225198173188
after 9: Juniper2228414715225198174187118
after 10: Bracken222841469225198172188120161

Read down a column. SCAN is 222 at all ten checkpoints. Meridian moves between 81 and 84. Adyghe G2P between 146 and 147. COGS stays at 225. Sable is 198 at every checkpoint but one, where it is 197. PCFG SET 172 to 174, Tessel 187 to 188, Juniper 118 to 120. Oriel is the exception: 12, 14, 15, 16, 14, 15, then 9.

Accuracy on the fixed 225-row panels at every accepted checkpoint measured Rows: checkpoint after each teach. Columns: skill. Cell: correct of 225 (exact match) blank = not yet taught at that checkpoint SCAN Meridian Adyghe G2P Oriel COGS Sable PCFG Tessel Juniper Bracken after 1: SCAN 222 after 2: Meridian 222 83 after 3: Adyghe G2P 222 84 147 after 4: Oriel 222 82 146 12 after 5: COGS 222 81 146 14 225 after 6: Sable 222 81 146 15 225 198 after 7: PCFG 222 82 146 16 225 197 174 after 8: Tessel 222 83 146 14 225 198 173 188 after 9: Juniper 222 84 147 15 225 198 174 187 118 after 10: Bracken 222 84 146 9 225 198 172 188 120 161 Each cell is the number of exactly-correct answers on that skill's fixed 225-row held-out panel at that checkpoint. Source: plots/data/derived/tables.json (matrix), from the panel analysis receipt.
Figure 4. The checkpoint-by-skill matrix: each skill is scored from its own teach onward, so cells above the diagonal are blank. Read down a column to see a skill's score while later skills are taught: SCAN 222 at every one of ten checkpoints; Meridian moves within 81 to 84; Oriel is the only column that moves by more than two rows.

Own-end versus final

skillown-endfinalchangeown-end %final %
1. SCAN222222+098.7%98.7%
2. Meridian8384+136.9%37.3%
3. Adyghe G2P147146-165.3%64.9%
4. Oriel129-35.3%4.0%
5. COGS225225+0100.0%100.0%
6. Sable198198+088.0%88.0%
7. PCFG SET174172-277.3%76.4%
8. Tessel188188+083.6%83.6%
9. Juniper118120+252.4%53.3%
10. Bracken161161+071.6%71.6%
all ten (of 2,250)1,5281,525-367.9%67.8%

Summed over all 2,250 rows the model answered 1,528 correctly at own-end and 1,525 at the end, a net change of -3 rows. Excluding Oriel, the nine acquired skills sum to 1,516 at own-end and 1,516 at the end. Both denominators are stated.

Retention as loss

What happens to a skill after its teach ends measured SCAN Meridian Adyghe G2P Oriel COGS Sable PCFG Tessel Juniper Bracken Held-out loss of every earlier skill while later skills are trained lines: 8 monitored rows every 25 updates · dots: separate 24-row sweeps at teach endpoints; nats, log scale 1e-05 0.0001 0.001 0.01 0.1 1 10 0 10,000 20,000 30,000 40,000 50,000 60,000 update (whole stream) SCAN 6.9e-05 Meridian 0.16 Adyghe G2P 0.37 Oriel 0.061 COGS 0.00068 Sable 0.03 PCFG 0.035 Tessel 0.035 Juniper 0.015 Bracken 0.093 English control (24 rows) 3 In the first five teaches an earlier skill was measured only at teach endpoints; in the sixth teach two earlier skills were monitored continuously, and from the seventh every earlier skill was. The 8-row monitor and the 24-row endpoint sweeps are different samples of the panel and are drawn as separate marks. A line is drawn only across consecutive measurements; where a skill was not monitored there is a gap, and isolated endpoint measurements are dots. Dashed verticals mark teach boundaries. Source: plots/data/series/held_out_loss.json.
Figure 5. Retention as a loss curve: after a skill's own segment ends, its held-out loss is nearly flat across the remaining tens of thousands of updates on other skills. The English control panel (never trained) stays where it started.

Acquisition per skill

Each skill during its own teach: training loss and held-out loss measured training loss, smoothed (moving average, span 31) raw training loss, min to max held-out loss on 8 monitored panel rows, every 25 updates 1. SCAN 9,850 updates · held-out loss at the end 9.1e-05 nats 1e-05 0.0001 0.001 0.01 0.1 1 10 0 2,000 4,000 6,000 8,000 update within the skill 2. Meridian 3,557 updates · held-out loss at the end 0.16 nats 1e-05 0.0001 0.001 0.01 0.1 1 10 0 1,000 2,000 3,000 update within the skill 3. Adyghe G2P 3,600 updates · held-out loss at the end 0.37 nats 1e-05 0.0001 0.001 0.01 0.1 1 10 0 1,000 2,000 3,000 update within the skill 4. Oriel 3,568 updates · held-out loss at the end 0.059 nats 1e-05 0.0001 0.001 0.01 0.1 1 10 0 1,000 2,000 3,000 update within the skill 5. COGS 21,309 updates · held-out loss at the end 0.00061 nats 1e-05 0.0001 0.001 0.01 0.1 1 10 0 5,000 10,000 15,000 20,000 update within the skill 6. Sable 3,911 updates · held-out loss at the end 0.031 nats 1e-05 0.0001 0.001 0.01 0.1 1 10 0 1,000 2,000 3,000 update within the skill 7. PCFG 7,871 updates · held-out loss at the end 0.034 nats 1e-05 0.0001 0.001 0.01 0.1 1 10 0 2,000 4,000 6,000 update within the skill measured where the teach resumed and at its end 8. Tessel 2,835 updates · held-out loss at the end 0.045 nats 1e-05 0.0001 0.001 0.01 0.1 1 10 0 1,000 2,000 update within the skill measured at the start and the end of this teach 9. Juniper 3,025 updates · held-out loss at the end 0.015 nats 1e-05 0.0001 0.001 0.01 0.1 1 10 0 1,000 2,000 3,000 update within the skill 10. Bracken 3,756 updates · held-out loss at the end 0.093 nats 1e-05 0.0001 0.001 0.01 0.1 1 10 0 1,000 2,000 3,000 update within the skill All ten panels share one loss axis (1e-5 to 10 nats, log scale). The held-out curve is the mean per-token loss on eight fixed panel rows that are never trained on, measured every 25 updates; it is a monitor, not the 225-row evaluation. During the PCFG and Tessel teaches the every-25-update monitor watched the previously taught skills and the English control rather than the skill being taught; that skill was measured on the same eight rows at the gate instead, once before its first update and once after its last, and those two points are drawn with a dashed segment between them. The PCFG point sits 792 updates in because that teach was resumed there; the Tessel point sits at the start of its teach. Source: plots/data/series/{training_loss,held_out_loss}.json.
Figure 6. Per-skill training and held-out loss during each skill's own segment, on one shared scale. Held-out loss falls with training on all ten and settles at very different levels: near zero for SCAN and COGS, far higher for Oriel and Juniper.
Ten skills, one stream: training loss per update measured training loss, smoothed (moving average, span 31) raw per-update loss, min to max One example per update, 63,282 updates in the order they were applied loss on the example being trained, nats (log scale) 0.0001 0.001 0.01 0.1 1 10 0 10,000 20,000 30,000 40,000 50,000 60,000 update SCAN Meridian Adyghe G2P Oriel COGS Sable PCFG Tessel Juniper Bracken Ten skills taught one after another; shaded columns alternate per skill. Losses below 1e-4 are drawn at 1e-4. The pale band spans the lowest and highest raw loss in each of 780 slices of the stream, so every spike is inside it; the line is the smoothed loss. Source: plots/data/series/training_loss.json.
Figure 7. Training loss over the whole stream: each skill starts high, drops within its own segment, and hands over to the next. Nothing in the stream tells the learner where one skill ends and the next begins.
Held-out loss does not predict exact-match accuracy measured Each skill at the end of its own teach x: held-out loss at the endpoint sweep (nats, log scale) · y: correct of 225 0 45 90 135 180 225 1e-05 0.0001 0.001 0.01 0.1 1 held-out loss at own-end (nats) correct of 225 SCAN Meridian Adyghe G2P Oriel COGS Sable PCFG Tessel Juniper Bracken Loss is the mean per-token loss over the panel rows in the last sweep of the skill's own segment; accuracy needs every token right. Source: plots/data/series/held_out_loss.json, plots/data/derived/tables.json.
Figure 8. A low held-out loss is necessary but not sufficient for exact-match accuracy: Oriel and Meridian sit at loss levels similar to skills that score far higher, because a single wrong token anywhere in a long structured answer costs the whole row.

Training dose

skillupdatespresented tokenssupervised tokens (incl. EOS)cumulative updates
1. SCAN9,850499,982297,2329,850
2. Meridian3,557349,92568,15913,407
3. Adyghe G2P3,600232,03742,12217,007
4. Oriel3,568349,908179,89020,575
5. COGS21,3091,799,9811,099,47941,884
6. Sable3,911349,98273,15045,795
7. PCFG SET7,871699,825195,15153,666
8. Tessel2,835600,00073,15956,501
9. Juniper3,025699,873209,27859,526
10. Bracken3,756340,15057,75163,282
total63,2825,921,6632,295,371

These totals are the examples the final checkpoint was trained on. Tokens are counted with the base model's tokenizer.

How much each skill was trained measured updates applied (one example each) supervised tokens, including the end-of-sequence token Updates examples the learner was trained on Supervised tokens tokens the loss was computed on SCAN 9,850 Meridian 3,557 Adyghe G2P 3,600 Oriel 3,568 COGS 21,309 Sable 3,911 PCFG 7,871 Tessel 2,835 Juniper 3,025 Bracken 3,756 SCAN 297k Meridian 68k Adyghe G2P 42k Oriel 180k COGS 1099k Sable 73k PCFG 195k Tessel 73k Juniper 209k Bracken 58k Totals: 63,282 updates, 2,295,371 supervised tokens, 5,921,663 presented tokens. Source: plots/data/derived/tables.json (dose).
Figure 9. Training dose per skill. COGS received a third of all updates and half of all supervised tokens; the other nine received between 2,835 and 9,850 updates each.

7. Oriel: what the changed answers look like

Oriel is the skill that moved: 12 of 225 at its own teach, 15 after Juniper, 9 after Bracken. The last step lost seven rows and gained one, a net loss of six (2.67 percentage points of the panel, 40% of the 15 it had). A second bounded generation on the same checkpoint reproduced the 9 exactly.

The lost rows share one pattern. Re-scoring each component of each answer with the frozen scorer shows that on all seven the header, the transformed payload, the escaping and the decoded length are still correct after Bracken; only the checksum value changed. The gained row is the mirror image: only its checksum was wrong before. Across all 225 rows the checksum was right on 15 before and 11 after, while the other components stayed within one or two rows. Another 37 rows wrong at both checkpoints also changed, so the answers moved, not the scoring.

The checksum is the final integer of the packet (written as an escape pair when it equals 11 or 15, so the raw list can change length while the decoded length does not). A wrong checksum invalidates the packet, so this is a real decline, localized to the last value of a skill the model never learned well. The monitored held-out loss moved only from 0.0586 to 0.0605 nats: a one-value change flips an exact grade while barely moving the mean token loss.

rowpayloadtargetafter teach 9after teach 10
00008[12, 1, 7, 9, 2, 9, 12][15,13,7,1,5,6,4,12,6,14,12,7,7,14,12,1,5,15,13][15,13,7,1,5,6,4,12,6,14,12,7,7,14,12,1,5,15,13]
correct		[15,13,7,1,5,6,4,12,6,14,12,7,7,14,12,1,5,15,9]
wrong
00130[12, 10, 10, 9, 7, 8, 10][15,13,7,1,5,15,9,15,9,15,9,15,9,14,12,12,6,13,9,15,9,15,9,15,13][15,13,7,1,5,15,9,15,9,15,9,15,9,14,12,12,6,13,9,15,9,15,9,15,13]
correct		[15,13,7,1,5,15,9,15,9,15,9,15,9,14,12,12,6,13,9,15,9,15,9,1]
wrong

In the first row the escaped checksum pair 15,13 (which decodes to 11) became 15,9 (which decodes to 15). In the second the pair became the single value 1: the decoded list is the same length, and only the checksum value differs. All seven lost rows and the gained row, with both complete predictions, are in the table below (each row id opens that row in the evidence explorer) and in plots/data/derived/oriel_flipped_rows.json; the per-row component analysis for all 225 rows is in the evidence explorer.

Oriel from teach 9 to teach 10: which part of the answer changed measured after teach 9 (15 of 225 correct) after teach 10 (9 of 225 correct) Rows with each component correct, all 225 panel rows a row counts as correct only when every component is right 0 45 90 135 180 225 rows of 225 223 225 escaping 225 225 header 208 208 payload 212 214 length 15 11 checksum Paired 225 rows at both checkpoints: correct→wrong 7, wrong→correct 1, correct→correct 8. Every lost row had header, payload, escaping and length right and the checksum wrong. Source: plots/data/derived/oriel_components.json.
Figure 10. The Oriel decline is localized: between the two checkpoints the header, transformed payload, escaping and decoded length stay correct on essentially all rows, while the final checksum value is right on 15 rows, then 11. Seven rows lost that one value; one row gained it.
All eight rows that changed grade
rowpayloadtargetafter teach 9after teach 10
00008[12, 1, 7, 9, 2, 9, 12][15,13,7,1,5,6,4,12,6,14,12,7,7,14,12,1,5,15,13][15,13,7,1,5,6,4,12,6,14,12,7,7,14,12,1,5,15,13]
correct		[15,13,7,1,5,6,4,12,6,14,12,7,7,14,12,1,5,15,9]
wrong
00021[0, 6, 1, 11, 7, 12, 3, 12][15,13,8,5,1,15,13,3,6,4,0,2,12,6,1,5,8,10,1,5,15,13][15,13,8,5,1,15,13,3,6,4,0,2,12,6,1,5,8,10,1,5,15,13]
correct		[15,13,8,5,1,15,13,3,6,4,0,2,12,6,1,5,8,10,1,5,15,9]
wrong
00130[12, 10, 10, 9, 7, 8, 10][15,13,7,1,5,15,9,15,9,15,9,15,9,14,12,12,6,13,9,15,9,15,9,15,13][15,13,7,1,5,15,9,15,9,15,9,15,9,14,12,12,6,13,9,15,9,15,9,15,13]
correct		[15,13,7,1,5,15,9,15,9,15,9,15,9,14,12,12,6,13,9,15,9,15,9,1]
wrong
00265[1, 8, 12, 0, 14, 0][15,13,6,6,4,13,9,1,5,5,1,3,15,13,5,1,15,13][15,13,6,6,4,13,9,1,5,5,1,3,15,13,5,1,15,13]
correct		[15,13,6,6,4,13,9,1,5,5,1,3,15,13,5,1,15,9]
wrong
00321[12, 10, 15, 8, 15, 7, 7, 2, 8][15,13,9,1,5,15,9,15,9,4,14,13,9,4,14,12,6,12,6,7,7,13,9,15,13][15,13,9,1,5,15,9,15,9,4,14,13,9,4,14,12,6,12,6,7,7,13,9,15,13]
correct		[15,13,9,1,5,15,9,15,9,4,14,13,9,4,14,12,6,12,6,7,7,13,9,1]
wrong
00420[11, 15, 10, 1, 2, 4, 12][15,13,7,0,2,4,14,15,9,15,9,6,4,7,7,9,13,1,5,1][15,13,7,0,2,4,14,15,9,15,9,6,4,7,7,9,13,1,5,1]
correct		[15,13,7,0,2,4,14,15,9,15,9,6,4,7,7,9,13,1,5,14]
wrong
00485[6, 8, 11, 6, 9, 11, 7, 10][15,13,8,15,13,3,13,9,0,2,15,13,3,14,12,0,2,12,6,15,9,15,9,15,9][15,13,8,15,13,3,13,9,0,2,15,13,3,14,12,0,2,12,6,15,9,15,9,15,9]
correct		[15,13,8,15,13,3,13,9,0,2,15,13,3,14,12,0,2,12,6,15,9,15,9,15,13]
wrong
00496[14, 8, 9, 1, 6, 7, 5, 6, 1][15,13,9,3,15,13,13,9,14,12,6,4,15,13,3,12,6,10,0,15,13,3,6,4,1][15,13,9,3,15,13,13,9,14,12,6,4,15,13,3,12,6,10,0,15,13,3,6,4,15,13]
wrong		[15,13,9,3,15,13,13,9,14,12,6,4,15,13,3,12,6,10,0,15,13,3,6,4,1]
correct

8. Base capability

The base does not move materially on the measured benchmarks. Across the ten-skill stream, the full base-capability battery remains near its starting level, and the endpoint probes remain stable.

Two measurements. A 498-item sentinel (ARC 128, HellaSwag 128, MMLU 114, WinoGrande 128; likelihood choice among the options) was run at all eleven checkpoints; it stayed between 348 and 353 correct throughout. The full likelihood battery of 16,481 items (MMLU 14,042 over 57 subjects, ARC-Challenge 1,172, WinoGrande 1,267; 5-shot, lm-eval 0.4.12 port, seed 0) was run at teaches 0, 5 and 10. The battery is likelihood-scored only; no free-generation benchmark was run for this experiment.

batteryafter teach 0after teach 5after teach 10
MMLU, macro over 57 subjects0.86340.86310.8654
MMLU, micro0.86250.86210.8636
ARC-Challenge (acc / normalized)0.7056 / 0.74150.7022 / 0.73980.7099 / 0.7466
WinoGrande0.79480.79240.7885

The largest movement is WinoGrande, from 0.7948 to 0.7885 (0.63 points down over ten teaches); ARC-Challenge goes from 0.7056 to 0.7099 and the MMLU macro average from 0.8634 to 0.8654. These are deterministic likelihood scores, each measured once per checkpoint. The sentinel per checkpoint:

after teachcorrect of 498normalizedARCHellaSwagMMLUWinoGrande
035037278/12879/12897/11496/128
135037379/12878/12897/11496/128
234837178/12878/12896/11496/128
334837278/12878/12896/11496/128
435037479/12879/12896/11496/128
535037279/12878/12896/11497/128
634937179/12878/12896/11496/128
735037678/12878/12897/11497/128
835237581/12878/12896/11497/128
935337681/12878/12898/11496/128
1035137481/12878/12897/11495/128
General capability across the stream measured sentinel, raw likelihood choice sentinel, length-normalized full battery (16,481 items) Endpoint sentinel at all eleven checkpoints arc 128 · hellaswag 128 · mmlu 114 · winogrande 128; correct of 498 300 330 360 390 420 0 1 2 3 4 5 6 7 8 9 10 after teach 348–353 raw Full battery at 0 / 5 / 10 5-shot likelihood; accuracy 0.65 0.70 0.75 0.80 0.85 0.90 0 5 10 after teach MMLU (57-subject macro) WinoGrande ARC-Challenge Sentinel and battery are likelihood-based choices scored by the reference harness port. Values: plots/data/derived/tables.json (sentinel, base_battery).
Figure 11. The base does not move materially on the measured benchmarks: the 498-item sentinel stays between 348 and 353 correct across all eleven checkpoints, and the full battery after ten teaches reads MMLU 0.8654, ARC 0.7099, WinoGrande 0.7885 against 0.8634, 0.7056, 0.7948 at step zero.

9. Interpretation, and how to check it

Acquisition varied by skill, from complete (COGS 225, SCAN 222) to weak (Oriel 9 to 16) at the dose each received, and the dose was set in advance without pilots. Retention on the acquired skills was within two rows of own-end on every one of them after up to 53,432 further updates on other skills, with no replay and no label. The base does not move materially on the measured benchmarks (section 8). The one decline, Oriel, is real, reproducible and localized to the checksum of a skill that was never learned well.

What this does not show: transfer beyond new inputs of the same procedure, behavior past ten skills or past this budget, or that the same holds for other orders. Independent replication has not happened yet; the released data, scorers and answers exist so that it can.

To check the numbers: every training row, panel row, scorer and stored answer is in the replication repository (ten_skills/), including the 2,250 final and 2,250 own-end answers with their grades. The evidence explorer lets you filter every held-out answer by skill and result. Figures regenerate from the released data with one command (plots/build.py).

Run the experiment through the API. Use the supplied client, training data and evaluation driver to teach the ten skills sequentially and score their held-out answers. Start with a small end-to-end example, or run the full training schedule. See the API replication guide for access, setup, commands and estimated costs.

10. Provenance

The result is the evaluated accepted sequence of eleven checkpoints: the base model and the checkpoint after each of the ten teaches. It was not one uninterrupted job. Training stopped and resumed from saved checkpoints between teaches and once within a teach, each time restoring the full training state. Every accepted checkpoint is identified by its digest in the repository's lineage file, and every answer file names the checkpoint it was generated from.

Ten skills, one model, one pass · Learner Labs