Demonstrations
Two invented languages, taught in sequence
One learner taught Velenic, then Morvath, 22,000 words each; a matched LoRA adapter given the same bytes in the same order. Every generation and both training curves are in the repository.
What was taught, and what happened
This demonstration teaches a skill rather than a set of facts: two entire invented languages. Does learning the second destroy the first?
Setup
Velenic, then Morvath: 22,000 words of running text each, generated from 300-stem lexicons with distinct morphology, taught back-to-back onto one learner (demos/two-languages/data/: corpora byte-exact, lexicons included so you can re-score every generation). Invented, so the base model has zero prior exposure.
Results
Held-out loss fell from 4.6625 to 2.8880 for Velenic and from 4.3636 to 2.2842 during the Morvath teach. After both languages, Velenic remained at 2.8871—a change of −0.0009 nats after Morvath was taught. The table gives all three readings for both languages, measured on the two 256-token windows withheld from each corpus.
| Held-out loss (nats) | Before teaching | After Velenic | After Morvath |
|---|---|---|---|
| Velenic | 4.6625 | 2.8880 | 2.8871 |
| Morvath | 4.4150 | 4.3636 | 2.2842 |
The figure below shows the recording’s 627 training-loss points behind their 15-update average, with the same held-out readings marked at updates 0, 274 and 627.
Cross-language contamination was 0.00 in every measured cell, in both directions, with greedy decoding and sampled at temperature 0.9, at all three stages. The four English control questions were answered correctly at every stage. Held-out loss shows acquisition; the continuation probe shows no consistent gain in vocabulary or morphology scores. The values below are scorer means over eight prompts per cell, not sentence accuracy.
| Continuation probe: lexicon / morphology / contamination | Before teaching | After Velenic | After Morvath |
|---|---|---|---|
| Velenic, greedy | 0.14 / 0.25 / 0.00 | 0.08 / 0.12 / 0.00 | 0.21 / 0.25 / 0.00 |
| Velenic, temperature 0.9 | not sampled | 0.17 / 0.38 / 0.00 | 0.16 / 0.38 / 0.00 |
| Morvath, greedy | 0.00 / 0.00 / 0.00 | 0.00 / 0.00 / 0.00 | 0.00 / 0.00 / 0.00 |
| Morvath, temperature 0.9 | not sampled | 0.00 / 0.00 / 0.00 | 0.00 / 0.00 / 0.00 |
A Velenic continuation generated after the second language was taught:
We trained a conventional LoRA adapter (rank 256, 1.107 billion trainable parameters, 0.97× the learner's own trainable count) on the same bytes in the same order, one adapter learning both languages in sequence, standard recipe. It learned language 1 (loss 4.35 → 2.22), and then language 2 destroyed it:
Same data, same order, opposite outcome. This is catastrophic forgetting measured in LoRA and its absence measured in ours. Per-language fluency at this dose is modest. These runs show accumulation without erasure rather than eloquence.
See all 164 prompts, answers and probe scores
Conclusion
Learner 1.0 learned from both corpora in one pass. Velenic’s held-out loss fell to 2.8880 and stayed at 2.8871 after Morvath; Morvath finished at 2.2842. The four English control questions remained correct at every stage. The adapter wiped language 1 at +3.78 nats and a 96% identity swap on identical bytes.
The training unit
Each language is one text file. The file is cut into 256-token windows under Learner 1.0's tokenizer: 276 windows for Velenic and 355 for Morvath. Each training window is presented exactly once, in a fixed order: 274 Velenic windows followed by 353 Morvath windows, batch size 1. Each window contains 256 tokens. Two windows per language are held out from training. That is 627 updates and 160,512 presented training tokens. The exact windows, in the order they were trained and with the held-out windows marked, are in the replication repository: windows_velenic.jsonl and windows_morvath.jsonl.
A real training excerpt. The opening of the Velenic corpus, which falls inside one of its training windows:
Sesaenta romaenta zamiave. Nomoul nozloul masoos. Vrivre lovreen liroos. Ziva viskien rereos vrivoul meliir. Zloseir rinaenta sezleos. Zloseul zlazoen zozoave. Leskeen revro vrozaave.
Velenic is an invented language; the corpus was generated for this study.
The training data. The two corpora are in the replication repository, byte for byte as they were taught: velenic.txt and morvath.txt. The continuation shown earlier on this page is an evaluation example, not training text.
Which recording each number comes from
Every Learner 1.0 number on this page comes from one one-pass recording: the held-out loss readings, the training-loss series, the 80 continuations with their scores and the 12 English control answers (demos/two-languages/answers/session.json, answers/generations.json, data/training-loss.jsonl). The LoRA numbers come from the original comparator recording (data/comparator-standard-adapter.json), which was not re-run.
What the two comparisons share, and what they do not
The learner and the adapter were given identical corpus bytes in the identical language order, and generation identity is scored the same way for both: a continuation prompted in one language is checked for stems of the other. The training schedules are not the same: Learner 1.0 saw each 256-token window once, and the adapter used its own schedule, so this is not a matched one-pass comparison. Retention is not the same measurement on both sides. The learner's is held-out loss on the two withheld Velenic windows, read before and after the Morvath teach; the adapter's is cross-entropy on held-out first-language text computed by our harness (rank 256, 1.11 billion trainable parameters, learning rate 2e-05). Read the two as each method against its own earlier level, not as one number against the other.