Source: https://learnerlabs.ai/demos/skills/two-languages
Format: alternate Markdown of the approved capability-first article. All methods and records are included, even where collapsed in HTML. Tables retain their HTML cells and spans to preserve data associations.
Exported: 2026-10-13
Context: [Evidence guide](/agents/overview.md) · [Evidence questions](/agents/questions.md). Publisher: Learner Labs. Inline SVG labels are transcribed below. Raster figures retain original image links and captions.
Demonstrations
Two invented languages
# Learning a second invented language preserves the first language’s loss
We taught Learner 1.0 two invented languages in sequence, 22,000 words each, in one pass. Learning Morvath did not erase what it had learnt of Velenic: the first language’s held-out loss was 2.8880 after its own teach and 2.8871 after the second. Morvath’s loss also fell during its teach. This is the capability being tested: add another skill without undoing the earlier one.
A LoRA adapter given identical text in the same language order lost ground on its first language after learning the second, with loss rising from 2.22 to 6.00 nats. The training schedules and retention measurements differ, so each method must be read against its own earlier level. Learner 1.0’s measured continuations kept the two languages apart, but vocabulary and morphology gains were inconsistent. The result is retention of learnt structure at this dose, not fluent mastery.
## The second language is acquired without erasing the first
This demonstration teaches a skill rather than a set of facts: two entire invented languages. Does learning the second destroy the first?
### Two invented corpora are taught back to back
Velenic, then Morvath: 22,000 words of running text each, generated from 300-stem lexicons with distinct morphology, taught back-to-back onto one learner ([demos/two-languages/data/](https://github.com/learnerlabs-ai/replications/tree/main/demos/two-languages): corpora byte-exact, lexicons included so you can re-score every generation). Invented, so the base model has zero prior exposure.
### First-language loss stays stable and measured identities stay separate
Held-out loss fell from 4.6625 to 2.8880 for Velenic and from 4.3636 to 2.2842 during the Morvath teach. After both languages, Velenic remained at 2.8871. That is a change of −0.0009 nats after Morvath was taught. The table gives all three readings for both languages, measured on the two 256-token windows withheld from each corpus.
| Held-out loss (nats) | Before teaching | After Velenic | After Morvath |
|---|
| Velenic | 4.6625 | 2.8880 | 2.8871 |
| Morvath | 4.4150 | 4.3636 | 2.2842 |
The figure below shows the recording’s 627 training-loss points behind their 15-update average, with the same held-out readings marked at updates 0, 274 and 627.
Swipe the figure to read every label →
### Learning Morvath preserves Velenic’s held-out loss
[Original chart: Learning Morvath preserves Velenic’s held-out loss](/agents/figures/two-1.svg)
Figure labels: Two invented languages, taught one after the other in one pass | training loss, each update | 15-update average | held-out loss on 2 withheld windows | 1. Velenic | 274 training windows of 256 tokens (70,144 tokens), each | presented once | 0 | 1 | 2 | 3 | 4 | 5 | 0 | 274 | 627 | update | loss (nats) | 4.6625 | 2.8880 | 2.8871 | 2. Morvath | 353 training windows of 256 tokens (90,368 tokens), each | presented once | 0 | 1 | 2 | 3 | 4 | 5 | 0 | 274 | 627 | update | loss (nats) | 4.4150 | 4.3636 | 2.2842 | Held-out loss is read at three fixed points (before any teaching, after Velenic, after Morvath) on two 256-token windows | withheld from each corpus. It is not a continuous validation curve. The vertical rule marks the end of the Velenic teach, update | 274. Source: plots/data/two_languages.json (learner_onepass block).
**[Figure 1](/demos/skills/two-languages#tl_f1_curves).** Learner 1.0, one-pass recording. Each panel shows one language’s training loss at every update with a 15-update average drawn over it, and the held-out loss read before teaching, after Velenic and after Morvath. Velenic’s held-out loss is 2.8880 after its own teach and 2.8871 after Morvath is taught. The horizontal axis is training update and the vertical axis is loss in nats. The amber points use two withheld 256-token windows per language.
**Cross-language contamination was 0.00 in every measured cell, in both directions**, with greedy decoding and sampled at temperature 0.9, at all three stages. The four English control questions were answered correctly at every stage. Held-out loss shows acquisition. The continuation probe shows no consistent gain in vocabulary or morphology scores. The values below are scorer means over eight prompts per cell, not sentence accuracy.
| Continuation probe: lexicon / morphology / contamination | Before teaching | After Velenic | After Morvath |
|---|
| Velenic, greedy | 0.14 / 0.25 / 0.00 | 0.08 / 0.12 / 0.00 | 0.21 / 0.25 / 0.00 |
| Velenic, temperature 0.9 | not sampled | 0.17 / 0.38 / 0.00 | 0.16 / 0.38 / 0.00 |
| Morvath, greedy | 0.00 / 0.00 / 0.00 | 0.00 / 0.00 / 0.00 | 0.00 / 0.00 / 0.00 |
| Morvath, temperature 0.9 | not sampled | 0.00 / 0.00 / 0.00 | 0.00 / 0.00 / 0.00 |
A Velenic continuation generated *after* the second language was taught:
**Prompt (Velenic):** Vavien sovrair lezlienta zasaave. Zana skolo lozla…
Lizla vavien skoloir zasaave. Zlolo mevre skizla zanaave.
*This continuation: lexicon 1.0, morphology valid 1.0, contamination 0.0, over the six words left after the prompt’s own words are removed. The mean for its cell, over eight prompts, is 0.21 / 0.25 / 0.00. The lexicons are published, so every row can be re-scored*
We trained a conventional LoRA adapter (rank 256, 1.107 billion trainable parameters, 0.97× the learner's own trainable count) on **the same bytes in the same order**, one adapter learning both languages in sequence, standard recipe. It learned language 1 (loss 4.35 → 2.22), and then language 2 destroyed it:
Swipe the figure to read every label →
### LoRA’s first-language loss rises above its starting level
[Original chart: LoRA’s first-language loss rises above its starting level](/agents/figures/two-2.svg)
Figure labels: The same two corpora through a single LoRA adapter, in sequence | language 1 | language 2 | English control | Cross-entropy on each language after each stage | one adapter carried through both teaches, nats, lower is better | 1 | 3 | 5 | 7 | 9 | base | after language 1 | after language 2 | cross-entropy (nats) | 4.35 | 2.22 | 6.00 | 3.97 | 8.51 | 1.63 | 3.13 | 3.06 | 3.00 | Rank 256 on the feed-forward projections, 1.11 billion trainable parameters, learning rate 2e-05. Source: | plots/data/two_languages.json (lora block).
**[Figure 2](/demos/skills/two-languages#tl_f2_lora_sequential).** A single adapter learns language 1 (4.35 to 2.22 nats), then learning language 2 pushes language 1 to 6.00 nats, worse than before it was ever taught. English is untouched: the damage is between the two taught skills. Follow the first language’s loss across its own teach and the second-language teach. Lower loss is better. The adapter used its own training schedule, so this is not a matched one-pass comparison with Learner 1.0.
Swipe the figure to read every label →
### Learner 1.0 keeps the languages apart. LoRA swaps them
[Original chart: Learner 1.0 keeps the languages apart. LoRA swaps them](/agents/figures/two-3.svg)
Figure labels: Generation identity after each stage: Learner 1.0 versus LoRA | cross-language contamination (share of generated text in the other language) | Learner 1.0 (greedy) contamination is 0.00 in every cell | LoRA, original comparator recording: 1.00 after language 1 on language-2 prompts, 0.96 after language 2 on language-1 prompts | before | after language 1 | after language 2 | language 1 prompts, Learner 1.0 | language 1 prompts, LoRA | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.96 | language 2 prompts, Learner 1.0 | language 2 prompts, LoRA | 0.00 | 0.00 | 0.00 | 1.00 | 0.00 | 0.00 | Means over 8 prompts per cell, greedy decoding. Learner 1.0: one-pass recording (answers/session.json), English controls 4/4 at | every stage. LoRA: original comparator recording. Source: plots/data/two_languages.json.
**[Figure 3](/demos/skills/two-languages#tl_f3_generation_matrix).** Where the LoRA adapter answers language-1 prompts in language 2 after the second teach (0.96), Learner 1.0 keeps the two apart (0.00 in every cell). Learner 1.0: one-pass recording. LoRA: original comparator recording. Each cell measures cross-language contamination at a teaching stage: stems from the other language in a continuation. Lower is better. The cells are means over eight prompts at greedy decoding. These are identity scores, not fluency scores.
Same data, same order, opposite outcome. This is catastrophic forgetting measured in LoRA and its absence measured in ours. Per-language fluency at this dose is modest. These runs show accumulation without erasure rather than eloquence.
[See all 164 prompts, answers and probe scores](/demos/qa#two-languages)
### Retention survives the second teach, while fluency remains modest
Learner 1.0 learned from both corpora in one pass. Velenic’s held-out loss fell to 2.8880 and stayed at 2.8871 after Morvath. Morvath finished at 2.2842. The four English control questions remained correct at every stage. The adapter wiped language 1 at +3.78 nats and a 96% identity swap on identical bytes.
### 627 one-window updates teach both languages in one pass
Each language is one text file. The file is cut into 256-token windows under Learner 1.0's tokenizer: 276 windows for Velenic and 355 for Morvath. Each training window is presented exactly once, in a fixed order: 274 Velenic windows followed by 353 Morvath windows, batch size 1. Each window contains 256 tokens. Two windows per language are held out from training. That is 627 updates and 160,512 presented training tokens. The exact windows, in the order they were trained and with the held-out windows marked, are in the replication repository: [windows_velenic.jsonl](https://github.com/learnerlabs-ai/replications/blob/main/demos/two-languages/data/windows_velenic.jsonl) and [windows_morvath.jsonl](https://github.com/learnerlabs-ai/replications/blob/main/demos/two-languages/data/windows_morvath.jsonl).
**A real training excerpt.** The opening of the Velenic corpus, which falls inside one of its training windows:
Sesaenta romaenta zamiave. Nomoul nozloul masoos. Vrivre lovreen liroos. Ziva viskien rereos vrivoul meliir. Zloseir rinaenta sezleos. Zloseul zlazoen zozoave. Leskeen revro vrozaave.
Velenic is an invented language. The corpus was generated for this study.
**The training data.** The two corpora are in the replication repository, byte for byte as they were taught: [velenic.txt](https://github.com/learnerlabs-ai/replications/blob/main/demos/two-languages/data/velenic.txt) and [morvath.txt](https://github.com/learnerlabs-ai/replications/blob/main/demos/two-languages/data/morvath.txt). The continuation shown earlier on this page is an evaluation example, not training text.
### Learner scores share one recording. LoRA uses its original comparator
Every Learner 1.0 number on this page comes from one one-pass recording: the held-out loss readings, the training-loss series, the 80 continuations with their scores and the 12 English control answers (`demos/two-languages/answers/session.json`, `answers/generations.json`, `data/training-loss.jsonl`). The LoRA numbers come from the original comparator recording (`data/comparator-standard-adapter.json`), which was not re-run.
### The text and order match, but schedules and retention probes differ
The learner and the adapter were given identical corpus bytes in the identical language order, and generation identity is scored the same way for both: a continuation prompted in one language is checked for stems of the other. The training schedules are not the same: Learner 1.0 saw each 256-token window once, and the adapter used its own schedule, so this is not a matched one-pass comparison. Retention is not the same measurement on both sides. The learner's is held-out loss on the two withheld Velenic windows, read before and after the Morvath teach. The adapter's is cross-entropy on held-out first-language text computed by our harness (rank 256, 1.11 billion trainable parameters, learning rate 2e-05). Read the two as each method against its own earlier level, not as one number against the other.