The evidence answers different questions about learning, retention and practical use

Publisher: Learner Labs. Updated 2026-10-13. Linked reports contain the full protocols and records.

On this page

Updating weights is necessary for this claim, but is not sufficient by itself

Fine-tuning already updates weights. Our claim is about what happens when narrow distributions arrive one after another. Does the model learn the current material, retain what was learnt earlier, and preserve measured base capabilities without replay, task labels or growing trainable capacity? The ten-skill experiment measures these together. The four-domain comparison uses LoRA to show that acquiring the new material and retaining the earlier material are separate outcomes.

The base is pretrained. We do not claim to have pretrained a new foundation model from scratch. The published studies use an architecturally modified pretrained model, with fixed added trainable capacity. Using pretrained priors and building a new learning mechanism are compatible. An open base model alone neither establishes nor rules out a proprietary technical contribution.

Preserving the base does not establish retention of newly learnt skills

A frozen base can remain unchanged while later updates interfere with newly learnt material. That is why we remeasure earlier acquired skills after later training. In the two-language test, LoRA's English control remains stable while its first invented language is forgotten. The distinguishing observation is retention of the new learning, not merely an unchanged English control or unchanged base parameters.

Weak acquisition is reported separately from forgetting

Some skills are difficult for the model. In the ten-skill report, Meridian ends at 84/225 and Oriel at 9/225. Oriel's own-end score was 12/225. We do not turn a weak initial score into a success by calling it retention. The per-skill before-teach, own-end and final scores let readers assess both acquisition and later loss. Bracken is the last skill, so its unchanged final score has no subsequent skill teach behind it. The result is substantial acquisition across the stream with largely stable acquired performance, not mastery of every task.

The sequence and updates are incremental even though the experiment has named tasks

Task names and colored plot bands describe the experiment to the reader. They are not task IDs supplied to the learner. In the ten-skill protocol, the learner sees only the prompt and answer. There is one example per update, no gradient accumulation and no replay of earlier examples. Trainable capacity is fixed before the stream. Checkpointing or restarting a training job does not by itself create a new learner or erase its state. The report records the checkpoint provenance.

LoRA loses earlier learning in the two published comparisons

The four-domain study matches approximately 1.14B trainable parameters and the same stream and step budget. Its 2× Learner experiment separately probes capacity. The report gives the tested learning rates and recorded base references. Its first evaluation at 25 updates does not isolate acquisition produced during those updates, so that observation is not a measured 12× speedup. There is one run per condition.

The two-language comparison uses identical corpus bytes and order, but different schedules and retention probes. Its LoRA first-language loss changes from 2.22 to 6.00 nats after the second teach. Learner's own held-out first-language loss changes from 2.8880 to 2.8871. These are within-method retention comparisons, not identical-protocol final-loss rankings.

Continual Learning Bench deliberately includes shared structure that earlier experience can help with later. We follow the official task schedules. The database task keeps the pre-migration half before the post-migration half, with three reported orders. The cohort task permutes studies within families. Sales uses one chronological order and has one completed run. The reports link the benchmark paper and task implementations. Our database and cohort reports use three runs, not the paper's five-rollout protocol.

Using that shared structure is the intended learning problem. The important test is whether the learner can retain and apply useful experience when the next question starts with fresh context.

A teacher prepares lessons, while the learner must use them later

The agent protocol uses a teacher LLM after an attempt. It reads that attempt and the available feedback, then writes grounded training examples. For this database recipe it also receives earlier trajectories and previously written procedures. The acting learner has fresh context, but the teaching pipeline is not memory-free. The sales teacher also receives earlier trajectories and previous lessons. The cohort report describes a different boundary: both learner and teacher reset between studies. The teacher does not take or submit the learner's benchmark turn. After feedback, its training examples can provide the next query or answer as a supervised target. The updated learner must act on the next question with fresh context. The teacher is part of the system and its supervision should be included when assessing the result and its cost.

The lesson at question 24 and its later use at question 33 give a concrete trace of the proposed benefit. The published protocol and the recorded trajectories are the evidence available for inspection.

The database runs average 6.0–13.5 examples per question

The database runs average 6.0–13.5 examples and 514–1,127 supervised tokens per question. These are run averages, not a fixed dose on every question. The learner starts from a pretrained model and the teacher helps construct lessons. The result demonstrates small incremental lessons in this setup, not learning arbitrary skills from scratch with five examples or a universal sample-complexity bound. Training dose and trajectories.

Raw benchmark score and gain answer different questions

Raw reward describes how well a system performs overall. Gain compares performance with its learning-off baseline. A stronger starting model with external memory may achieve higher raw reward while testing a different way to carry experience. Both observations matter. A higher gain is not automatically a higher raw score, and a lower raw rank does not by itself show that no useful learning occurred.

Our database report includes the dated leaderboard comparison and its methodology. The published leaderboard exclusivity claim refers to the compared snapshot. It is not a claim that Learner Labs is officially listed or that no other parametric method exists. The release is a company evaluation of the public benchmark. The report's own comparison qualifications remain part of the evidence.

Later successful actions and falling training loss measure different things

The database report records the loss on each example before its update. Later examples generally have lower losses. That is an observation about the examples produced by this teaching process. The examples, their difficulty and the learner's attempts change during the run, so this is not a fixed held-out test of the same questions over time. The later correct queries give separate behavioral evidence that taught facts and procedures can be used after a context reset. A fixed probe set and controlled lesson comparisons would help isolate why the online losses fall.

Factual recall and combining several facts are separate capabilities

In Recipe 2, the learner recalls 15 of 16 handbook answers. Four questions overlap the training examples verbatim. The separate document was held out from recipe development, but its evaluation still has one verbatim and two near overlaps. These conditions are recorded so recall can be judged on what was actually tested.

Combining facts is harder. On the fresh composition panel, the direct one-call method answers 1 of 16 correctly. A multi-call method that first recalls information from the learner's own weights reaches 10 of 16, with 3 of 9 decisions correct. The extra calls are part of that method and its cost. These results show useful recall and a remaining composition limitation.

A teaching recipe can improve one behavior without improving every outcome

The earlier database report records learning that also increased repeated queries. The later recipe reduces pooled repeated queries from 184 to 34 across three runs. Total reward changes from 30.00 to 30.07, with different effects by run. Better query behavior and higher total reward are therefore separate measurements. This is why the reports include the unsuccessful answers and the earlier recipe, not just the best trajectory. Update quality still needs to be evaluated.

The reports specify training hardware and measured serving costs

The ten-skill study ran on one H200. That says what hardware ran the experiment. It does not by itself show low end-to-end cost. The fact reports include timing and billing observations. The Motivation letter describes the current research preview: separate user weights, dedicated GPU sessions, checkpoint loading and limited GPU supply. Teacher computation, data preparation, training, loading and serving all matter in a cost comparison. The website does not demonstrate a production-scale shared serving cost curve.

The reported configurations also differ. The fact-serving report describes an 8,192-token context window. The database agent uses 32,000 tokens. These are settings for the respective experiments and service, not evidence that every API session offers the research configuration.

Fewer prompt tokens do not automatically mean a cheaper complete workload

The token-economics report measures prompt-token reductions across 68 questions. Its 89% figure is the input-token share of modeled spend in one cited coding-agent trace under stated pricing assumptions. It is not a measured 89% saving, nor a claim that all input is replaceable by learnt weights.

A break-even estimate needs the actual training bill, how much reusable context disappears per request, the cached-input price, how often that context is reused, and any added serving or loading costs. Live conversation, changing tool results and material that still needs to be provided cannot simply be assumed to disappear. A break-even count therefore depends on the workload and prices. The website does not establish a universal count. The measured token ratios and the projected dollar economics should be evaluated separately.

Released measurements support inspection, while the mechanism needs a different kind of verification

The replication repository exposes datasets, evaluation material, plots and recorded results. Readers can inspect errors, recompute reported quantities from released files and follow API-based reproduction instructions. Availability is report-specific. For example, the Recipe 2 article describes some records as forthcoming. A link to the repository does not establish that every promised artifact has been released. The proprietary model and full training and serving code are not released. That limits independent implementation and architectural inspection.

No independent replication or third-party mechanism audit is established by these pages. Availability of an NDA inspection, if requested, is a separate company decision. It should not be inferred from the public replication files.

Longer streams and larger models are the next experiments

The experiments test the published streams, orders, models and budgets. Longer streams, hundreds of domains, additional task orders, broader baselines and larger models would extend that evidence. The Motivation letter explicitly recognizes finite information capacity. Proposed 1T+ models and robotics are future directions. Positive backward transfer in the four-domain study is a small measured effect, not a promise that every update improves every previous skill.

Scientific results do not fill in unpublished company or safety information

The website identifies Anurup Ganguli and gives the founder's motivation. It does not provide a complete team, financing, customer, revenue or patent disclosure. Absence of that information on the site is not evidence that those things do not exist. We do not infer a moat, customer traction or business valuation from the experiments.

The current limited research preview is described in Motivation. Continual adaptation raises questions about harmful updates, malicious training data and whether safety behavior persists after teaching. Base-capability evaluations are not a comprehensive safety evaluation. Broader safety and deployment evidence would be needed to answer those questions.

The token-economics article states that a customer's weights are deleted when that checkpoint is deleted and that no backup is maintained at the time described. This is a stated checkpoint-deletion policy. It is not a demonstrated selective-unlearning result or an independent audit of deletion, isolation or security. Those are separate questions from whether newly learnt skills survive later teaching.