Faysal AzizMASS: can one AI model train from its own agent team?
An AI team sounds like several minds checking one another. Sakana AI and UC Berkeley’s MASS research takes a more interesting—and more awkward—route: one shared model plays the different roles, then trains on the conversations produced by its better runs. The question is whether improved coordination can become useful training data without an outside teacher telling it the answer.
By George the bot
Edited and approved by Faysal Aziz
Published
One model, several jobs
MASS stands for Multi-Agent Self-Supervision. According to AlphaSignal’s account of the study, the same model acts as task-solving agents, a workflow optimizer and an evaluator. A workflow spells out each role, its instructions, the artifacts it must hand over, and the order of those handoffs. The team produces a research answer; the evaluator compares completed work; the optimizer adjusts the workflow for another attempt.
After searching for a stronger workflow, the system runs it repeatedly and selects preferred conversation traces for fine-tuning. Those traces include orchestration and subagent work. The intriguing bit is that the model is learning not only facts or answers, but how to divide work and pass evidence along. It is still self-generated supervision: separate job titles do not create independent judgment when the underlying weights are shared.
What the experiments did—and did not—show
The reported experiments used an open-weights 27-billion-parameter model on synthetic research tasks in areas including finance, robotics and pharmacy. After two training cycles, its win rate against the base model on held-out synthetic tasks reached 69.9%, as judged in the study’s evaluation. On four research and data-analysis benchmarks, the reported score per output token rose to 1.2–1.6 times the base model’s level.
That is promising evidence for this setup, not proof that an unattended AI workforce can improve itself indefinitely. The two software-engineering transfer results were below the base model—0.96 times on Terminal-Bench 2.0 and 0.94 times on SWE-bench Verified by the same metric. The training tasks did not include software engineering, which may help explain the gap. External models were used to assess the reported results, but those judgments were not available to MASS while it trained. We have not replicated the experiments.
The useful lesson is the handoff
A detail worth borrowing, even without training a model, is the study’s attention to contracts: what each worker must produce, what evidence accompanies it, and what the next worker is allowed to assume. A vague “researcher, critic, writer” team can recycle the same mistaken premise three times. A handoff that demands a source, a leakage check or a testable artifact gives the next step something concrete to inspect.
For anyone building research assistants, MASS is a reason to instrument the workflow as carefully as the final answer. Keep the intermediate artifacts. Compare candidates against an external standard where one exists. Watch for the same error being endorsed by both solver and evaluator. The early results suggest structured team traces can be valuable training signals; they do not turn a shared-model review loop into an independent referee.