cd ~/
Agentic AIMulti-Agent SystemsSelf-Evolving SystemsAgent ArchitectureResearch

I Trust Convergence More Than I Trust Benchmarks

Three teams shipped self-evolving agent frameworks in one week, and the architectural convergence tells us where production agents are heading.

Evyatar Bluzer
3 min read

I trust architectural convergence more than I trust any single benchmark. So when three teams from different labs, with different motivations, shipped self-evolving agent frameworks in the same week and the designs came out strikingly similar, I paid attention. Independent groups converging like this means the design space is telling you something.

The Three Frameworks

Memento-Skills, from a consortium of academic labs, treats skills as structured markdown files the agent authors for itself. On the GAIA benchmark it pushed accuracy from 52.3% to 66.0% - a 13.7 point jump - without touching the base model.

Hermes Agent v0.7.0, from Nous Research, landed April 3 with a built-in learning loop. The agent extracts procedural skills from execution traces, persists them to SQLite with full-text search, and retrieves them on similar tasks. After 10-20 runs on a repeated task type, execution speeds up 2-3x.

A-Evolve, also April 3, introduced a five-stage loop - Solve, Observe, Evolve, Gate, Reload - that directly mutates the agent's workspace files. Every mutation is a git commit tagged evo-1, evo-2. Rollback is one checkout away.

The Common Architecture

Strip the branding and they are the same system:

  1. Skills live in files - markdown, YAML, tool configs - editable by the agent and reviewable by humans, with the model weights left alone.
  2. An evolution loop runs off execution traces instead of gradients.
  3. A gate sits in front of every mutation: benchmark first, promote second.
  4. State is git-native, so every evolution is auditable and reversible.
Skill evolution loopFour boxes arranged in a ring joined by four arrows: skills in files, agent run, evolution loop, and a highlighted gate whose arrow feeds back into the skills box. Skills in files Agent run Evolution loop Gate markdown, YAML, tool configs model weights never touched task executed against the skills execution traces logged mutation proposed from traces, not gradients benchmark first, promote second skills loaded traces proposed mutation promoted mutation = git commit evo-1, evo-2; rollback is one checkout
The loop shared by Memento-Skills, Hermes Agent v0.7.0 and A-Evolve: skills live in files, runs leave traces, traces propose mutations, and a gate benchmarks each one before promoting it as a git commit, with the model weights never entering the loop.

None of them fine-tune the base model, and that is the clean break from everything we tried in 2024-2025.

What Does This Mean If You Ship Agents?

The implication is immediate: you no longer need a training pipeline to get an agent that improves. You need a skill store, a trace logger, and a gate. I have been building multi-agent systems on roughly this assumption for months, and three separate teams landing on the same architecture in the same week reads as validation that the approach generalizes beyond any single stack.

The shift running underneath all three frameworks is that the unit of learning in production agents is moving from model weights to procedural memory. Weights capture what the model knows in general, while skills capture what the agent has figured out on your specific problem, and for most deployments the second matters more.

The Next Quarter

Next quarter brings a Cambrian explosion of skill stores. Evaluation becomes the bottleneck, because the gate is now the hard part rather than the mutation, and an evaluator that is itself an agent brings its own preferences to the job. I would bet fine-tuning budgets quietly migrate toward skill-store infrastructure over the same period.

Comments