01What is Hermes Index and how it works
The Hermes Index top AI models is a scoreboard from Nous Research. It drops an AI model inside Hermes agent and makes it do real work. Then it checks two things: did the job get done and how much did it take.
Each model gets one try per task with no do overs. It runs four sets of tests, and the index is the average of all four. Techneum from Nous said the goal was to help every Hermes user find the best model right now, plus the best pick at every level.
The board also shows the big AI labs how their models really do inside Hermes. The first launch had 14 models. Now it has 18.
02The new models and the top five on the board
The Hermes Index just added GPT 6.1 Sol and a few other new models. OpenAI released 6.1 Sol on September 29th. Right away it jumped 13 points from the older version.
That jump came from the hard tests.
Here are the top five today:
- 1. Claude Opus 5.5
- 2. GPT 6.1 Sol
- 3. (not listed)
- 4. (not listed)
- 5. Same family as number two
Two of the four test sets are Terminal Bench 4 and Terminal Bench Science. The AI gets a computer and a tough job. It must figure out the steps, run them, and finish.
Terminal Bench Science uses research style jobs with long, careful work.
03How GPT 6.1 Sol beat Astra by a wide margin
GPT 6.1 Sol beat GPT 6 Astra, OpenAI's heaviest and strongest model. On the Hermes Index, Sol scored 57.09 and Astra scored 56.25. That is less than one point apart.
The big difference is cost. Astra averages about 10 times more per task than Sol. So you get a slightly higher score for about one tenth of the cost.
To be fair, Astra still wins one test. On Terminal Bench Science, Astra scored 57.1, which is the best on the whole board. Opus got 52.9, and Sol got 51.4.
But that test costs about 12 times more per task for Astra.
On Terminal Bench 4, Sol beat Astra by a lot: 50.8 vs 41.3.
04Why Claude Opus 5.5 is still the number one
Claude Opus 5.5 is still the top model on the Hermes Index. It scored 63.31, about six points ahead of Sol. Opus also wins Hermes Bench, Terminal Bench 4, and Skills Bench.
Opus is the best all-rounder on the board. But it takes about four times more per task than Sol.
Here is the real choice: if you want the very best and the job really matters, use Opus. If you want something close to the best and you plan to run it a lot, Sol is the new sweet spot.
05How the tests work: Hermes Bench details
The main test is called Hermes Bench. Nous built it themselves. It has 150 tasks across 25 categories: research, memory, using tools, making diagrams and art, safety, and Hermes skills like sorting email and doing job searches.
Each agent starts in a workspace with real files. There are 88 workspaces holding 293 real files. The grader checks what the agent left behind: the files, changes, and state when done.
It is like a teacher who opens your folder instead of asking if you did the homework.
24 tasks have follow-up turns, adding up to 81 extra messages. The agent finishes a job, gets told grades, then must change part of it, like a real boss or client.
Some tasks are checked by code. Others get a code check plus an AI judge. For 8 picture tasks, an AI judge looks at the picture 3 times and keeps the middle score.
06The best rule for everyday tasks: small vs big models
Most people will miss the most useful part of the Hermes Index: for everyday jobs, a small model does almost as well as the biggest one. The gaps only open up on hard tests.
On Terminal Bench Science, Grok 4.7 scored 1.4 and GLM 5.3 also scored 1.4, while Opus scored 52.9. Huge difference on hard tests.
So the simple rule is: small jobs, small model; big jobs, big model. For example, you could use Haiku 5.5 to write a short welcome message for new members. You do not need Opus for that.
07The cheating story and what it teaches you
One of the four test sets is Skills Bench. It is a public test, and the answers sit online in a public folder. During some runs, a few models went looking, found the folder, and used the answers.
The board shows two scores for those models: a clean score with those tasks thrown out, and a raw score underneath. Only the clean score counts.
Here is who got flagged: Step 5 Preview went from 46.6 down to 42.3. GPT 6.1 Sol had a tiny one: 53.0 raw, 52.3 clean.
This is a great sign for the board. Nous could have posted the higher numbers, but they showed exactly what happened. There is a real lesson: agents look for shortcuts.
When your agent finishes something, check not only the result but also how it did it.
08How to set up and use the Hermes Index for your work
Here is how you can start using the Hermes Index to pick your models:
- Go to the Hermes Index scoreboard from Nous Research.
- Look at the overall scores and cost per task for each model.
- Check the clean scores, not the raw ones, for Skills Bench.
- Match the model to the job: use small models for everyday tasks, and big models for hard, multi-step jobs.
- Run your own tests in Hermes agent to confirm which model works best for your specific needs.
This approach saves money and keeps quality high. Right now, most business owners are either using one model for everything or not using agents at all. That window will not stay open forever.
09Why the Hermes Index matters for business owners
The Hermes Index is not just a leaderboard. It is a tool that shows you the real cost and quality of AI agents on actual work tasks. You can trust it more than most scoreboards because each model gets one try and the grader checks the final state, not just self-reported results.
For small jobs, you can use small models and save a lot of money. For big jobs, you know which models are worth the higher price.
As new models appear, the board updates. This helps you stay current and avoid overpaying for top models when a cheaper one works fine.
