← back to work
case study · /llm-eval

LLM evaluation rubrics.

How I picked the models behind the QA agent stack. Browser-action accuracy, tool-call correctness, minimum tool calls, cost per task, and token usage — logged via runtime hooks. Eight model families compared.

research · staging model evaluation cost & tokens runtime hooks
↗ medium ↗ linkedin

Four metrics, weighted by use case.

  • Browser-action accuracy

    Given a UI task, does the agent click the right thing? Scored on a held-out suite of 50 tasks per agent surface.

  • Tool-call correctness

    When the agent has access to typed MCP tools, does it pick the right one with the right args? Catches "hallucinated shell commands" before they run.

  • Minimum tool calls

    Penalize agents that explore by trial-and-error. The right model gets to the answer in fewer calls, which scales cost.

  • Cost per task

    Real dollars on the eval suite. Includes input + output tokens, all retries, and any vision-token costs.

What we tested.

US-based and US-accessible LLMs. The list shifted over the year — this is the latest cut.

openai
ChatGPT / GPT family
Strong tool-call discipline, slightly higher cost per task.
anthropic
Claude family
Best minimum-tool-call score. Reliable for RCA-style summarization.
google
Gemini (incl. Gemini 3)
Multimodal lead — the only practical choice for the video-to-steps triage pipeline.
meta
Llama family
Strong open-weights option for cost-sensitive batch tasks. Self-hostable.
mistral
Mistral
Good cost / latency tradeoff for non-critical routing tasks.
nvidia
Nemotron
Strong reasoning, good cost on NVIDIA infra if you're already there.
liquid ai
Liquid
Promising for small-context fast routing. Watched for production-readiness.
+ stacks
ChatGPT Enterprise, GitHub Copilot org-MCP, Codex
Wrappers and deployment surfaces also compared — same model can score differently based on how the org wraps it.

Hooks for the truth.

The published model cards lie a little. Or rather, they tell the truth about idealized cost — not your cost. Your cost includes retries, tool-call loops the agent runs when it's unsure, vision tokens when the model auto-decides to look at a screenshot.

So I wired runtime hooks at the agent runtime layer:

  • Every tool call logged with model, input-token count, output-token count.
  • Every retry counted separately.
  • Every "decided to look at an attached image" charged at vision token rate.
  • Per-task aggregated cost surfaced in the evaluation dashboard.

That's how you find out the agent that scored best on accuracy was actually costing 3x the runner-up because it kept loop-retrying on edge cases. Made the model picks easier.

What we use, by use case.

multimodal triage
Gemini 3
Only practical choice for video → steps extraction.
RCA & summarization
Claude (Slack RCA agent)
Minimum tool calls, clean prose, low hallucination.
browser tasks
ChatGPT / Codex
Tool-call discipline. Built-in browser surface.
batch normalization
Llama (self-hosted)
Best cost/quality for the AIO TCC bulk-relinting work.