Daily AI benchmark brief: 2026-10-11
The new Two-Stage Measurement of Update Adherence benchmark tests whether a memory-enabled language-model agent follows a revised requirement rather than…
Every collection day on record
The complete archive of Benchmark Radar daily briefs, one page for every day the radar collected evidence.
Daily AI benchmark brief: 2026-10-10
The new milipoint-runs artifact exposes split leakage in the MiliPoint radar benchmark: overlapping windows let most test examples share frames with…
Daily AI benchmark brief: 2026-10-09
TRACE is a new diagnostic protocol for checking whether an agent’s benchmark score changed because its behavior changed or because the evaluator changed…
Daily AI benchmark brief: 2026-10-08
A new runtime-verification study exposes a data-design gap in existing agent-safety benchmarks. It replayed AgentDojo and STAC trajectories through formal…
Daily AI benchmark brief: 2026-10-07
The new Agent Network Traffic benchmark (ANT) tests whether security teams can audit large language model agents through network traffic without…
Daily AI benchmark brief: 2026-10-05
The new “Passing the Test You Trained On” study evaluates 15 prompt-injection detectors on benign and injected outputs reconstructed from real agent…
Daily AI benchmark brief: 2026-10-04
The new 2026 Cryo-EM Benchmark Dataset provides 12 protein structures deposited after every test set disclosed for MICA, an artificial intelligence system…
Daily AI benchmark brief: 2026-10-03
The updated retinal benchmark audit provides image-level manifests for patient-grouped, source-matched, and leave-one-source-out evaluations, plus…
Daily AI benchmark brief: 2026-10-02
Video-Index is a new meta-benchmark that audits whether video tests actually require video understanding. Its five-level “attack pyramid” tests…
Daily AI benchmark brief: 2026-10-01
The new study “Dating the Model” reports that hidden current-date text injected into system prompts changed measured performance across nine large…
Daily AI benchmark brief: 2026-09-30
The newly released UserProxyBench tests a hidden dependency in interactive agent benchmarks: whether the language model playing the user follows its…
Daily AI benchmark brief: 2026-09-29
TraceDance is a new release that turns real agent deployment traces into targeted benchmarks for user-specified undesirable behaviors. It evaluates the…
Daily AI benchmark brief: 2026-09-28
The new Finance Agents Benchmark packages financial due diligence as an end-to-end agent test: 50 tasks, 160 documents, 231 grading criteria, an execution…
Daily AI benchmark brief: 2026-09-27
The new Kalyvox Voice Benchmark 2026 evaluates one voice agent through 240 controlled French and English calls across 12 scenario families. It reports…
Daily AI benchmark brief: 2026-09-26
The new repo2bench release converts Python repositories into coding-agent tasks mined from version history or created by removing functions. It admits…
Daily AI benchmark brief: 2026-09-25
The new PartHackBench release tests whether partial-credit scorers for long-running tool agents reward misleading progress. It compares honest and…
Daily AI benchmark brief: 2026-09-24
The new Uncheatable Eval benchmark evaluates base language models by measuring how efficiently they losslessly compress newly published text that is…
Daily AI benchmark brief: 2026-09-23
A new B3DB audit tests whether stereoisomers inflate blood–brain barrier prediction scores because common two-dimensional molecular representations map…
Daily AI benchmark brief: 2026-09-22
The new Curation-Bench release evaluates whether coding agents can improve training-data selection through an iterative workflow: agents inspect data…
Daily AI benchmark brief: 2026-09-20
The newly released TC-Slang Guardrail Benchmark evaluates Traditional Chinese prompt-injection defenses by slang year and pairs each slang example with a…
Daily AI benchmark brief: 2026-09-18
The new SonoCorpus and SonoBase release pairs 456,963 ultrasound images and 1,626,085 expert segmentation masks with an interactive segmentation model…
Daily AI benchmark brief: 2026-09-17
The new lexEN benchmark replaces disputed word-sense labels with a conservative, human-adjudicated correction layer: 211 labels changed and 56 removed…
Daily AI benchmark brief: 2026-09-16
A new audit of 254 SWE-bench coding-agent submissions reports that the leaderboard cannot reliably order its leading entries: the top two resolve the same…
Daily AI benchmark brief: 2026-09-15
The new Sophea release evaluates a production Greek-English speech recognizer against nine simultaneous gates covering both languages, language…
Daily AI benchmark brief: 2026-09-14
The new KNOWS Benchmark evaluates web agents on 110 tasks that combine web research with producing documents, spreadsheets, and slide decks in Google…
Daily AI benchmark brief: 2026-09-13
The newly captured MusicAI background-music benchmark tests audio-capable large language models on 2,000 questions under clean audio, white noise, and 55…
Daily AI benchmark brief: 2026-09-12
The newly released Benchmark Radar combines a searchable catalog of AI evaluations with links to datasets and code, mentions in model cards and technical…
Daily AI benchmark brief: 2026-09-11
The new oncology visual question answering benchmark builds test items from private, single-institution radiology reports paired with three-dimensional…
Daily AI benchmark brief: 2026-09-10
The new “Double Measurement Confound” paper identifies a recurring agent-evaluation problem in this feed: a fixed software scaffold may make execution…
Daily AI benchmark brief: 2026-09-09
The new “Style Over Substance” study tests automated safety judges by preserving an answer byte-for-byte while adding tone-only wrappers, such as…
Daily AI benchmark brief: 2026-09-08
The new URL Intelligence Benchmark evaluates agents and web-analysis tools on operational hazards that ordinary answer-accuracy tests miss: redirects…
Daily AI benchmark brief: 2026-09-07
TruthInsightBench is a new benchmark for scientific-discovery agents that replaces reproduction-style tasks with 40 blind tasks from peer-reviewed…
Daily AI benchmark brief: 2026-09-06
The new Amharic Automatic Speech Recognition benchmark evaluates open models on 1,548 clips that its publisher says were unavailable for training. It…
Daily AI benchmark brief: 2026-09-05
Several new releases in this captured feed turn agent evaluation into domain-specific workflow testing. The clearest is…
Daily AI benchmark brief: 2026-09-04
A new preregistered audit, “Clean Engineering, Unstable Measurement,” tests whether a black-box large language model judge behaves like a stable…
Daily AI benchmark brief: 2026-09-03
Among today’s captured releases, EarlyEval introduces early outcome prediction: it estimates an agent’s final result from intermediate behavior and stops…
Daily AI benchmark brief: 2026-09-02
The newly released InSight benchmark tests agents that must actively interact with visualizations to verify 21,349 claims, rather than answer once from a…
Daily AI benchmark brief: 2026-09-01
EleutherAI released Language Model Evaluation Harness v0.4.13 with fixes that can change prior scores: test questions could leak into their own few-shot…
Daily AI benchmark brief: 2026-08-31
PCFBench reflects a recurring push in this captured feed to inspect an agent’s process rather than only its final answer. It separately tests…
Daily AI benchmark brief: 2026-08-30
The new INSIDER LLM Detection Benchmark evaluates models that may take harmful actions by comparing the model’s self-reported action log with an…
Daily AI benchmark brief: 2026-08-29
The new NBPO benchmark-generations dataset publishes every decoded model response used in three judge-based comparisons, allowing another evaluator to…
Daily AI benchmark brief: 2026-08-28
The new Same Model, Different Harness study holds the coding model and tasks fixed while changing how the agent harness manages conversation history and…
Daily AI benchmark brief: 2026-08-27
OpenCompass v0.5.4 is a substantive harness update, adding native VLMEvalKit-based multimodal evaluation, multi-round inference with the Multi-IF…
Daily AI benchmark brief: 2026-08-26
Across this captured feed, three new agent benchmarks make the evaluated unit an interactive model-plus-runtime system rather than a final answer…
Daily AI benchmark brief: 2026-08-25
New release SUSVIBES evaluates 12 coding-agent settings on 186 real-world feature requests for which human developers previously committed vulnerable…
Daily AI benchmark brief: 2026-08-24
No material GPT insight: No category moved far enough, persistently enough, or across enough independent sources to support a decision-useful finding in…
Daily AI benchmark brief: 2026-08-23
No material GPT insight: No category moved far enough, persistently enough, or across enough independent sources to support a material finding in today’s…
Daily AI benchmark brief: 2026-08-22
No material GPT insight: No material pattern was supported: the captured items did not show a sufficiently large, persistent, cross-source shift. Only 19…
Daily AI benchmark brief: 2026-08-21
No material GPT insight: No material pattern cleared the feed’s persistence and cross-source thresholds today. Only 65 of 198 corpus evidence records were…
Daily AI benchmark brief: 2026-08-20
No material GPT insight: No category changed far enough, persistently enough, and across enough sources to support a decision-useful pattern in this…
Daily AI benchmark brief: 2026-08-19
No material GPT insight: No category changed far enough, persistently enough, or across enough sources to support a material finding in this captured…
Daily AI benchmark brief: 2026-08-18
No material GPT insight: No category change was sufficiently large, persistent, and cross-source to support a material finding. Tracked metric movement…
Daily AI benchmark brief: 2026-08-17
Several new releases in the captured feed redesign evaluation around conditions hidden by static averages: evolving evidence and temporal cutoffs…
Daily AI benchmark brief: 2026-08-16
Agent benchmarking in the captured evidence is being designed around controlled execution, not just task sets: a new arena adds side-by-side and blind…
Daily AI benchmark brief: 2026-08-15
In today’s captured feed, three independent new releases evaluate agents as operational systems: LCAB preserves complete repair sessions and hardware…
Daily AI benchmark brief: 2026-08-14
Two new paper releases in this captured feed question whether aggregate agent scores measure deployable capability: one reports task interactions…
Daily AI benchmark brief: 2026-08-13
Across today’s captured new releases, several benchmarks require inspectable process evidence rather than scoring only final outcomes: replayable…
Daily AI benchmark brief: 2026-08-12
Several new releases in this captured feed bind benchmarks to deployment context: web-access APIs are compared across quality, latency, cost, and error…
Daily AI benchmark brief: 2026-08-11
Multiple new releases in the captured feed move agent evaluation beyond final-task success toward inspecting collaboration graphs, requirement recovery…
Daily AI benchmark brief: 2026-08-10
Several new releases in this captured feed evaluate whether a system reached an answer through valid evidence or execution, not merely whether the answer…
Daily AI benchmark brief: 2026-08-08
Across this feed’s new releases, benchmark design is moving toward specialized, structurally difficult tasks: open-ended scientific extraction…
Daily AI benchmark brief: 2026-08-07
Four newly released agent benchmarks in this captured feed expand evaluation beyond terminal task success: acquisition-stage privacy, learning across…
Daily AI benchmark brief: 2026-08-06
Across this captured feed, four newly released benchmarks evaluate behavior beyond a static, well-specified task: proactive bug discovery without issue…
Daily AI benchmark brief: 2026-08-05
Agentic artifacts rose to 26.3% of our captured feed over the last 5 days, against a 13.6% baseline across the prior 4 days (+12.7 percentage points).
Benchmark evidence summary: 2026-08-04
Benchmark Radar collected 212 evidence observations from 3 sources on 2026-08-04.
Benchmark evidence summary: 2026-08-03
Benchmark Radar collected 152 evidence observations from 3 sources on 2026-08-03.
Benchmark evidence summary: 2026-08-02
Benchmark Radar collected 89 evidence observations from 2 sources on 2026-08-02.
Benchmark evidence summary: 2026-08-01
Benchmark Radar collected 69 evidence observations from 2 sources on 2026-08-01.
Benchmark evidence summary: 2026-07-31
Benchmark Radar collected 60 evidence observations from 2 sources on 2026-07-31.
Benchmark evidence summary: 2026-07-30
Benchmark Radar collected 200 evidence observations from 3 sources on 2026-07-30.
Benchmark evidence summary: 2026-07-29
Benchmark Radar collected 201 evidence observations from 3 sources on 2026-07-29.
Benchmark evidence summary: 2026-07-28
Benchmark Radar collected 186 evidence observations from 3 sources on 2026-07-28.
Benchmark evidence summary: 2026-07-27
Benchmark Radar collected 30 evidence observations from 2 sources on 2026-07-27.
Benchmark evidence summary: 2026-07-26
Benchmark Radar collected 30 evidence observations from 1 sources on 2026-07-26.
Benchmark evidence summary: 2026-07-25
Benchmark Radar collected 32 evidence observations from 1 sources on 2026-07-25.
Benchmark evidence summary: 2026-07-24
Benchmark Radar collected 32 evidence observations from 1 sources on 2026-07-24.
Benchmark evidence summary: 2026-07-23
Benchmark Radar collected 20 evidence observations from 1 sources on 2026-07-23.