Agent benchmark archive

November 2026

November 9, 2026 · 6

1. WorkflowArena: Benchmarking Executable Workflow Orchestration in LLM-Based Multi-Agent Systems Work / office
2. Joi-Bench-Task: Benchmarking Task Coordination in Full-duplex Voice Agents through Interactive Simulation Tool use
3. Benchmarking language models and agents for scientific reasoning in evolving materials laboratories Science & research
4. Infrathon: Benchmarking Language Agents for Real-World, Long-Horizon AI Infra Optimization General
5. LEGION: Benchmarking Language Agents on Sparse-Signal Investigation Under Adversarial Noise Tool use
6. Publications as benchmarks: auditing AI agents on spatial transcriptomics alignment Safety

November 8, 2026 · 4

1. A Physical Testbed for Self-Driving Laboratories: Comparing Bayesian and Agentic Optimization in a Minimal Closed-Loop Color Formulation Task General
2. How to "train" a surgeon? Benchmarking generalist agents in surgical scene understanding GUI / computer use
3. ScrapeBench: Evaluating Legal Compliance of AI Agents in Website Scraping General
4. ErosionBench: Probing Alignment Decay in Enterprise Multi-Agent Systems Multi-agent

November 7, 2026 · 5

1. CyberBinGym: Can AI Agents Turn Binary Patches into Real Exploits? General
2. DFT-Agent Bench: Scaffold Effects and Execution Provenance in Autonomous DFT General
3. Agentic discrete-event simulation for manufacturing system design: an open 3D simulator, a benchmark and an industrial case Tool use
4. InfoOrgBench: Can Agents Turn Scattered Information into Coherent Hierarchies? General
5. TurnFlow: Benchmarking User-State Reasoning in Multi-Turn LLM Agents via Scene-Guided User Simulation General

November 6, 2026 · 8

1. MemoryDojo: Benchmarking Poisoning Through LLM Agent Memory General
2. Benchmarking Speaker Embedders in the Sub-Second Regime for Real-Time Voice AI Agents General
3. ABHBench: Evaluating Moral Decision-Making of Foundation Models from an Agentic Perspective Mobile
4. Done Is Not Correct: Measuring Silent Failures and Self-Verification Calibration When LLM Agents Take CAD Actions Tool use
5. Frontier3DBench: Benchmarking Coding Agents for 3D Creation across Objects, Scenes, and Interactive Open Worlds Coding / SWE
6. Phantom Consensus: Measuring Social Reasoning in Multi-Agent LLM Decision-Making Multi-agent
7. Flex-RSI: Benchmarking Recursive Self-Improvement of Frontier Agents RSI / AI4AI
8. DRAFT-Bench: Evaluating Frontier Agents Across the Professional Design Lifecycle Tool use

November 5, 2026 · 6

1. Benchmarks as Software: Terminal-Bench and the Case for Continuous Benchmarks Coding / SWE
2. Multimodal-SEA: Benchmarking AI Agent Security under Multimodal Social Engineering Attacks Multimodal
3. Terminal-style Mathematical Modeling Bench: The First Agentic and Verifiable Benchmark for Mathematical Modeling Multimodal
4. RoboARCade: Benchmarking Agentic Robot Control Embodied / robotics
5. MCPHallu: Benchmarking Reasoning, Execution, and Memory Hallucinations in MCP Agents Tool use
6. Beyond Instruction Override: Benchmarking Indirect Prompt Injection with Task-Tampering Attacks Tool use

November 4, 2026 · 7

1. τ-bench-LILT: Evaluating Conversational Agents on Realistic Multilingual Tool-Use Tasks Tool use
2. VABAR: A Variational Automation Benchmark for Agentic Robotics Coding / SWE
3. ServerBench: How can your coding agents design specialized LLM serving systems? Coding / SWE
4. LifeVal: Evaluating AI Model Performance on Personalized Agentic Consumer Tasks General
5. PMRecBench: A Benchmark for Personalization in Recommendation Agents under Multimodal Queries Multimodal
6. SWEOps-Bench: Evaluating Agents for End-to-End Resolution of Software-Induced Cloud Incidents Coding / SWE
7. TandemBench: Benchmarking Decision Models for Multi-Agent Collaboration Multi-agent

November 3, 2026 · 11

1. PersonaCodeBench: Evaluating Coding Agents in Persona-driven Human-Interactive Sessions Coding / SWE
2. Construct Validity of Coding Benchmarks for LLM-based Agentic Coding Tools Tool use
3. ViBench 2.0: A Benchmark for Long-Horizon Software Building Web & browser
4. SlackBench: Benchmarking Agents on Collaborative Projects Grounded in Real Code Repositories Work / office
5. Towards Robust Long-Lived Agents: Benchmarking Experience Lifecycles in Evolving Environments Safety
6. VisSearch-Bench: Evaluating Long Horizon Visual Search in Multimodal Agents Multimodal
7. AgenticFinBench: Benchmarking Financial Agents on Diverse Tasks from Real-World Requests General
8. TuneBench: Benchmarking AI Agents for Power System Dynamic Model Calibration Tool use
9. $τ$-Rec-Books: A Disclosure-Controlled Benchmark for Requirement Elicitation in Agentic Recommender Systems General
10. Silent Instrument Errors Reorder Agent Leaderboards General
11. DualWorldBench: Can Agents Plan Deliveries across Symbolic and Grounded Worlds? Embodied / robotics

November 2, 2026 · 15

1. SLATE: A Large-Scale Benchmark for Long-Horizon API Selection in E-commerce Tool Use Tool use
2. VeriHarm-Bench: Measuring Harmful Success in LLM Coding Agents Verifiably Coding / SWE
3. DistDebug-Bench: Can LLM Agents Diagnose Bugs in Distributed Systems? Coding / SWE
4. DeliverBench: Benchmarking AI Agents on Delivering Realistic Multi-Stage Professional Projects Safety
5. VagueBench: Benchmarking the Robustness of Embodied Agents on Ambient Context Illusions Embodied / robotics
6. Do Agents Still Follow the Rules? Benchmarking Rule Compliance under Agent Context Compaction Tool use
7. TLAPS-Bench: Formally Proving Correctness of Real-World System Specifications with TLA+ Proof System Tool use
8. RxnMiner: A Multimodal Agent for Structured Reaction Data Extraction from Chemistry Figures Multimodal
9. SelfImprove-Bench: Evaluating Generalization in Self-Improving Agents RSI / AI4AI
10. GeoGuesser Bench: Benchmarking Vision-Language Agents on Interactive Geolocation Multimodal
11. Parametric CAD Bench v3: Evaluating Agentic Parametric CAD Authoring from Text and Engineering Drawings General
12. CR-Bench-v2: Evaluating the Real-World Utility of AI Code Review Agents Coding / SWE
13. AMLBench: A Real-Incident Benchmark for Agentic Blockchain Anti-Money Laundering Tool use
14. Benchmark of THReD: Evaluating an LLM Agent and Its Preprocessing Tools for Turning Electronic Health Records into AI-Ready Datasets Tool use
15. MAGAR: Multimodal Agents for GUI-Based 3D Mechanical Assembly and Minimal Rework Multimodal

November 1, 2026 · 7

1. Switch-SWE: Do Coding Agents Hold Focus When Switching Between Unfinished Tasks? Coding / SWE
2. Launder-Bench: Can Repository-Level Agentic Code Laundering Be Detected? Coding / SWE
3. TriPhysBench: Evaluating Physical Intelligence for Embodied Agents Embodied / robotics
4. MAC-Bench: Benchmarking Multi-Agent Collaboration via Interconnected Tasks Multi-agent
5. MemHarmBench: Evaluating Cross-Session Prompt Injection in Memory-Persistent AI Agents Safety
6. CrossSWE: Benchmarking Cross-Repository Joint Development for Coding Agents Coding / SWE
7. I-PTC: Interactive Programmatic Tool Calling for Stateful Tool-Augmented Agents Tool use

October 2026

October 31, 2026 · 3

1. CivicBench: A Benchmark and Memory Framework for Web-Based Civic Data Analytics Tool use
2. Decision-Centric Research: A Calibrated Decision Model and Benchmark for Scientific Agents Science & research
3. AStockArena: Beyond Returns, Can We Entrust Real Capital to the Decisions of Financial Agents? Safety

October 30, 2026 · 7

1. SpBench: An Agent-Native Framework for Cross-Engine Lakehouse Benchmarking General
2. Decentralized Multi-Agent Coordination and Communication in Shared Codebases Multi-agent
3. OmniPiano: Diverse Dexterous Piano-Playing Challenges for Standard, Robust, Safe, and Multi-Agent RL Multi-agent
4. MemU-Bench: Benchmarking GUI Agents on Long-Horizon User Interactions GUI / computer use
5. TradeWar: A Benchmark for Execution-Grounded Trade Negotiation with Adaptive LLM Agents Dialogue
6. BlindToChange: AI Agents Can Find Changes When Asked But Do They Look When Needed? General
7. ANGLER: Verifier-Grounded Benchmarking with an Adaptive Attacker for Web Agents Web & browser

October 29, 2026 · 16

1. BABA-Bench: Evaluating Long-Horizon Patient-Advocacy Agents General
2. ReplayRCA: Measuring Post-hoc Diagnosis and Live Oversight of Agent Failures by Counterfactual Replay Multimodal
3. P-ContextBench: From Personal Fact Recall to Personalized Agent Behavior Safety
4. Benchmarking Forecasting Agents on Yesterday's Tomorrows Tool use
5. TSFLab: A Self-Improving Agent-Agnostic Workbench for Time Series Forecasting Research RSI / AI4AI
6. More Rules, Longer Lists? Benchmarking Self-Evolving Agents on a Hidden-Rule Curriculum General
7. MAServeBench: Benchmarking Multi-Agent LLM Serving Systems with Hierarchical Collaboration Workloads Multi-agent
8. DynaRev: Benchmarking Web Agents under Dynamic Goal Revision Web & browser
9. Bank-World: A Benchmark for Evaluating Agents on Banking Workflows General
10. Judging Before Propagation: Task-Graph Contextual Integrity for Multi-Agent Privacy Multi-agent
11. ATLAS-Bench: An Agentic Tool-Use Benchmark for Local Area Search Tool use
12. Well-Specified and Well-Tested? Assessing SWE-bench-Style Datasets with a Self-Evolving Agent Coding / SWE
13. AgentDecept: Benchmarking Deceptive Behavior in Tool-Using LLM Agents under Pressure Tool use
14. Can Image Agents Complete Visual Commissions? A Benchmark and Modular Framework Tool use
15. What Does "Now" Mean to Your Computer-Use Agent? Stale Screenshots, Page Loads, and Which Check to Run Before a Click GUI / computer use
16. PrivacyFlow: Privacy in AI Agents through Information Flow Analysis and Consequence-Aware Risk Management Safety

October 28, 2026 · 7

1. TreasuryQA Bench: Can AI Agents Work an Archive of Scanned Financial Records? Tool use
2. How Deterministic SAST and AI Security Agents Divide the Work: Coverage, Harness Policy and Cost in Snyk VulnBench 2.0 Coding / SWE
3. An Alignment Benchmark for Enterprise Agents on Era by Eon Work / office
4. Investor's Last Exam: Can AI Agents Do Equity Investment Without Seeing the Future? Work / office
5. EPSBench: Benchmarking LLM Agents for Earnings Forecasting General
6. DecompGym: Benchmarking LLM Agents on Designing Decomposition Algorithms Coding / SWE
7. SkillCT: A Multidimensional Benchmark and Execution-Free Baseline for Agent Skill Assessment Coding / SWE

October 27, 2026 · 11

1. M4Bench: Evaluating Procedural Specification for Clinical EHR Derivation Agents Coding / SWE
2. FeatureArc: Benchmarking Coding Agents on Evolving Features Coding / SWE
3. LiveSearch: Questions as Executable Verifier Code for a Live Benchmark of Professional-Source Financial Search Agents Tool use
4. OpenGray: a benchmark for evaluating the decisions of radiotherapy planning agents General
5. A Simulation Testbed for Self-Governing Human-Machine Institutions Multi-agent
6. AutoVend: Benchmarking LLM Agents on Long-Horizon Autonomous Vending Operations General
7. InfraDev: Can Coding Agents Develop AI Infrastructure, and How Can We Tell? Coding / SWE
8. Benchmarking LLM Agents for Behaviorally Adequate Test Generation from Software Requirements Coding / SWE
9. NextEval: An Agent-Based Solver and Benchmark for Derivative-Free Optimization Tool use
10. Benchmarking Task Takeover Across Agent Handoffs Coding / SWE
11. CogArena: Benchmarking Multimodal Agents on Interactive Behavioral Experiments Multimodal

October 26, 2026 · 14

1. Benchmarking Coding Agents under Evolving Requirements Coding / SWE
2. Omni-InteractScope: Benchmarking Proactive and Agentic Interaction in Continuous Omnimodal Streams Tool use
3. EmbodiedLab: Benchmarking Embodied Agents That Act to Know Embodied / robotics
4. PROBE: Frontier Coding Agents Find Different Bugs Than Maintainers Fix Coding / SWE
5. AgenticCity-Bench: Longitudinal Multimodal Observations of Urban Dynamics with Agentic Reasoning Multimodal
6. SkillLifeBench: Benchmarking Security Across the Lifecycle of Agent Skills Web & browser
7. V$^3$AST-Bench: Verifiable, Valid, and Versatile Safety Evaluation for Coding Agents Coding / SWE
8. UnityBench-Mini3D: Evaluating LLM Agents for 3D Game Development in Unity Coding / SWE
9. Co-Design Bench: Multimodal Collaborative Coding Evaluation for Agentic Models Multimodal
10. PatentAgentBench: Benchmarking LLM Agents with Professional Patent Task Suites Safety
11. NumTrace: Auditing the Number Provenance of Financial Analyst Agents General
12. The Right Fact, the Wrong Decision: SciValidityBench for Scientific Agent Memory Science & research
13. SearchHorizonBench: A Benchmark for Search Value in AI Agents Tool use
14. SlowLab: Benchmarking Autonomous AI Agents on a Year of Experiments They Cannot Undo General

October 25, 2026 · 7

1. AgentProofBench: Benchmarking Reusable Verifiers for Web Agents Web & browser
2. SentryBench: A Cost-Sensitive Benchmark for Adversarial Robustness of Tool-Using LLM Agents in Public-Safety Deployments Safety
3. $Δτ$-Bench: Pushing Agents Toward a Moving Capability Tool use
4. R2A-Sales: A Policy-Atom-Matched Benchmark for Rule-to-Action Gaps in Goal-Directed Sales Agents Tool use
5. Task Success Is Not Enough: Side-effect-Aware Evaluation of Tool-Using Language Model Agents Safety
6. Can LLMs Trust Appropriately? Evaluating Calibrated Reliance on Imperfect Partners General
7. Continual Learning Exam: Benchmarking Long-Horizon RSI and Experiential Learning in LLM Agents RSI / AI4AI

October 24, 2026 · 1

1. SkillShift-Bench: Benchmarking Skill Learning under Environment Evolution for Web Agents Web & browser

October 23, 2026 · 13

1. AOS-Bench: Can LLM Agents Manage Dynamic Workloads? Coding / SWE
2. Navigating Epistemic Parity in LLM Agents: A Benchmark for Cross-Source Conflict Resolution Between Memory and Procedural Skills Tool use
3. On Task, Off Track: Benchmarking LLM and Agent Robustness to Near-Target Prompt Injection Safety
4. SCI-AGENT-EXAM: Benchmarking Agents on Scientific Research Tasks Science & research
5. Async-RBench: Benchmarking Asynchronous Task Execution in LLM Agents General
6. Can LLM Annotators Match Human Consistency? Benchmarking Human Agreement and Optimizing Multi-Agent with Contrastive Knowledge Multi-agent
7. 4DMemory: Benchmarking Long-Horizon Spatiotemporal Memory with Harness-Augmented Agents Multimodal
8. DMAIC-Bench: From Self-Reflection to Self-Restraint for Agents in Multi-Stage Analytical Tasks General
9. Trial by Trace: Benchmarking AI Agent Harnesses for Reliability and Failure Attribution Tool use
10. SWE-Shift: Evaluating Coding Agents in Evolving Workspaces Coding / SWE
11. ImageRSI: Verification-Driven Recursive Self-Improvement for Agentic Image Generation RSI / AI4AI
12. PersonaScienceBench: Evaluating Scientific Agents with Diverse Simulated Researchers Science & research
13. CRM-Agent-Bench: Benchmarking Tool-Using Agents in Stateful Enterprise Environments Coding / SWE

October 22, 2026 · 12

1. SWE-Clar: Can Coding Agents Clarify Incomplete Client Requirements and Deliver Greenfield Software? Coding / SWE
2. EpiSEM-Bench: Evaluating Safety and Evidence Recovery in LLM Agents under Missing Evidence Safety
3. DuoCook: An Overcooked-Inspired Benchmark for Visual Planning and Coordination Coding / SWE
4. AgentVQA: A Unified Benchmark for Agentic Visual Understanding Web & browser
5. AgentPhishing: A Benchmark of LLM Email Agents Under Adaptive Attack GUI / computer use
6. RIDAR: Safeguarding Research Integrity in Autonomous Research Agents via Automated Detection and Runtime Intervention Science & research
7. Beyond Watching and Searching: Benchmarking Agents for Video-centered Deep Research Reports Multimodal
8. ColaBench: Evaluating Memory-Guided Knowledge Acquisition in Personal Agents General
9. UserArena: Benchmarking Interactive User Simulation for Long-Horizon Shopping Agents General
10. What the Specification Did Not Say: Specification-Oracle Gaps in Test-Graded Coding-Agent Benchmarks Safety
11. MBABench-2.0: Evaluating AI Agents on Constructing Financial Spreadsheets Beyond Simple Verifiability Work / office
12. Compact-Bench: Measuring What Context Compaction Costs Long-Horizon Coding Agents Coding / SWE

October 21, 2026 · 8

1. IFC-MCQ: Automatically Generating Multiple-Choice Benchmarks for Evaluating LLM Agents on IFC Data Retrieval Tool use
2. Why Bother Me Again? Benchmarking Personal Agent Memory for Interaction Preferences General
3. Quant-Arena: Benchmarking LLM Agents for Quantitative Trading Research General
4. Did the Problem Actually Get Harder? Testing LLMs on Paired P and NP-Complete Problems Hardened by Adversarial Multi-Agent Rewriting Multi-agent
5. Benchmarking Coding Agents for System-Level Performance Optimization Coding / SWE
6. Benchmarking and Error Analysis of Deep Search and Deep Research LLM Agents Safety
7. MOSAIC: An Omni-Modal Benchmark for Long-Term Memory and Cross-Modal Reasoning in Conversational Agents General
8. GenXR-Bench: Benchmarking Coding Agents for Interactive XR Development Coding / SWE

October 20, 2026 · 22

1. Behavior Beyond Text: Evaluating Refusal Transfer Failure in LLM Agents Safety
2. Back Up Before You Regret: Measuring Backup Awareness and Automating Backup in Coding Agents Coding / SWE
3. Automating ML for Science: Can Frontier Agents Climb Scientific Hills in the Wild? Science & research
4. After the Attacker Leaves: Benchmarking Carryover Attacks on LLM Agents General
5. CAPSafe: A Benchmark for Contextual Action and Plan Safety Reasoning in Embodied Agents Safety
6. MMEnterprise-Bench: Evaluating Enterprise Agents over Evolving Multimodal Records Work / office
7. RITE: Evaluating End-to-End Resilience to Silent Tool Errors in E-Commerce Agentic Systems Tool use
8. CUA-Recovery: Benchmarking Error Recovery in Long-Horizon Computer-Use Agents GUI / computer use
9. PVBench: Quantifying Physical Plausibility in Video Generation with Object-Level Physical Measurements Multi-agent
10. StebBench: Diagnosing Real-World Procedural Understanding in AI Agents General
11. Fake News, Real Crashes: Information Injection Attacks on LLM Financial Trading Agents General
12. When Coding Agents Outpace Review: Evaluating PRD-Driven Development in Mature Enterprise Repositories Coding / SWE
13. Discovery-Probability Graph Planning for Embodied Agents in Tasks with Hidden Cooperation Requirements Embodied / robotics
14. Reliable Before Profitable: QuantHarness for Governing Financial Strategy Development with LLM Agents General
15. LeakBench: Auditing Reward Hacking through Prediction-Time Leakage in Clinical Modeling Agents General
16. SWE-Residual: Evaluating Starting-State Robustness of Coding Agents Coding / SWE
17. Explore or Confirm? LLM Agents Misallocate Measurements Under Noise Tool use
18. CME-SpectrumBench: Can LLMs Analyze Condensed Matter Spectral Data? Coding / SWE
19. BlenderWorld: Benchmarking Visual Agents for 3D Creation from Real-World References Multimodal
20. DuplexLedger: Benchmarking Transactional Memory in Full-Duplex Speech Agents General
21. Act on Your Behalf, Beyond Your Authorization: Benchmarking LLM Agent Authorization Alignment Safety
22. MemDev-Bench: Does Memory Help Code Agents?Benchmarking Memory Systems over Real Repository Histories Coding / SWE

October 19, 2026 · 15

1. Evolving Self-tuning Benchmarks for Long-Horizon Tool Use Tool use
2. RealControlBench: A Benchmark for Interpretability-Informed Fine-Grained Control of LLMs and Agents Safety
3. EGI-BENCH: An Enterprise General Intelligence Benchmark Suite Coding / SWE
4. LoWAgent: A Low-Resource Multilingual Benchmark for Web Agents Web & browser
5. RefCraft: Benchmarking Visual-Reference-Conditioned Agentic Generation through Web Development Multimodal
6. ReuF2F: Benchmarking Coding Agents on Formalized Undergraduate Research Problems Coding / SWE
7. Embodied Playground: A Long-Horizon Agentic Robotics Benchmark for Physical, Strategic, and Spatial Reasoning Embodied / robotics
8. VerifierBench: Benchmarking the Generation of Executable Verifiers for LLM Agents Tool use
9. SpatialScout: Can VLM Agents Actively Explore and Map 3D Scenes? Multimodal
10. VibeTraining-Bench: Data-Trainer-Harness Co-Design for Iterative System Improvement RSI / AI4AI
11. From Code to Playable Systems: CEGame-Bench for Long-Horizon Game Creation Agents Coding / SWE
12. FinHopper: A Progressive Multi-Hop Benchmark for Complex Financial Agents with Step-wise Evaluation Tool use
13. CXBench: A Benchmark for Process-Compliant LLM Agents in Regulated Customer-Experience Workflows Tool use
14. AdaExploreBench: Benchmarking LLM Agents for Proactive Exploration and Test-time Adaptation in Evolving Environments General
15. When Should AI Chime In? Benchmarking Timely Contribution of Proactive Agents in Multi-Party Chat General

October 18, 2026 · 5

1. MemDerive: A Diagnostic Benchmark for Agent Memory over Evolving Data General
2. BrandEval: A Two-Track Multi-Agent Benchmark for Risk-Sensitive LLM Crisis Communication Multi-agent
3. MemBubble: Benchmarking How Agent Memory Shapes LLM Response Diversity General
4. Auto-MD-bench: Evaluating Autoresearch Agents for Real-World Medical Tasks Safety
5. MAMath-Bench: Process-Aware and Robust Evaluation of Multimodal Multi-Agent Mathematical Reasoning Multi-agent

October 17, 2026 · 3

1. HSPR: Recovering Software-Agent Benchmark Scores via Hierarchical Posterior Inference Coding / SWE
2. HABIT-Bench: Evaluating Habit Induction from Longitudinal Weak Evidence in Agent Memory Dialogue
3. Can Agents Keep Secrets They Were Never Told? Benchmarking Latent Privacy Risks Across Sessions with MemJoin General

October 16, 2026 · 27

1. ROAR: Benchmarking LMM Agents under Real-Time Constraints in Multiplayer Action-Shooter Games Multi-agent
2. AM-Bench: A Unified Taxonomy and Evaluation Suite for Agentic Misalignment Safety
3. SWE-Bench Pro V2: A Cleaner, Harder-to-Game Coding Benchmark Coding / SWE
4. AI-Driven Reproducibility and Analytical Robustness in Scientific Discovery Science & research
5. ABF-Bench: A Verifiable Benchmark for Agentic Blockchain Forensics General
6. GCAM-Gym: A Benchmark for Agent-Based Climate Policy Search Tool use
7. MLOps-Bench: Benchmarking AI Agents on Production ML Engineering in Real Repositories Coding / SWE
8. GatewayBench: Unified Capability and Security Evaluation Platform for MCP Gateways Tool use
9. MobileInject: Evaluating Reasoning-to-Action Security of MLLM-Based Mobile Agents under Cross-Modal Consistency Injection Mobile
10. MobileJailBench: Red Queen Jailbreak Benchmarking for Mobile Agents Mobile
11. Beyond Marginal Influence: Interaction-Aware Memory Selection for LLM Agents General
12. PersonaCUABench: Benchmarking Computer-Use Agents Serving Diverse Personas GUI / computer use
13. KiranaBench: Can a Language Model Run a Neighbourhood Shop? Multi-agent
14. X-RIC: An Agentic RAN-Intelligent Controller Benchmark for Enhancing Reinforcement Learning General
15. Beyond Downstream QA: Rethinking How We Evaluate Agent Memory General
16. OntoAct: Hypothesis-Based Neuro-Symbolic Agents through Ontological Abstraction General
17. From Research Goals to Scientific Closure: Evaluating Agents Through Evidence-Supported Physics Research Science & research
18. Auditing Escalation Reliability in Healthcare LLM Agents: A Perturbation-Controlled Benchmark of Workflow-State Failures General
19. Kolab: Benchmarking Human-Agent Collaboration in Long-Horizon Workflows Work / office
20. ToolBottleneckBench: Measuring Tool-Agent Performance under Controlled Pressures and Fixed Harness Budgets Tool use
21. Sci$\Pi$vot: Benchmarking Principal-Investigator Judgment at Pivotal Moments of Scientific AutoResearch Agents Science & research
22. OmicsBench: a long-horizon multi-turn benchmark of AI agents on research-study-scale bioinformatics Science & research
23. SCARCE: Emergent Institutions in Long-Horizon Multi-Agent Coding Systems Multi-agent
24. INFUSE: A Unified Surface Benchmark for Indirect Prompt Injection in Agentic Systems Tool use
25. HPCAgent-Bench: A Framework for Sound Benchmarking of Code-Optimizing Agents, Harnesses, and Skills Coding / SWE
26. MS-Eval: Benchmarking Sub-Agent Delegation and Orchestration in MCP-Intensive Working Scenarios Tool use
27. Benchmarking and Enhancing Agentic Causal Discovery General

October 15, 2026 · 2

1. AuditKIT: A Unified Evaluation Spine for Benchmarking, Judging, and Comparing Large Language Models and Agentic Systems Tool use
2. WEAVE: Weaving Real-World Code Evolution into Dependency-Structured Tasks for Benchmarking and Training Code Agents Coding / SWE

October 14, 2026 · 11

1. Doc2Harness: Benchmarking LLMs on Long-Horizon Harness Engineering of Coding Agents Coding / SWE
2. What Makes a Social Agent's Interactions Effective? Learning from Verifiable Outcomes in Paired Worlds General
3. HM3D-M3: A Benchmark for Multi-Agent, Multi-Modal, Multi-Object Goal Navigation Multi-agent
4. When Non-Adversarial Noise Hurts Learning: Benchmarking Skill Self-Evolution in Agents Safety
5. How GUI Agents Fail: GraceBench for Execution Stability Diagnosis GUI / computer use
6. RuleTrek: Benchmarking Agentic Rule Reasoning in Complex Scenarios General
7. J-tau: A Fully Localized Japanese Benchmark for Tool-Agent-User Interaction Dialogue
8. JockeyBench: Evaluating LLM Agents on Horse-Racing Decision-Making Safety
9. $τ$-Emotion-Bench: Exposing the Interaction Fragility of LLM Agents through Real-World User Emotions Tool use
10. MMA-SafetyBench: A Benchmark for Multimodal Agent Safety Evaluation Safety
11. OS-Bench: Evaluating SWE Agents on Cross-OS Code Migration Coding / SWE

October 13, 2026 · 16

1. Into the Unknown: Benchmarking AI agents on open scientific research problems Science & research
2. AgentCIScope: Grounded Contextual Integrity Evaluation for LLM Agents Tool use
3. CLASHarness: Benchmarking Multi-Agent Conflict in Computer Use via a Controllable and Verifiable Generation Pipeline GUI / computer use
4. CyberArena: Benchmarking AI Agents in Live Symmetric Attack-and-Defense Combat General
5. Autonomous Science Bench: Evaluating AI Agents for Autonomous Scientific Discovery Science & research
6. I2O2I-Nav: A Benchmark with a Modular Agent for Long-Horizon Indoor-to-Outdoor-to-Indoor Navigation Embodied / robotics
7. SWE-MISBEHAVIOR: Benchmarking and Mitigating Boundary Violations in Software Coding Agents Coding / SWE
8. LifeSearch: Benchmarking Deep Research Agents for Real-World Life and Productivity Science & research
9. AM-Bench: Understanding Agents under Task Underspecification across Models and Harnesses Coding / SWE
10. Can Agents Forget When Asked? A Benchmark for Memory Unlearning in LLM Agents Dialogue
11. Searching for What Matters: Open-Ended Discovery of Multiple Hypotheses with Agentic Data Exploration General
12. ProtoArena: Can AI Agents Discover and Exploit Real-World Cross-Component Protocol Vulnerabilities? General
13. Benchmarking and Training Reward Models for Cross-Source Evidence Grounding in Search Agents General
14. Empowering Edge Agents: A Systematic Analysis of Edge-Cloud Collaboration in LLM Agent Execution General
15. GIFT-Eval-CTX: A Benchmark For Contextual Forecasting Model Evaluation Tool use
16. LatentRiskBench: Measuring State-Dependent Risks in Enterprise Agents Work / office

October 12, 2026 · 13

1. Get In, Decide, Prove: A Merchant-Side Readiness Benchmark for Agentic Commerce Web & browser
2. PhysCodeGym: Benchmarking Coding Agents for Physically Verifiable 4D Reconstruction Coding / SWE
3. OpenDGO: Benchmarking Multi-Agent Organizations Across Diverse Application Domains Multi-agent
4. CWE-bench: Refocusing Agentic Security from Exploitation to Defense Coding / SWE
5. SpecGuardBench: Can Coding Agents Construct Reliable Validation Harnesses from Natural Language Specifications? Coding / SWE
6. ANCHOR: Assessing Naturalistic Constraint Hierarchies and Ordering in Repository-Grounded Agentic Coding Coding / SWE
7. Learn from Move: the Next Step for GUI Agents GUI / computer use
8. CineRecon: Benchmarking Frontier Models for Agentic Cinematic Reconstruction Multimodal
9. CivWorldBench: Evaluating Language World Models in Structured Multi-Agent Sandbox Worlds Multi-agent
10. EvoWeb: A Benchmark for Coding Agents on Evolving Web Applications Coding / SWE
11. TRAP: Tool-result Replay Attack for Probing Misbinding in LLM Agents Tool use
12. Embodied Skill Bench: Benchmarking How Well Agents with Embodied Skills Perform on Physical Tasks Embodied / robotics
13. When Agent Evaluation Becomes Unreliable: An Empirical Study of Oracle Failures and Configuration-Dependent Measurement in an LLM Agent Benchmark Tool use

October 11, 2026 · 2

1. A Decade in a Day: Synthesizing Agent-Native Interfaces from Human-Oriented Software Evolution GUI / computer use
2. MedHarnessBench: A Unified Benchmark for Healthcare Agent Evaluation Multimodal

October 10, 2026 · 6

1. Trace Evolution Bench: Measuring Whether Agents Improve from Their Own Environment Traces General
2. SkillHorizon: Constructing Long-Horizon Agent Tasks through Skill Composition Tool use
3. SkillForgeBench: Evaluating Agent Reliability Grounded in Agent Skill Specifications Work / office
4. ASIWorld: Benchmarking Autonomous Agents in a Persistent Survival Sandbox General
5. DSABench: Benchmarking Source Discovery, Coordination, and Execution for Multi-Database Agents General
6. MemForm: Benchmarking Memory Formation Poisoning in LLM Agents General

October 9, 2026 · 11

1. RATED: Reliability-Aware AI Agent Tool Retrieval via Execution Dynamics Safety
2. MC-EvoBench: Auditing Experience-Driven Self-Improvement in Minecraft Agents RSI / AI4AI
3. (When) Should a Software Factory be Lights-off? Benchmarking Code Review Agents on Bug Detection Coding / SWE
4. APEX-Harm: Evaluating Agent Safety in Shared Professional Workspaces Safety
5. MAGMA-Bench: Evaluating Long-Horizon Embodied Agents under Dynamic Tasks and Execution Failures Embodied / robotics
6. AgentSWE: Can Coding Agents Build the Agent You Actually Want? Coding / SWE
7. Understanding and Mitigating Cross-Session Attacks via Persistent Artifacts in Agent Skill Composition General
8. DRYLAB-Bench: Benchmarking the Biosecurity Risk of Agentic LLM Mutation Design Safety
9. EmbodiedCLBench: Evaluating Continual Learning for Self-Evolving Embodied Agents Embodied / robotics
10. SmartSalesBench: A Resistance-Aware Multi-Turn Benchmark for Evaluating LLM Sales Agents Dialogue
11. When Charts Deceive Web Agents: A Paired Benchmark for Misleading Visualization-Grounded Tasks Web & browser

October 8, 2026 · 12

1. MADS: Multi-Agent Dialog Simulation for Robust Conversational Agents Multi-agent
2. SpireBench: Evaluating Long-Horizon Strategic Play of Agentic Models with Slay the Spire Safety
3. SWE-Duel: Towards a Self-Scaling Arena for Coding Agents Coding / SWE
4. What Should Embodied Agents Remember? Benchmarking and Evaluating Memory Modality for Embodied Agents Embodied / robotics
5. BrickBench: Evaluating Agentic Brick Design Coding / SWE
6. Can AI Agents Learn Their Way to the Top? Evaluating Heuristic Learning in a Long-Running Game Agent Competition General
7. DataSense-Bench: The First Step Toward an AI Scientist RSI / AI4AI
8. A Closer Look at Agentic BBO: Benchmarking LLM Agents for Black-Box Optimization Tool use
9. Evaluating Local Language Model Agents for Reproducible Data Engineering: An Empirical Software Engineering Study of Mobility Workflows Coding / SWE
10. Closed-loop evaluation of LLM agents for embedded software development Coding / SWE
11. Mine Odyssey: Benchmarking Spatial Agentic Intelligence in the Wild General
12. AgentHorizon: Evaluating Agentic Judges for Long-Horizon Computer-Use Tasks GUI / computer use

October 7, 2026 · 9

1. StoreBench: A Live-Commerce Environment for Evaluating and Training Autonomous Operator Agents Tool use
2. RobotWorld: Benchmarking Multimodal Agents for Robot Use Across Diverse Tasks and Embodiments Embodied / robotics
3. RoboQuest: Generalist Physical Agents that Search, Inspect and Test Embodied / robotics
4. CANDO: Cooperative Agentic Network for Layout Design Optimization Multi-agent
5. AgentTime: Can Agents Estimate and Control Their Own Runtime? GUI / computer use
6. Agent4RE: A Self-Refining Multi-agent Framework for End-to-End Software Requirements Engineering and Benchmarking Multi-agent
7. Coding-Agent Benchmarks Should Match Their Users' Task Flows Coding / SWE
8. DUDA-Bench: Benchmarking LLM Agents on Multimodal Data-Driven Urban Diagnosis Multimodal
9. RT-Safe: Benchmarking Agent Safety in Real-Time Embodied Environment Safety

October 6, 2026 · 19

1. GeoNatureAgent (GNA): A Framework and Benchmark for Pre-Production Evaluation of Tool-Using Agents on Geospatial and Environmental Tasks Tool use
2. Agent in a Bottle: Can LLM Agents Turn Their Capabilities Into Cheap, Scalable Artifacts? General
3. WorldSolver: Can LLM Agents Simulate the Physical Dynamics via Solver Generation? Coding / SWE
4. ScienceClaw: Benchmarking Continual Self-Evolution of AI-for-Science Agents Across the Natural and Social Sciences Science & research
5. ParanoiaEval: Benchmarking Unnecessary Defensive Work in Agentic Coding Coding / SWE
6. Wiki-Talkie: Multilingual Benchmarking of Persona-Based Agents on Real-World Discussions Safety
7. MASC: A Multi-Agent Self-Calibration Framework with Latent Construct Alignment for Consistent Client Role-Playing in Psychological Counseling Multi-agent
8. Learn2Play Bench: How Well Do LLM Agents Learn from Experience in Unfamiliar Environments? General
9. DSV-Mem: Evaluating Multimodal Memory in Professional Workflows for MLLM Agents Work / office
10. SpeedrunBench: Challenging LLM Agents with Video Game Speedrunning General
11. DecepEval: A Benchmark for Evaluating Deception in LLM Agents General
12. ServeLearnBench: How Well Can Agents Self-Improve from Serving Experience? RSI / AI4AI
13. CredLeakBench: Evaluating Credential Leakage and Recovery in LLM Agents General
14. ST-Bench: A Spatial-Temporal Benchmark for Multi-Agent System Generation on Scientific Research Tasks Multi-agent
15. HarnessSecurity-Bench: Do Security Mechanisms Really Protect Coding Agent Harnesses? Coding / SWE
16. BioStudyBench: Evaluating Agents on Post-Cutoff Biomedical Studies Tool use
17. BiGym 2.0: Benchmarking Learned and Agent-Developed Policies for Humanoid Household Manipulation Coding / SWE
18. Personal-Agent Mediated Recommendation with Cross-Platform User History General
19. CheckerBench: Can Long-Horizon Agents Synthesize Static-Analysis Checkers? Coding / SWE

October 5, 2026 · 8

1. Verifying Coordination in Parallel Coding Agents: NP-Bench and a Scheduling Planner Coding / SWE
2. BazaarBench: Delegation Safety in Decentralized C2C Marketplaces Run by LLM Agents Safety
3. ANT: A Multi-Granularity Network Traffic Dataset and Benchmark for Agents Behavior Auditing Safety
4. AgentPrivArena: Evaluating and Auditing Real-world AI Agent Privacy Tool use
5. From Benchmark to Bench: Can Agents Survive Real-World Drug Discovery? Tool use
6. Correct Code, Broken Contributions? SWE-CC: Benchmarking Repository Policy Compliance for Coding Agents Coding / SWE
7. Benchmarking Jailbreak Guardrails for Embodied Agents Safety
8. Measurement-First Auditing of Agentic Leaderboards: Contamination Susceptibility, Matched-Control Re-evaluation, and Scorer Validation Coding / SWE

October 4, 2026 · 5

1. UndoBench: Separating Task Competence from Recovery Capability in Tool-Using AI Agents Safety
2. DelegationBench: Measuring When AI Agents Should Ask Before Acting Tool use
3. AutoSciBench: Autonomous Benchmark Generation for Evaluating Scientific Agents Science & research
4. SearchJev: A Fast and Calibrated System-1 Model for Search Agents General
5. Assembling Insights for Agentic Machine Learning Engineering Systems RSI / AI4AI

October 3, 2026 · 6

1. MedImageOSWorld: Benchmarking GUI Agents for Medical Image Consoles GUI / computer use
2. MASBench: Benchmarking LLM-based Multi-Agent Collaboration under Partial Observability Multi-agent
3. Video2World: Benchmarking Coding Agents for Interactive World Modeling from Embodied Videos Coding / SWE
4. LMBuild: Evaluating LLM Agents for Generating Buildable and Functional Structures Tool use
5. Benchmarking Psychological Dynamics in Generative Agents General
6. EvalResearchBench: Can AI Agents Design Their Own Evaluations? RSI / AI4AI

October 2, 2026 · 20

1. SkillScriptBench: Benchmarking Self-Evolution of Executable Agent Skill Packages Beyond Markdown General
2. 4DCodeBench: Benchmarking Agents on Inverse Graphics of Dynamic Scenes Coding / SWE
3. ReFract: Benchmarking Perspective Awareness in Language Model Agents with Text World Models Tool use
4. D2K-Bench: Can LLM Agents Turn Expert Designs into Efficient GPU Kernels? Tool use
5. EvoRiskBench: An Evolving Benchmark for Runtime Security Risks in Workspace Agents Safety
6. DyadMem: A Long-Term Memory Benchmark of How Agents Work with Users General
7. KaliBench: A Fine-Grained Benchmark for Cybersecurity Tool Use on Kali Linux with Runtime-Free Verifiable Rewards Tool use
8. Argo-Bench: Evaluating Data Agents on Enterprise-Scale Workflows Science & research
9. HumanoidToolBench: Benchmarking Humanoid Tool Use from Selection to Mobile Execution Tool use
10. Auditing Web Agent Evaluation on WebArena-Lite: Human Review of Outcomes and Trajectories Web & browser
11. Pay for the Fault, Not the Flow: Label-Free In-Flow Multi-Agent Workflow Optimization Multi-agent
12. Beyond Leaderboards: Tokenomics of Agentic Small Language Model Ensembles General
13. Agent Evaluation Reliability: More Tasks Won't (Always) Fix An Agent Leaderboard General
14. Incident-Arena: Getting agents to the last nine of reliability Coding / SWE
15. CompMat-Bench: Benchmarking AI Agents for Computational Materials Science Science & research
16. Legal Research Bench: Measuring End-to-End Reliability in Long-Horizon Legal Research Agents Science & research
17. EurekaBench: Measuring Agentic Ability to Discover New Scientific Insights Science & research
18. DuplexSpeechBench-Document Grounding: Benchmarking Document Grounding and Hallucinations in Voice Agents Work / office
19. A2Z GameSpec-Bench: How Faithfully Can Coding Agents Generate Games from Game Design Specifications? Coding / SWE
20. WebUIProof: Benchmarking WebUI Code Generators with UI-Agent Execution Harness Coding / SWE

October 1, 2026 · 6

1. Open-Endedness Bench: Measuring Epistemic Process from Agent Records General
2. OpenGameEval: Benchmarking Agentic Programming and Exploration in a Stateful Game Engine Tool use
3. Finding the Move Is Not Winning the Game: XiangqiBench for Closed-Loop Evaluation of LLM Agents General
4. VisAudit: Evaluating Multimodal Agents for Visual Diagnosis and Repair Multimodal
5. ArrivalBench: Agent-Generated Data Pipelines Are Correct Once and Wrong Under Time General
6. SimuVerity: Benchmarking Agents for Engineering-Grade Simulink Model Generation Safety

September 2026

September 30, 2026 · 10

1. WorldAuditBench: Interactive 3D World Auditing with Multimodal Agents Multimodal
2. cua-speedrun: Standardized Benchmarking of the Speed of Computer-Use Agents GUI / computer use
3. DoGBench: Can Agents Meet Expert Standards for User-Facing Documentation? Coding / SWE
4. OSWorld-Science: A Benchmark of Computer Use Agents for Learning and Using Scientific Software GUI / computer use
5. GraphMAS: A Systematic Benchmark of Multi-Agent Coordination for Graph Learning Multi-agent
6. EHR-RobustGym: Benchmarking and Training Agents for Robust Clinical Reasoning Safety
7. MADBench: Benchmarking the Security of Multi-Agent Debate Multi-agent
8. Trustworthy Runtime Error Healing in Real-World Repositories: A Benchmark and Guardrail Coding / SWE
9. RealWorldShop: Benchmarking and Improving Conversational Shopping Agents in Real-World E-commerce General
10. Talk2Agent: Benchmarking Voice Interfaces for Text Agents GUI / computer use

September 29, 2026 · 16

1. AgBench: Agentic AI Benchmarks for Personal AI Devices Tool use
2. VAmoS Part Deux: Harder, More Realistic Voice-Agent Simulation Tool use
3. NAQD Env: A benchmark for selective withdrawal in language agents General
4. E2E-SWE: Benchmarking LLMs on Building Working Codebases from Scratch Coding / SWE
5. UserProxyBench: Evaluating LLM User Simulators for Agent Benchmarks and Training Dialogue
6. EnterpriseBench: Benchmarking LLM Agents on Enterprise-Level Strategic Reasoning and Decision-Making Work / office
7. Zero2Repo: Can Coding Agents Build Repositories from Scratch? Coding / SWE
8. Do Agent Benchmarks Do What They Say? An Executable-Contract Audit of Tool-Using Agent Environments Tool use
9. LoLBench: Evaluating Coding Agents with Long-Horizon Proposals on Large Software Systems Coding / SWE
10. MatToolBench: Benchmarking Multimodal Agents in Real-World Materials Science Workflows Multimodal
11. PrecogUI: Proactive GUI Agents via Pre-cognitive Simulation and Experience Retrieval GUI / computer use
12. Code4Scene: Benchmarking Coding Agents for Constructing and Editing 3D Scenes Coding / SWE
13. Frontier Autolab: Organizational Memory, Adversarial Dissent and Temporal Leakage in Multi-Agent LLM Firms Across Fifty Years of Technological Change Multi-agent
14. WitnessGym: Benchmarking Coding Agents on the Construction of Bug Witnesses Coding / SWE
15. MAADBench: The Refreshable Paradigm for Anomaly Detection in Multi-Agent Systems Multi-agent
16. AdaptArena: Evaluating Test-Time Personalization of Web Agents Web & browser

September 28, 2026 · 35

1. CheatBench: Measuring Reward Gaming in AI Agents Work / office
2. From Dead Code and Static Requirements to Working Engines: Software Revival with Coding Agents Coding / SWE
3. Failure-Transparent Agents: Benchmarking Post-Failure Reporting in Tool-Using Language Models Tool use
4. GPUPhysBench: Benchmarking Coding Agents for Correct and Efficient GPU Physics Simulation Coding / SWE
5. SEABench: Benchmarking Endogenous Misalignment In Self-Evolving Agents Safety
6. MechBench: Can AI Scientific Agents Discover Mechanisms Beyond Phenomenal Laws? Science & research
7. Timeline-Bench: Evaluating Agents on Realistic Video-Editing Tasks, from Raw Footage to Final Cut Dialogue
8. JRDB-AVR: An Active Visual Reasoning Benchmark for Embodied Agents in Real-World Environments Embodied / robotics
9. AutoDataBench: Can Agents Write the Data That Feeds the Self-Improvement Loop? RSI / AI4AI
10. APEX-Voice: Can Voice Agents Complete Professional Workflows Through Full-Duplex Interaction Work / office
11. PDEU-Bench: Benchmarking the Personalized Planning Lifecycle of Tool-Calling LLM Agents Tool use
12. DGF-Bench: A Benchmark for Simulating and Auditing Deception Against Multi-Agent Governance Boards Multi-agent
13. CEO Arena: Evaluating Long-Horizon Multi-Agent Decision-Making in Competitive Markets Multi-agent
14. CoSec: Benchmarking Agent Security in Communities Dialogue
15. LongPuzzleBench: Evaluating GUI Agents on Long-Horizon Visual Puzzles GUI / computer use
16. VehicleArena: A Realistic Urban Environment for Multi-Agent Driving Multi-agent
17. FromPitch2Board: Benchmarking LLM Agents in Long-Horizon Football Management General
18. AgentPerfBench: A Benchmarking and Evaluation Suite for Inference Performance of Agentic LLMs Coding / SWE
19. The Marathon of Scientific Reasoning: Robustness of Scientific Agents to Perturbations in Multi-Turn Interactions Science & research
20. MASTraceBench: Diagnosing Collaboration Gains through Proposal Trajectories in LLM-Based Multi-Agent Systems Multi-agent
21. PowerBench: A Benchmark for Agentic Retrieval and Reasoning in Power Systems Tool use
22. ReproBench: Benchmarking LLM Agents on Reproducing Vulnerability From Scratch General
23. AgentHop: A Diagnostic Benchmark for Agentic Multi-Hop Scientific Question Answering Science & research
24. BIABench: Evaluating AI agents on real-world bioimage analysis tasks Multimodal
25. AgentWorld: Benchmarking Long-Horizon Collaboration of Multi-agent LLMs Multi-agent
26. Compact Documentation for Coding Agents: A Benchmark, an Optimizer, and Why It Does Not Transfer Coding / SWE
27. PriceBench: A Diagnostic Benchmark for Price, Quality, and Brand Preferences in LLM Booking Agents General
28. SciHorizon-eLab: An Agentic Protocol-to-Task Compiler for Scalable Benchmarking of Scientific Embodied Agents Science & research
29. A Benchmark and Diagnostic Study of Epistemic Admission in Shared Agent Memory Multi-agent
30. RecToolBench: Benchmarking Recommendation-Specific Tool Orchestration under Fuzzy User Intent Tool use
31. The Hard Part Comes After Search: Benchmarking Web Agents on Synthesizing, Organizing, and Displaying Knowledge Web & browser
32. Stealth Apart, Harm Together: Skill Cascading Attacks on Skill-Based Agent Systems Multi-agent
33. ScopeBench: Do Agents Preserve Engagement Boundaries Under Goal Pressure? General
34. RLE-Bench: A Qualifying Exam for Coding Agents as Robot Learning Engineers Coding / SWE
35. PainterBench: A Figural Divergent-Thinking Benchmark for Tool-Using Language Models Multimodal

September 27, 2026 · 9

1. DynGraphAgentBench: A Benchmark for Agentic Lifecycle Control in Dynamic Graph Anomaly Detection General
2. Identical Runs, Different Results: Benchmarking AI Coding Agents on Open-Weight Models Coding / SWE
3. CodeActionBench: Evaluating Agentic Code-as-Policy for Embodied Manipulation Embodied / robotics
4. Type-Safe Decision Frameworks for Agentic 5G Control: A Theory-Driven Testbed Characterization of Where They Can Be Applied Safety
5. SWE-Game: Can Coding Agents Build the Games We Want? Coding / SWE
6. StoryEngine: A State-Grounded Agentic Framework for Video Storytelling General
7. Long-Horizon Analog Design Bench: Benchmarking Agents on Hours-Long Analog and Mixed-Signal Circuit Design Tasks Coding / SWE
8. ECG-Scroll: A Long-Horizon, Streaming Benchmark and Agent Environment for Interpretation of Ambulatory Electrocardiograms Multimodal
9. Agentic Commerce Bench: Measuring Fraud Detection for Agents That Spend Money Safety

September 26, 2026 · 14

1. SRE-Marathon: A Continuous, Change-Driven Benchmark for Autonomous Site Reliability Agents General
2. AgentTell: Behavioural Side-Channel Leakage in Browser-Use Agents Web & browser
3. AgentHabit: Characterizing Distinct Behaviors of Agents on Everyday Tasks General
4. AsynCodeBench: Benchmarking Collaboration of Asynchronous Multi-Agent Systems in Software Engineering Multi-agent
5. "You're Right, Let Me Fix It": How LLM Agents Damage Correct Work When Falsely Accused Safety
6. VulContextBench: A Benchmark for Security Context Retrieval in Coding Agents Coding / SWE
7. CUA-SWE: When Computer-Use Agents Meet Visual Software Engineering GUI / computer use
8. OmniSmartHome: A Multimodal Reasoning Benchmark for Smart-Home Agents Multimodal
9. RepoMAS: Solving Progressively Specified Tasks with Issue-Driven Multi-Agent Systems Multi-agent
10. CyberClear: A Benchmark for LLM Agent Systems on APT Attack Chain Provenance Multi-agent
11. DashAct: A Progressive Diagnostic Benchmark for GUI Agents in Interactive Dashboard Analysis GUI / computer use
12. What Can a Leaderboard Certify? Compositional Controllability for Fair Evaluation and Training of Biomedical Literature-Review Agents Science & research
13. AutoPDEBench: Benchmarking LLM Auto-Research for Neural PDE Solver Design Multi-agent
14. CoMemBench: Benchmarking Collaborative Memory Boundaries across Multi-Agent Workflow Topologies Multi-agent

September 25, 2026 · 9

1. IndicFDB: Benchmarking Full-Duplex Voice Agents across Indian Languages Safety
2. CaptchaArena: A Large-Scale, Fine-Grained Dataset for Training Computer-Use Agents on Interactive CAPTCHAs GUI / computer use
3. EmailBench: A Benchmark for Evaluating LLM Agents on Enterprise Email and Productivity Tasks Tool use
4. Era by Eon: Benchmarking Enterprise Agents on Hidden Knowledge Work / office
5. PPTBench: Can Coding Agents Reconstruct the Visual World through Structured, Editable Slides Work / office
6. PartHackBench: Certified Equal-Progress Stress Tests for Partial-Credit Tool-Agent Evaluation Tool use
7. Delay-of-Gratification as a Multi-Agent Survival Micro-benchmark for Long-Horizon LLMs: Social Exposure, Personas, and Tool Use Budgets Multi-agent
8. Forecast-Dojo: Replayable Environments for Benchmarking and Training LLM Forecasting Agents Tool use
9. RECLAIM: Can Agents Reproduce the Claims of Machine Learning Papers? General

September 24, 2026 · 5

1. PASTABench: Proactive Assessment of Sequential Trajectories for Agent Safety Safety
2. Schr\"odinger's Code Repository: Have LLMs Learned SWE-bench or Memorized It? Coding / SWE
3. FDE-Bench: Evaluating LLM Agents for Deployment Environment Configuration Tool use
4. WhatWorkedBench: Benchmarking Experimental Understanding in AI Agents Science & research
5. What Makes a Terminal-Bench Task Hard? Separating Genuine Hardness from Fake-Hardness on an Adjudicated Agentic Corpus General

September 23, 2026 · 5

1. SWE-Serve: Benchmarking Agentic Engineering For Production Inference Serving Coding / SWE
2. Evaluating Coding Agents on Kernel Exploit Generation Coding / SWE
3. Passes Alone, Fails Together: Benchmarking Semantic Coordination in Parallel LLM-Agent Development Coding / SWE
4. Trains but Doesn't Learn: A Post-Training Delivery Benchmark for LLM Agents as Forward-Deployed Engineers General
5. FinFIRST: Benchmarking Search Agents for Financial Information Retrieval, Sourcing and Traceability Tool use

September 22, 2026 · 18

1. onPanda: Efficient Annotation of On-Policy Alignment Data for LLMs and Agents via Token-Level Correction Tool use
2. DolphinBench: Mapping the Pareto Frontier of Agent Memory General
3. OSWorld-Pro: Process-based Evaluation for Computer Use Agents GUI / computer use
4. MSI-Bench: Evaluating Multi-Speaker Voice Interaction for Collaborative AI Agents Tool use
5. Beyond Endpoint Performance: Process-Level Evaluation of Self-Evolving Agents Tool use
6. DUMA-Bench: A Dual-Control Multi-Agent Benchmark for Evaluating LLM Agent Security Multi-agent
7. MemCalib: Benchmarking and Optimizing Memory Use in LLM Agents Safety
8. WidgetVA: A Widget-Centric Framework and Benchmark for Agentic Visual Analytics Multimodal
9. Synthesizing Reactive Character Behaviors for Continuous Games via Programmatic Policy Search Coding / SWE
10. ACLArena: Agent Continue Learning in Multi-stage Post-training General
11. MobileCybench: Evaluating Agent Vulnerability Discovery via Executable Probes Coding / SWE
12. PhysAI-Bench: A Benchmark for LLM-Based Agentic Decision-Making in Autonomous UAV-Centric Physical AI Embodied / robotics
13. VibeMemBench: Evaluating Memory Systems for Coding Agents on Real Repository Coding Tasks Coding / SWE
14. BabelArena: A Large-Scale Multilingual Benchmark for LLM Agents Tool use
15. TicTacBench: Benchmarking Timing Closure Capabilities of Coding Agents Coding / SWE
16. CraftBench-UE: Deterministic Evaluation for Coding Agents in Unreal Engine Coding / SWE
17. GameReplica: A Benchmark for Black-Box Visual Game Replication by Vision-Language Agents Multimodal
18. Recognition, Simulation, and Refusal: A Contamination-Aware Study of Classic Psychological Effects in LLM Agents Safety

September 21, 2026 · 7

1. APort Vault: Benchmarking AI Agent Payment Authorization with the Open Agent Passport Tool use
2. GameLogicBench: Evaluating Coding Agents on Runtime Game Logic with Tick-Level State Assertions Coding / SWE
3. OpenMAS-GCom. A Diagnostic Benchmark for Graph-enhanced Multi-Agent Systems Multi-agent
4. Omni Demand Understanding: A Benchmark for Contextual User-Intent Inference in Multimodal Interaction Multimodal
5. AgentVidBench: A Multi-Hop Video Question Answering Benchmark for Evaluating MLLM Agents Multimodal
6. Efficient Benchmarking in Production: A Study of an Evolving LLM Agent General
7. BI-Agent and BI-Bench: Towards Automating End-to-End Business Intelligence Tool use

September 18, 2026 · 2

1. MTVA-Bench: Evaluating the Language Model Inside Cascaded Voice Agents Tool use
2. ClashBench: Conflicts Leading Agents to Seize and Harm Safety

September 17, 2026 · 7

1. StableEval Arena: A Cost-Aware Agentic Benchmark for Stablecoin Price Stability Prediction Safety
2. Beyond Outcomes: Dual-View Relational Learning for Efficient Agent Benchmarking Coding / SWE
3. PACT: Can Enterprise AI Assistants Be Trusted Under Pressure? Safety
4. AutoTuneBench: Trustworthy Measurement for Agent Auto-Tuning of LLM Serving Engines General
5. RideWay: Benchmarking Efficient Task Completion for Tool-Using Language Agents Dialogue
6. Locating Hidden Failures Makes Long-Horizon Agents More Reliable Coding / SWE
7. ERPBench: A State-Grounded Evaluation Paradigm for Computer-Use Agents in Enterprise Software Work / office

September 16, 2026 · 2

1. Coding Agents Have Converged: Why the SWE-bench Leaderboard Can No Longer Order Its Top Entries, and What to Measure Instead Coding / SWE
2. CADWorld: Computer-Use Benchmark for Long-Horizon Computer-Aided Design GUI / computer use

September 15, 2026 · 9

1. V-ICAL Bench: Evaluating Video In-Context Learning for Multimodal Agents in Interactive Environments Multimodal
2. BVB: Benchmarking Agentic Video Understanding via Programmatic Reconstruction in Blender Multimodal
3. MTAC-IFBench: Benchmarking Instruction-Following in Multi-Turn Agentic Coding Coding / SWE
4. MemRiskBench: Trace-Aware Risk-Preserving Evaluation for Long-Horizon LLM Agents Safety
5. CoArena: Evaluating Computer-Use and Multi-Agent Systems in Real Time GUI / computer use
6. PriMobiBench: Characterizing Visual Privacy Leakage in VLM-Driven Mobile GUI Agents GUI / computer use
7. Identity Is More Than Recall: A Benchmark for Persistent Identity in Deployed AI Agents General
8. $\tau$-Elicitation: Benchmarking multi-turn entity extraction in voice agents Safety
9. BudgetBench: A Budget-Tiered Protocol and Pilot Harness for Memory Strategy Evaluation in Local Large Language Model Agents Coding / SWE

September 11, 2026 · 4

1. MP-Bench: Evaluating Voice Agents as a Multiparty Conversation Participant General
2. K-Bench: A Benchmark for LLM Unlearning in Agentic Deployments Tool use
3. VRL-Bench: Benchmarking agents on computer control tasks under finite trial budgets General
4. ParaRecover: A Process-Level Benchmark for Error Localization and Recovery in Parallel Tool-Use Agents Tool use

September 10, 2026 · 8

1. GTA: Graph Theory Agent and Benchmark for Algorithmic Graph Reasoning with LLMs Coding / SWE
2. ChurnBench: A Drift-Aware Benchmark Demonstrating That Refresh Scheduling, Not Cache Age, Governs Staleness in Agentic AI General
3. Mr.LHDR: A Benchmark for Multimodal Real-World Long-Horizon Deep Research Agents Science & research
4. Sci-MMR: Benchmarking Multi-Step Evidence-Grounded Scientific Reasoning in Multimodal Agents Science & research
5. Can LLMs Normalize Databases? A Benchmark and Multi-Agent Framework for Schema Normalization Multi-agent
6. Overview of the NLPCC 2026 Shared Task 11: Agent-Based Experiment Reproduction from Scientific Papers Science & research
7. Benchmark Radar: A Living Database and Search Engine for AI Benchmarks and Evaluation Safety
8. ProMediConv: Benchmarking Proactive Conversational Agents in Legal Dispute Mediation Dialogue

September 9, 2026 · 3

1. SpeechAnnotator: A Context-Aware Multi-Agent Framework and Benchmark for Multidimensional Speech Annotation Multi-agent
2. The Era by Eon Benchmark: A Generated Enterprise Estate with Exact Ground Truth for Benchmarking LLM Agents Tool use
3. LexAgentHallu: A Hierarchical Benchmark for Profiling Hallucinations in Legal Agents Tool use

September 8, 2026 · 8

1. Benchmarking Agentic HLS Design Tasks With HLS-Eval Coding / SWE
2. From Fixed Keys to Readable Schemas: Small Language Models for Vehicle Agent Function Calls Mobile
3. Benchmarking Hybrid Deep Research Across Database Querying and Web Search Tool use
4. SAEScientist-Bench: Can AI Agents Conduct Autonomous SAE Interpretability Research? RSI / AI4AI
5. Q2D-Web: A Large-Scale Benchmark for Retrieval in Agentic RAG Systems General
6. MorphoOrgaAgent: A Foundation-Model-Based Multi-Agent System for Autonomous Organoid Analysis Multi-agent
7. SWE-Bench Pro Verified: A Reliable Benchmark for Software Engineering Agents Coding / SWE
8. SchemeArena: Factorized Stress Testing of Scheming in LLM Agents Safety

September 7, 2026 · 7

1. VEX-Bench: Benchmarking LLM Agents for Assessing Exploitability of Software Supply Chain Vulnerabilities General
2. What Does an LLM-Agent Leaderboard Rank Actually Compare? Coding / SWE
3. APPSim-Bench: Bridging Real-world Apps and Reproducible Evaluation for Mobile GUI Agents GUI / computer use
4. AgentIdeaBench: Benchmarking Scientific Ideation in the Agent Era Science & research
5. FinCUABuild: Can Agents Build Reliable Benchmarks for Dynamic Financial Computer Use? GUI / computer use
6. Beyond Fluent Generation: A CPU Reliability Benchmark for MCP-Style Tool Calling in Sub-2B Small Language Models for Edge Deployment Tool use
7. AgentDrift: A Step-Labeled Benchmark of Injection-Hijacked LLM Agent Trajectories Tool use

September 6, 2026 · 1

1. The Double Measurement Confound in Agent Benchmarks: De-Scaffolding, Ground-Truth Scoring, and Reliability Beyond the Mean Safety

September 5, 2026 · 3

1. DAREBench: Deployment-Aware and Reliable Evaluation of Models as Agents Multimodal
2. Evaluating Deep-Search Agents under Hierarchical Web Evidence Poisoning General
3. Agentic BAIM-LLM Evaluation (ABLE): Benchmarking LLM Use of Protein Design Tools Tool use

September 4, 2026 · 6

1. TruthInsightBench: An Evidence-Grounded Benchmark for Automated Evaluation of Open-Ended Scientific Discovery Agents Science & research
2. ElderBench: Benchmarking Autonomous Mobile Agents for Older Adults Mobile
3. FinalityBench: An Effect-Level Benchmark for Agent Decisions Under Delayed and Conflicting Financial Finality General
4. ERPBench: Evaluating LLM Agents for Enterprise Decision-Making Across Competitive Market Ecologies Work / office
5. A Cost-Aware Agentic Architecture for NL-to-SQL over Nested Enterprise Schemas, with a New Benchmark General
6. $τ^τ$-Bench: An Environment for End-To-End, Realistic Agent Construction Coding / SWE

September 3, 2026 · 9

1. HarvestBench: Measuring Whether LLM Agents Will Pay to Avoid Killing Animals Safety
2. ICM-Bench: Person-Level Identity Reasoning in Multimodal Agents with Long-Term Memory Multimodal
3. SWE-Gate: Passing Functional Tests Is Not Enough for Software Engineering Agents Coding / SWE
4. PatchBench: Evaluating AI Agents for Vulnerability Patching Coding / SWE
5. KC-Bench: A Dynamic Interactive Benchmark for Evaluating Knowledge Conflicts in LLM Agents Tool use
6. When Users Don't Ask: Benchmarking Context-Driven Memory Retrieval in Conversational Agents Dialogue
7. Do GUI Agents Know When Not to Act? Enabling Conflict-Aware Termination for Multimodal GUI Agents GUI / computer use
8. DuplexSpeechBench-IFEval: Evaluating Implicit Instruction Following in Full-Duplex Voice Agents Safety
9. EVOHARNESSBENCH: Can Your Agents Keep Pace with an Evolving Harness? Tool use

September 2, 2026 · 1

1. CivBench: A Long-Horizon Benchmark for Tool-Mediated Agents in Civilization VI Tool use

September 1, 2026 · 6

1. Efficient SWE Agent Benchmarking via Trajectory-Aware Evaluation Coding / SWE
2. CordisBench: Can Language Models Reason About Component Lifecycles in Dynamic Agent Harnesses? General
3. HarnessDev: Can LLMs Create and Evolve Their Own Agent Harness? Tool use
4. Evaluating Multimodal LLMs as Generalist Vision-Language-Action Agents for Drone Control: Commanding, Approaching, Tracking and Searching Multimodal
5. InSight: A Benchmark for Agentic Claim Verification in Interactive Visualizations Multimodal
6. WorldBench: Culturally Grounded Benchmark for Multilingual Agents General

August 2026

August 31, 2026 · 8

1. RestoreBench: Can AI Agents Restore Power Flow Convergence? Multi-agent
2. Toward Workflow-Aware Benchmarking for Healthcare NLP Agents General
3. MNIST-PRO: MNIST is Back as a Partially Observable World for AI Agents Multimodal
4. E-Commerce Bench: Evaluating LLM Agents on Long-Horizon Autonomous Business Operation General
5. BAITBENCH: Measuring Agent Reward Hacking with Optional Shortcuts Planted in ML Tasks Safety
6. SwarmBench: Can Large Language Models Act as Agent Swarm Orchestrators? Multi-agent
7. Framework and Benchmark for Code-Driven Agentic Testing in Web Development Web & browser
8. EvoSkill Injection: Red-Teaming Autonomous Skill Generation and Evolution in Self-Evolving Agents Safety

August 30, 2026 · 4

1. You Know What I Mean: A Benchmark for Agentic Conversational Reference Grounding Coding / SWE
2. GUI-CC: Benchmarking Contextual Consistency of GUI World Models as Agent Environments GUI / computer use
3. MedCache: Efficient and Temporally Valid Memory for Longitudinal Clinical Agents Multi-agent
4. SIC-Agents: Benchmarking and Building an Adaptive Simulator for Pediatric Serious Illness Communication Training Dialogue

August 29, 2026 · 2

1. AlgoWorlds: Benchmarking Tool Use for Global Optimization in Algorithmic Worlds Tool use
2. APIFlow-Bench: Measuring Whether Agents Survive Long, Dependent API Workflows Tool use

August 28, 2026 · 2

1. CoCoBench: A Cooperative Coordination Benchmark for Embodied Multi-Agent Task Planning Multi-agent
2. Benchmarking large language model agent societies against human behavioural distributions General

August 27, 2026 · 7

1. BTS-AgentBench: A Deterministic, Replayable Pipeline from Read-Only Telemetry Logs to Agent Benchmarks Tool use
2. BALMS: Benchmarking Agentic LLMs for Longitudinal Mental Health Sensing General
3. GRAIN: Bridging Name and Narrative Shifts in Real-World Graph Reasoning through Invariance-Rewarded Agentic RL Multi-agent
4. FaulT-Bench: Towards Benchmarking Network Troubleshooting LLM Agents under Unreliable User Tickets Safety
5. BekchiAI: Measuring, Observing, and Controlling LLM Agents in One Click Tool use
6. AgentJudgeBench: A Multi-Difficulty Benchmark for Evaluating LLM Judges on Agentic Tool-Calling Tool use
7. DuMateBench: Evaluating Autonomous Agents in Complex Real-World Workflows Safety

August 26, 2026 · 3

1. Paint What You See: Benchmarking Dexterous Visual Tool Use in Multimodal Agents Multimodal
2. HRGuard: Gating Relationship Manipulation in Multi-Turn Agentic AI Conversations Safety
3. BixBench3: Benchmarking AI agents on research-study-scale computational biology tasks Science & research

August 25, 2026 · 6

1. ADeptS-Bench: Measuring the Trustworthiness of Computer Use Agents Across Devices GUI / computer use
2. LifePlanner: Evaluating LLM Agents for Geo-spatial Planning with Social Media Data Multimodal
3. Benchmarking AI Agents for Hardware Design Automation via MCP Tool Calling Tool use
4. PeakBench: Benchmarking Resource-Aware Tool Invocation in LLM Agents Tool use
5. Adaptive Influence Graphs for Failure Attribution in Multi-Agent Systems Multi-agent
6. Benchmarking LLM Judges for Voice-Agent Evaluation: Reliability, Calibration, and Human Oversight Safety

August 24, 2026 · 8

1. SWE Refactor Bench: Can Coding Agents Complete a Long-Horizon, Whole-Repository Stack Migration? Coding / SWE
2. EarthVerse: Benchmarking Scientific Agents Across Dynamic Earth Systems and Natural Hazards Science & research
3. Thinking Beyond Videos: Unifying Video Reasoning and Deep Research for Open-World Video Agents Multimodal
4. NetConfArena: An Executable Benchmark for LLM Agents in Closed-Loop Network Configuration Safety
5. Beyond Executable Models: The Pufibara Agent Harness and the Modelica Agent Workflow Benchmark for Physical System Modeling Coding / SWE
6. Signal or Noise? A Benchmark Study of Agent Skills in Web Development Tool use
7. MobilePA-Bench: Benchmarking Mobile Planner Agents on Complex Real-World Tasks Mobile
8. CatchBench: When Can an Agent Failure Be Caught? General

August 23, 2026 · 3

1. ClawProBench: Trace-Aware Evaluation of AI Agents with Runtime Coverage and Frozen Workplace-Style Holdouts Work / office
2. ToolRobustBench: Stage-Wise Perturbation Evaluation and Failure Diagnosis for Tool-Calling Agents Safety
3. Noise Floor Audit for Agent Benchmarks Tool use

August 22, 2026 · 1

1. GameXpert-Bench: How Far Are Coding Agents from Expert Game Development? Coding / SWE

August 21, 2026 · 4

1. K-Bench: measuring model performance on real scientific agent requests Science & research
2. BC-Bench: Evaluating Agentic Engineering in a Domain-Specific Language for ERP Coding / SWE
3. Structural Inference in Undocumented Mobile Databases: A Reproducible Benchmark for Evaluating Agentic Reasoning in Digital Forensics Mobile
4. DreamBench-SWE: A Multi-Session Memory-Hygiene Benchmark for Software Agents Coding / SWE

August 20, 2026 · 5

1. AI4AI-Bench: Benchmarking LLM Agents in Algorithmic Design for Recursive Self-Improvement RSI / AI4AI
2. MaliciousSkillBench: A Comprehensive Benchmark for Malicious Agent Skill Detection Safety
3. SWE-bench Science: Can Coding Agents Resolve Engineering Tasks in Science? Coding / SWE
4. One Success Isn't Reliability: Thinkingbox, a Sandbox and Benchmark for Agents in Stateful Business Workflows General
5. DeltaML-Bench: Evaluating Machine Learning Agents on Real-World Research Repositories General

August 19, 2026 · 1

1. FM-Bench: A Benchmark for Long-Horizon Management with Competing Agents General

August 18, 2026 · 6

1. ComponentBench: Diagnosing Component-Level Failures in Computer-Use Agents GUI / computer use
2. VisDocAgentBench: Benchmarking Agents for Visually Rich Document Retrieval General
3. StartupBench: Benchmarking General-Purpose Agents on Market-Validated End-to-End Workflows Work / office
4. MobileWorldSafety: Benchmarking GUI Agent Safety Against Environmental Injection Attacks in Android Apps Mobile
5. HarnessRisk: A Lifecycle-Oriented Benchmark for Agent Harness Safety Safety
6. Evaluating RL Explainability Methods by How Much They Help Fix Bugs in Agents Coding / SWE

August 17, 2026 · 5

1. PDDLCoder: Agentic PDDL Generation for LLM-Assisted Symbolic Planning General
2. AeroCopilotBench: A Two-Tier Benchmark for Evaluating LLM Agents as Aviation Copilots in an Interactive Virtual Cockpit Environment General
3. CompoSkill: Compositional Skill Chain Attacks from Individually Scanner-Passing LLM Agent Skills Work / office
4. Governance at the Boundary: How Agent Decomposition Degrades Policy Compliance General
5. OpenHarmony Bench: Evaluating LLMs and Coding Agents on OpenHarmony App Development Coding / SWE

August 16, 2026 · 1

1. When Stories Evolve: Benchmarking LLM Storytelling Across Agent Architectures in Open-Ended World Simulations General

August 15, 2026 · 2

1. LongRCA Bench: Diagnosing Responsible Roles and Root Causes in Long-Horizon Agent Failures General
2. From LLM Inference to Agentic Workloads: Characterization and Implications for Serving Systems General

August 14, 2026 · 4

1. AgentRewind: Recoverable Execution for Long-Horizon LLM Agents General
2. TimeSage-EV: A Live Benchmark for Agentic Time Series Analysis in Evolving Environments General
3. Act2Intention: A Benchmark For Developing Active Mobile Agents Through Inferring User Intention from GUI Actions Mobile
4. MemoryLake on MemoryArena: A Matched Study of Agent Memory Backends General

August 13, 2026 · 1

1. PlayWorld: Benchmarking World Models with Agent Players over Long-Horizon Objectives General

August 12, 2026 · 10

1. SteerBench-Work: A Benchmark for Agent Steering at Action Boundaries Work / office
2. DiG-bench: Discovery in Games General
3. Do LLMs Beat Nash? Testing Decentralized Coordination in Self-Play Multi-Agent Games Multi-agent
4. CTBench: Evaluating Troubleshooting Capabilities of AI Agents in Realistic Telecom Network Operations General
5. Total Recall at What Cost? Benchmarking the Serving Cost of Agentic Memory Systems General
6. Advancing MLLM-based UAV Image Understanding and Reasoning: A Benchmark and a Training-Free Multi-Agent System Multi-agent
7. FrontierFinance: A Challenging Benchmark for Measuring Frontier Intelligence of Finance Agents General
8. AgenticTwin: An Agentic LLM Framework Integrated with Digital Twin for Anomaly Detection General
9. MBA: Multimodal Benchmark and Agents for Real-World Business Ideation Work / office
10. VideoGAIA: A Benchmark for General AI Assistants on Agentic Video Understanding Multimodal

August 11, 2026 · 4

1. Benchmarking LLM Judges for Mobile Agent Evaluation Mobile
2. VibeLifeBench: Can Your Life Agent Be Proactive and Persistent in a Living World? General
3. REDAgentBench: Executable Red Teaming and Faithful Measurement of LLM Agent Systems General
4. DSAgentBench: Can Agents Automate End-to-End Data-Science Workflows in Real Computer Environments? General

August 10, 2026 · 6

1. SWE-Bench ProMax: Benchmarking Agents on Large-Scale Multilingual Code Refactoring Coding / SWE
2. ActBench: Self-Evolving Benchmark of Behavioral Safety in Cowork Agents Safety
3. ComboShoppingBench: Evaluating LLM Agents for Budget-Constrained Basket Shopping with Coupons General
4. Social Gym and SPaRTan: Benchmarking and Improving LLM Social Reasoning via Multi-Agent Game Tournaments RSI / AI4AI
5. Evo-Bench: Can Language Models Improve Agent Harness? Work / office
6. A Unified Issue Resolution Benchmark for Requirement Clarification, Planning, and Code Generation for Coding Agents Coding / SWE

August 9, 2026 · 6

1. Findings of the First Teaching Monster Challenge: A Benchmark of Pedagogical Content Knowledge in AI Agents Multimodal
2. 360CityArena: A Realistic Virtual Urban Navigation Benchmark for Embodied Agents Embodied / robotics
3. PluginEval: A Diagnostic Benchmark for Fine-Grained Error Attribution in Function Calling Tool use
4. Business Arena: Benchmarking LLM Agents in a Realistic Marketplace General
5. ForestBench: A Unified Graph Framework for Evaluating Multi-Agent Collaboration Multi-agent
6. CAP: A Scalable Benchmark for Evaluating Cross-Site Browser Agents with Complex Actions and Perception Web & browser

August 8, 2026 · 5

1. WirelessOpsAgent: A Benchmark and Agent Design for Action Assurance in Wireless Networks General
2. Can LLM Agents Stick to the Script? A Benchmark for Long-Horizon Consistency in Interactive Narratives General
3. Lingjing: A Simulation Testbed for Multi-Agent Embodied Tasks in Open-Ended Cities Multi-agent
4. Compiling and Benchmarking Task-State Horizons for Embodied Agents Embodied / robotics
5. TelemetrySuffBench: Is Agent Telemetry Sufficient for Failure-Origin Diagnosis? General

August 7, 2026 · 2

1. HarnessSafe: Evaluating Safety Across Persistent Carriers in Agent Harnesses Safety
2. Long-Horizon Agent Trajectory Attribution: A Unified Benchmark and Fine-Grained Annotation Framework General

August 6, 2026 · 8

1. Do AI Personas Grow? Analyzing and Benchmarking Personality Evolution in LLM Agents After Life Events Dialogue
2. StepJack: Benchmarking Computer-Use Agent Safety Against Multi-Step Indirect Prompt Injection GUI / computer use
3. TRAJDEBUG: Tracing Error Lifecycle to Identify Critical Failures in Long-Horizon Agent Trajectories Coding / SWE
4. Benchmarking the Benchmarks: Evaluating Benchmarks for Conversational Agents General
5. FinEvo-Bench: A Longitudinal Benchmark for Self-Evolving Agents in Professional Financial Workflows Work / office
6. FormBharo: Designing and Evaluating a Voice Agent for Conversational Form Filling in Rural India General
7. SkillTV-Bench: Benchmarking How Well Judges Perform on Skill-Augmented Agentic Execution General
8. EcoAgent-Bench: Evaluating Economic Decision-Making in Budget-Constrained LLM Agents General

August 5, 2026 · 5

1. OrchestraBench: Evaluating Multi-Agent Orchestration Failure Modes, Recovery, and Decomposition Quality Work / office
2. Skill-Use: Can LLMs Actually Use Skills in Agentic Harnesses? General
3. Caching for the Future: Scrub Jay Episodic Memory Principles for Agent Memory Systems General
4. Active-SWE: Benchmarking Coding Agents for Proactive Bug Fixing without Issue Reports Coding / SWE
5. SearchAuditor: Auditing and Attributing Failures in Long-Horizon Search Agents General

August 4, 2026 · 12

1. When Agentic Executions Fail: Detecting and Localizing Runtime Faults from Telemetry General
2. FinPerMA: A Theory-Informed, Event-Grounded Personalized-Memory Benchmark for LLM Agents General
3. FinProBench: Evaluating Financial AI Agents with Role-Grounded Rubrics Derived from Professional Deliverables General
4. PAST-Bench: Benchmarking the Foundations of Recursive Self-Improvement in Personal Agents RSI / AI4AI
5. MultiGlobeQA: A Multilingual and Globally Diverse Benchmark for Geospatial Reasoning General
6. When Agents Learn to Be You: Benchmarking Privacy Leakage, Impersonation Risk, and Defenses in Persona Skills Dialogue
7. TARL: Transaction-Aware Reliable Ledgers for Executable Memory Management in Long-Term Agents General
8. DiagChain: A Diagnostic Benchmark for Evaluating LLM Agents on Evidence-Grounded Attack Chain Reconstruction General
9. MT-Web2Code: Benchmarking Coding Agents on Multi-Turn Regional Reconstruction and Localized Modification Coding / SWE
10. DataSpace: Benchmarking Data Agents for Verifiable Analytics over Heterogeneous Workspaces Multimodal
11. EduClaw-Bench: A Long-Horizon Benchmark for Pedagogical LLM Agents with Simulated Learners General
12. UrbanAgent: A Tool-Augmented Agent for Cross-System Urban Tasks General

August 3, 2026 · 8

1. Everyone Conforms, No One Believes: Pluralistic Ignorance in LLM Agent Populations Work / office
2. SWE-Touch: Benchmarking Coding Agents When Users Touch the Code Coding / SWE
3. ScrambleToolBench: Agents Search Exhaustively Even When Their Own Map Points to the Next Step General
4. From Profiling to Synthesis: Benchmarking Implicit Behavioral Alignment in Personalized LLM Agents General
5. VulnGym: Benchmarking Coding Agents for Repository-Level Vulnerability Detection Coding / SWE
6. CockpitHAT: Dependency-Graph-Driven Hierarchical Attribution for Embodied Multi-Agent Cockpits Multi-agent
7. GABench: A Comprehensive Benchmark for Evaluating LLM Agents on Graph Analysis Tasks General
8. GISAgentBench: A Practitioner-Sourced Benchmark for Evaluating LLM Agents on GIS Tasks Multimodal

August 2, 2026 · 4

1. HopRefusalBench: Diagnosing Refusal Failures in Search-Augmented Agents for Multi-Hop Reasoning General
2. PATH-Bench: Path-Dependent Evaluation of Lifelong Agents Coding / SWE
3. FraudBench: Stress-Testing Policy-Grounded Banking Agents Against Adaptive Fraud Dialogue
4. MasDrift: Benchmarking Authorization Preservation Across Multi-Agent Architectures Multi-agent

August 1, 2026 · 4

1. AdvPlan-Bench: Adversarial Evaluation of Structured Plan-Generation Agents Multi-agent
2. AgentSLABench: Evaluating and Benchmarking Agentic Systems Under Resource Constraints Coding / SWE
3. FinDeepIndicator: Benchmarking Deep Research Agents in End-to-End Financial Indicator Construction Science & research
4. OpenART: Scaling Agent Red Teaming via Open-Ended Environment Evolution General

July 2026

July 31, 2026 · 9

1. LoopsBench: From Harness Engineering to Loop Engineering in Coding Agent Evaluation Coding / SWE
2. ExtractBench: A Benchmark for Schema-Guided Enterprise Document Extraction Work / office
3. AgentHPOBench: A Benchmark For Evaluating LLM Agents as Sequential Hyperparameter Optimizers Coding / SWE
4. MDArena: Evaluating Coding Agents on Realistic Molecular Dynamics Workflows Coding / SWE
5. MetaRoute-Bench: Evaluating Meta-Decision Policies for Agentic Workflow Routing General
6. MMShopBench: A Real-Log Benchmark for Multimodal, Multi-Turn Shopping Agents Multimodal
7. InfraBench: Evaluating Infrastructure Agents Across Layers, Lifecycle, and Risk General
8. Backtrader-Bench: Benchmarking LLM Agents on Algorithmic Trading with Self-Generated MCQs Coding / SWE
9. MerchantBench: Benchmarking LLM Agents for Long-Term Coherence in E-Commerce Operations General

July 30, 2026 · 7

1. Can LLM Agents Price Competitively? A Dynamic Multi-Attribute Auction Benchmark for Agentic Commerce General
2. PAIChecker: Uncovering and Checking PR-Issue Misalignment in SWE-Bench-Like Benchmarks Coding / SWE
3. ORCA-bench: How Ready Are Language Model Agents for Oncall? Coding / SWE
4. Can Agents Deceive? Evaluating Reasoning and Deception in ParliamentBench using a Social Deduction Game General
5. Echoverse: Deep, Evolving Environments for Training Computer-Use Agents at Scale GUI / computer use
6. DataClawEval: A Benchmark for Data Engineering Agents in Real Industrial Harness General
7. SciToolAgent-Evo: An Ontology-Aware Self-Evolving Agent for Open-World Scientific Tool Acquisition Science & research

July 29, 2026 · 11

1. VAmoS Bench: Voice Agent Simulation Bench General
2. SWE-NFI: Studying and Benchmarking Coding Agents for Non-Functional Improvements Coding / SWE
3. LayerRAG-Bench: A Cross-Layer Reliability Benchmark for Agentic Retrieval-Augmented Generation General
4. OmegaUse-OfficeVal: Benchmarking LLM Agents on Long-Horizon Office-Suite Tasks with Economic Grounding Work / office
5. MemSecBench: Tracking Agent Memory Poisoning from Persistence to Consequence and Repair Work / office
6. AgentS4D: Benchmarking Runtime Risks across the Execution Lifecycle of LLM-Based Workspace Agents General
7. Setoka: A Benchmark for Hierarchical User Understanding in Personalized Agents over Heterogeneous Data General
8. TREK: A Travel Reasoning and Evaluation Kit for LLM Agents in Complex Trip Planning General
9. SecRespond: Benchmarking AI Agents for Real-World Post-Compromise Incident Response General
10. ExplainBench: Evaluating Code Explanations from Agents Coding / SWE
11. Exploring Structures in Physics Problems: Can AI Agents Discover Statistical Mechanical Mappings? General

July 28, 2026 · 8

1. StealthBench: Measuring Operational Stealth in Autonomous Offensive-Security Agents General
2. ClinLens: Towards Long-Horizon Coding Agents for Longitudinal Multimodal Clinical Data Science Coding / SWE
3. Desktop-Delta Bench: Do Computer-Use Models Understand Desktop GUI Transitions? GUI / computer use
4. Interactive Reward Agent: GUI Task Evaluation via Environment-State Verification GUI / computer use
5. WorkSurface-Bench: Benchmarking Enterprise Agents on Multi-Surface Knowledge Routing Work / office
6. OrchBench: Evaluating Multi-Agent Orchestration Plans in Isolation via Deterministic Simulation Multi-agent
7. PatientAgentBench: A Benchmark Framework for Evaluating Patient-Facing Health AI Agents General
8. HANDBOOK.md: A Benchmark for Long-Context Agentic Instruction Following General

July 27, 2026 · 4

1. GAUGE: Grading Agent-Built Financial Models Without a Golden Answer General
2. Keep It InMind: Benchmarking the Implicit-Association Blind Spot in Agent Memory General
3. Agent Retrieval Bench: Evaluating Repository Context Retrieval for Coding Agents Coding / SWE
4. ContainmentBench: Trace-Based Evaluation of Post-Exposure Containment in Tool-Using LLM Agents General

July 26, 2026 · 2

1. MulRobBench: A Decision-Level Benchmark for Safe and Security-Policy-Compliant Multimodal UAV Agents Embodied / robotics
2. E-Bench: Benchmarking Multi-Step Tool-Use Agents in Real-World Product Scenarios Tool use

July 25, 2026 · 2

1. AllocBench: Measuring Online Tool Allocation Capability in LLM Agents General
2. SQBench: A Benchmark for Evaluating Task Delivery by Language-Model Agents in Production-Oriented Workflows General

July 24, 2026 · 4

1. SceneActBench: Can Agents Act on the 3D Scenes They See? Multimodal
2. Do Agent Benchmarks Measure Capability? Protocol Validity in the Age of Agentic AI Coding / SWE
3. DBA-Bench: A Production-Fidelity Benchmark for LLM-Based Database Operations Agents General
4. Zero-Shot Mission-Level Evaluation for Aerial MLLM Agents Embodied / robotics

July 23, 2026 · 6

1. Protocol-Level Attacks on Agentic Commerce Platforms: A Cross-Platform Taxonomy, AIP-Bench, and Unified Defense General
2. Agentic Evaluation of Copyright Law Compliance General
3. Benchmarking Agents for Proving Theorems in Quantum Algorithms and Quantum Information General
4. GuardianAgentBench: Where Agents Fail and How to Guard Them General
5. SciExplore: Evaluating Autonomous Agents from Scientific Navigation to Information Integration Science & research
6. Tencent WorkBuddy Bench: A Multi-Domain Coding-Agent Benchmark with Contamination-Resistant Task Construction Coding / SWE

July 22, 2026 · 4

1. IssueTrojanBench: Benchmarking AI Coding Agents Against Malicious Issue Requests Coding / SWE
2. OpenSkillRisk: Benchmarking Agent Safety When Using Real-World Risky Third-Party Skills Safety
3. DocOps: A Verifiable Benchmark for Autonomous Agents in Complex Document Operations General
4. Same Game, Different Story: A Minimal Conservative Strategic Robustness Benchmark for Large Language Model Agents General

July 21, 2026 · 2

1. FinanceComplexQA: Benchmarking Agentic Reasoning on Industrial-grade Financial Documents General
2. SciHazard: A Benchmark for Measuring Scientific Safety Risks with Decomposed Harm Scoring Safety

July 20, 2026 · 6

1. Trusted Credentials, Untrusted Behavior: Benchmarking LLM-Agent Security in High-Performance Computing General
2. WorldCupArena: Fine-Grained Evaluation of Language Models and Deep-Research Agents on Football Forecasting Science & research
3. Adaptive Adversaries: A Multi-Turn, Multi-LLM Benchmark for LLM Agent Security General
4. FIFA World Cup 2026 as a Contamination-Free Benchmark for LLM Forecasting Agents: Four Models, a Bookmaker, and 104 Matches General
5. ProEvent: An Event-centric Benchmark for Proactive Agents General
6. Can AI Agents Really Complete RTL-to-GDS? Lessons from Benchmarking Tool-Interactive EDA Workflows Coding / SWE

July 19, 2026 · 1

1. DRNOISE: Benchmarking Deep Research Agents in Misleading Evidence Environments Science & research

July 18, 2026 · 4

1. Beyond Memory Leaderboards: Evaluating Scientific Memory as Budgeted Context Restoration Science & research
2. RECON: Benchmarking Agent Memory for Compositional Reasoning over Long Contexts General
3. CLOSER-Bench: Evaluating Budgeted Cross-Stage Design Closure for Hardware Agents Coding / SWE
4. Just A Rather Very Intelligent Spoken Agent General

July 17, 2026 · 2

1. Frontier AI performance across the business disciplines: a case-grounded benchmark of knowledge work and analytical reasoning Work / office
2. Beyond Sequential Interaction: Benchmarking Parallel Execution and Coordination for GUI Agents GUI / computer use

July 16, 2026 · 7

1. Coercion and Deception in AI-to-AI Management: An Agentic Benchmark of Unprompted Escalation Multi-agent
2. OmniaBench: Benchmarking General AI Agents Across Diverse Scenarios General
3. StructureClaw: Traceable LLM Agents and an Executable Benchmark for Structural Engineering Workflows Multimodal
4. MCPEvol-Bench: Benchmarking LLM Agent Performance Across Dynamic Evolutions of MCP Servers Tool use
5. Alipay-PIBench: A Realistic Payment Integration Benchmark for Coding Agents Coding / SWE
6. SafeRelBench: A Spatial-Relation-Aware Benchmark for Process-Level Safety in VLM-Driven Embodied Agents Safety
7. Are LLM-Generated GPU Kernels Production-Ready? A Trace-Driven Benchmark and Optimization Agent Coding / SWE

July 15, 2026 · 2

1. Do Agent Optimizers Compound? A Continual-Learning Evaluation on Terminal-Bench 2.0 General
2. DevicesWorld: Benchmarking Cross-Device Agents in Heterogeneous Environments GUI / computer use

July 14, 2026 · 4

1. Seeing What Is Actually There: PriVE-Bench and PriVE-Tools for Counterfactual Evaluation of Agentic Visual Evidence in VLMs Multimodal
2. PM-Bench: Evaluating Prospective Memory in LLM Agents General
3. How Many Tasks Are Enough for Agent Benchmark Decisions? A Replay Analysis of Public LLM Agent Benchmarks Coding / SWE
4. LakeQuest: A Three-Domain Benchmark for Grounded Question Answering across Data Lakes Multimodal

July 13, 2026 · 4

1. Operationalising Multi-Dimensional Evaluation for Conversational Agents: A Scalable, Governed Pipeline with Selective Re-evaluation and Model Benchmarking General
2. MM-ToolSandBox: A Unified Framework for Evaluating Visual Tool-Calling Agents General
3. IFCMemoryBench: Evaluating Long-Term Memory of LLM-Based Agents in BIM Information Retrieval General
4. BackendForge: Benchmarking Agentic End-to-End Code Generation with Backend Services Coding / SWE

July 12, 2026 · 4

1. Imaging-101: Benchmarking LLM Coding Agents on Scientific Computational Imaging Coding / SWE
2. Opti-Agent-Bench: Benchmarking End-to-End Optimization R&D Agents on Real-World Business Problems Work / office
3. UNIBROWSE: A Data-to-Agent Framework for Multimodal BrowseComp Multimodal
4. Agents Don't Just Agree, They Remember: Benchmarking Persistent Sycophancy in Stateful Personal Agents General

July 11, 2026 · 3

1. NetInjectBench: Benchmarking Indirect Prompt Injection in Tool-Using Large Language Model Agents for Network Operations General
2. MAG: A Web-Agent Benchmark and Harness for Multimodal Action and Guide Generation Multimodal
3. FORCE-Bench: A Benchmark, Dataset, and Evaluation Harness for Agentic AI in Enterprise Finance General

July 10, 2026 · 4

1. AppWorld-UL: Benchmarking Diverse Agent-User Interactions for Tool-Use Tool use
2. DynamicMCPBench: A Trace-Grounded, Effect-Scored Benchmark for LLM Agents over Live MCP Servers Tool use
3. LongMedBench: Benchmarking Medical Agents for Long-Horizon Clinical Decision-Making General
4. SLBench: Evaluating How LLM Agents Follow Logical Relations in Skills General

July 9, 2026 · 2

1. Long-Horizon-Terminal-Bench: Testing the Limits of Agents on Long-Horizon Terminal Tasks with Dense Reward-Based Grading Coding / SWE
2. CausalDS: Benchmarking Causal Reasoning in Data-Science Agents General

July 8, 2026 · 4

1. DeepSWE: Measuring Frontier Coding Agents on Original, Long-Horizon Engineering Tasks Coding / SWE
2. Does AI Understand Imaging? A Systematic Benchmark of Agentic AI for Computational Imaging Tasks Multimodal
3. PERFOPT-Bench: Evaluating Coding Agents on Software Performance Optimization Coding / SWE
4. CANN Bench: Benchmarking Agent Generated Kernels against Real NPU and Algorithmic Limits Coding / SWE

July 7, 2026 · 6

1. RuBench: A Repository-Level Agentic Coding Benchmark with Natively Authored Russian Task Specifications Coding / SWE
2. AgentLens: Production-Assessed Trajectory Reviews for Coding Agent Evaluation Coding / SWE
3. WebRetriever: A Large-Scale Comprehensive Benchmark for Efficient Web Agent Evaluation Web & browser
4. PolyWorkBench: Benchmarking LLM Agents for Cross-Lingual Long-Horizon Workflows Work / office
5. PCBWorld: A Benchmark Environment for Engine-Grounded PCB Design Automation General
6. Beyond the Leaderboard: A Synthesis of Tool-Use, Planning, and Reasoning Failures in Large Language Model Agents Tool use

July 6, 2026 · 8

1. SovereignPA-Bench: Evaluating User-Owned Personal Agents under Evolving Intent, Platform Mediation, and Consent Constraints GUI / computer use
2. PiSAs: Benchmarking Contextual Integrity in Multi-User Agentic Systems General
3. BatteryLake: Agentic, Physics-Grounded Curation of Heterogeneous Battery Aging Data and Benchmarking General
4. EvoAgentBench: Benchmarking Agent Self-Evolution via Ability Transfer Work / office
5. AgentGym2: Benchmarking Large Language Model Agents in De-Idealized Real-World Environments General
6. RustMizan: A Compilable, Contamination-Aware Benchmarking Framework for Rust Vulnerabilities General
7. ToolFailBench: Diagnosing Tool-Use Failures in LLM Agents Tool use
8. BioSecBench-Refusal: A paired metric for performance and alignment in agentic biosecurity risk assessment General

July 5, 2026 · 2

1. HAS-Bench: Evaluating LLM-Based Human-Agent Systems under Configurable Human Participation General
2. CausalGame: Benchmarking Causal Thinking of LLM Agents in Games General

July 4, 2026 · 2

1. ProACT: Towards Breakdown-Aware Proactive Agent in Multi-User Collaboration General
2. Telco-GAIA: Bilingual Benchmark for Agents in Telecom Domain Work / office

July 3, 2026 · 5

1. ArchEval: Measuring AI Agents as Computer Architects General
2. GameEngineBench: Evaluating Coding Agents on Real C++ Runtime Environments Coding / SWE
3. APeB: Benchmarking Personalization Ability of Large Language Model Agents General
4. ORBIT-Q: Dual-axis benchmarking of autonomous agents in scientific quantum programming Science & research
5. HoosierHelp: Benchmarking LLM Agents for Social Service Navigation General

July 2, 2026 · 8

1. SovereignNegotiation-Bench: Evaluating User-Owned Personal Agents In Delegated Bargaining Under Privacy, Consent, Evidence, And Institutional Pressure General
2. S-EMBER: A Large-Scale Benchmark for Streaming Egocentric Memory Retrieval General
3. Can Coding Agents Implement Missed Compiler Optimizations? Evaluating LLM Agents on LLVM Peephole Optimizations Coding / SWE
4. Understanding Agent-Based Patching of Compiler Missed Optimizations Coding / SWE
5. AgenticSTS: A Bounded-Memory Testbed for Long-Horizon LLM Agents General
6. A$^{2}$utoLPBench: An Auto-Generated, Agent-Friendly LP Benchmark via Inverse-KKT Construction General
7. SimWorlds: A Multi-Agent System for Dynamic 3D Scene Creation Multi-agent
8. AgenticDataBench: A Comprehensive Benchmark for Data Agents Work / office

July 1, 2026 · 6

1. Navigation Alone Is Not Enough: Evaluating Explanatory Assistive UI Agents General
2. MemSyco-Bench: Benchmarking Sycophancy in Agent Memory General
3. ChainSWE: Benchmarking Coding Agents on Multi-Bug Software Maintenance Coding / SWE
4. AGI Maze as a Benchmark Framework for World-Modeling Agents General
5. EgoGapBench: Benchmarking Egocentric Action Selection in Multi-Agent Scenes Multi-agent
6. PHREEQC-MCQ-200: A Diagnostic Benchmark for Tool-Augmented Scientific Simulator Agents Science & research

June 2026

June 30, 2026 · 6

1. AxDafny: Agentic Verified Code Generation in Dafny Coding / SWE
2. MECoBench: A Systematic Study of Multimodal Agent Collaboration in Embodied Environments Embodied / robotics
3. FinPersona-Bench: A Benchmark for Longitudinal Psychometric Stability of Autonomous Financial Agents General
4. HealthAgentBench: A Unified Benchmark Suite of Realistic Agentic Healthcare Environments for Challenging Frontier AI Agents General
5. ClawArena-Team: Benchmarking Subagent Orchestration and Dynamic Workflows in Language-Model Agents Multi-agent
6. PPT-Eval: A Benchmark for Computer-Use Agents on PowerPoint Tasks GUI / computer use

June 29, 2026 · 5

1. Behavioral Governance for Autonomous AI Agents: The AgentBound Framework Work / office
2. SWE-INTERACT: Reimagining SWE Benchmarks as User-Driven Long-Horizon Coding Sessions Coding / SWE
3. SpreadsheetBench 2: Evaluating Agents on End-to-End Business Spreadsheet Workflows General
4. EditPPT: Faithful Long-Deck Slide Editing via Structured Tool-Using Multi-Agent with Dual-Modal Validators Multi-agent
5. CLQT: A Closed-Loop, Cost-Aware, Strategy-Consistent Benchmark for Diagnostic Evaluation of LLM Portfolio-Management Agents General

June 28, 2026 · 3

1. OSWorld 2.0: Benchmarking Computer Use Agents on Long-Horizon Real-World Tasks Work / office
2. Bridging VideoQA and Video-Guided Agentic Tasks via Generalized Keyframe Extraction GUI / computer use
3. A Multi-Dataset Benchmark for Evaluating LLM Agents in Microservice Failure Diagnosis Web & browser

June 27, 2026 · 2

1. From Tool Connection to Execution Control: Benchmarking Security Invariants in MCP-Style Agent Runtimes Tool use
2. A Physics-Grounded Benchmark for Multi-Agent Dynamics in World Models Multi-agent

June 26, 2026 · 5

1. GPTNT: Benchmarking Real-Time Collaboration Between Multimodal Agents on Keep Talking And Nobody Explodes Multimodal
2. TUA-Bench: A Benchmark for General-Purpose Terminal-Use Agents GUI / computer use
3. ToolPrivacyBench: Benchmarking Purpose-Bound Privacy in Tool-Using LLM Agents General
4. NormAct: Benchmarking Embodied Agents' Proactive Compliance with Unspoken Social Norms Embodied / robotics
5. When Search Agents Should Ask: DiscoBench for Clarification-Aware Deep Search General

June 25, 2026 · 2

1. Ko-WideSearch: A Korean Breadth-Search Benchmark for Exhaustive Set Enumeration by Web Agents Web & browser
2. DMV-Bench: Diagnosing Long-Horizon Multimodal Agents' Visual Memory with Incidental Cue Injection Multimodal

June 24, 2026 · 5

1. CyberChainBench: Can AI Agents Secure Smart Contracts Against Real-World On-Chain Vulnerabilities? General
2. Beyond Function Calling: Benchmarking Tool-Using Agents under Tool-Environment Unreliability Tool use
3. Uncertainty Quantification for Computer-Use Agents: A Benchmark across Vision-Language Models and GUI Grounding Datasets GUI / computer use
4. SidConArena: An Environment Evaluating Agents in Open-Ended,Positive-Sum Bargaining Game General
5. FinBench: Time-Gated Calibration and Uncertainty Benchmarking for Agentic Financial Forecasting General

June 23, 2026 · 5

1. Buildrix: An Open Platform for Sharing and Benchmarking Agentic AI Skills in Building Engineering General
2. EvoClawBench: Can Agents Learn Reusable Skills from Their Own Runs? General
3. NatureBench: Can Coding Agents Match the Published SOTA of Nature-Family Papers? Coding / SWE
4. AGORA: An Archive-Grounded Benchmark for Agentic Workplace Document Reasoning Work / office
5. Can Language Model Agents be Helpful Circuit Explainers in Mechanistic Interpretability? General

June 22, 2026 · 6

1. RIFT-Bench: Dynamic Red-teaming For Agentic AI Systems General
2. MAS-PromptBench: When Does Prompt Optimization Improve Multi-Agent LLM Systems? Multi-agent
3. EnterpriseClawBench: Benchmarking Agents from Real Workplace Sessions Work / office
4. EHR-Complex: Benchmarking Medical Agents for Complex Clinical Reasoning General
5. Managing Procedural Memory in LLM Agents: Control, Adaptation, and Evaluation Work / office
6. IPO Finance Agent: Benchmark of LLM Financial Analysts Beyond Finance Agent v2, with Automated Rubric Generation, on the SpaceX (SPCX) IPO General

June 21, 2026 · 4

1. RigorBench: Benchmarking Engineering Process Discipline in Autonomous AI Coding Agents Coding / SWE
2. MacAgentBench: Benchmarking AI Agents on Real-World macOS Desktop GUI / computer use
3. Governance Decay: How Context Compaction Silently Erases Safety Constraints in Long-Horizon LLM Agents Safety
4. PlanBench-XL: Evaluating Long-Horizon Planning of LLM Tool-Use Agents in Large-Scale Tool Ecosystems Tool use

June 20, 2026 · 1

1. CFAgentBench: A Reproducible Environment and Benchmark for Autonomous Construction-Finance Agents Web & browser

June 19, 2026 · 3

1. Benchmarking LLM Serving Systems for Agentic AI Workloads with XPerf General
2. Trip+: Benchmarking Agents in Personalized Interactive Travel Planning General
3. Matching Matters: A Fair Quality-Efficiency Benchmark for Command-Line Agents General

June 18, 2026 · 7

1. Power Systems Agent Benchmark: Executable Evaluation of AI Agents in Electric Power Engineering General
2. NRT-Bench: Benchmarking Multi-Turn Red-Teaming of LLM Operator Agents in Safety-Critical Control Rooms Safety
3. PowerAgentBench-Dyn: A Benchmark for Agentic AI in Power System Dynamic Studies Embodied / robotics
4. ScholarQuest: A Taxonomy-Guided Benchmark for Agentic Academic Paper Search in Open Literature Environments General
5. ORAgentBench: Can LLM Agents Solve Challenging Operations Research Tasks End to End? General
6. Benchmarking Agentic Review Systems General
7. Beyond Static Leaderboards: Predictive Validity for the Evaluation of LLM Agents General

June 17, 2026 · 9

1. StaminaBench: Stress-Testing Coding Agents over 100 Interaction Turns Coding / SWE
2. IHBench: Evaluating Post-Interruption Recovery in Voice Agents with Structured Workflows Dialogue
3. MortarBench: Evaluating Mortgage Loan Origination Agents General
4. TxBench-PP: Analyzing AI Agent Performance on Small-Molecule Preclinical Pharmacology General
5. RTSGameBench: An RTS Benchmark for Strategic Reasoning by Vision-Language Models Multimodal
6. WorldLines: Benchmarking and Modeling Long-Horizon Stateful Embodied Agents Embodied / robotics
7. GateMem: Benchmarking Memory Governance in Multi-Principal Shared-Memory Agents General
8. PowerAgentBench-SS: A Benchmark for Agentic AI in Power System Steady-State Studies General
9. Deep Research in Physical Sciences: A Multi-Agent Framework and Comprehensive Benchmark Multi-agent

June 16, 2026 · 10

1. CEO-Bench: Can Agents Play the Long Game? Coding / SWE
2. SafeClawBench: Separating Semantic, Audit-Evidence, and Sandbox Harm in Tool-Using LLM Agents General
3. PseudoBench: Measuring How Agentic Auto-Research Fuels Pseudoscience General
4. Compositional Skill Routing for LLM Agents: Decompose, Retrieve, and Compose General
5. GameCraft-Bench: Can Agents Build Playable Games End-to-End in a Real Game Engine? Coding / SWE
6. EComAgentBench: Benchmarking Shopping Agents on Long-Horizon Tasks with Distributed Hidden Intent General
7. AgentMemBench: A Systematic Benchmark for Evaluating Long-Term Memory Management Strategies in Conversational AI Agents Dialogue
8. AISE-Bench: A Full-Cycle Curated Benchmark for Information Seeking on Academic Knowledge Graphs General
9. Can LLMs Be CEOs? Benchmarking Strategic Resource Reallocation with Multi-Role Agent Simulation Multi-agent
10. MapSatisfyBench: Benchmarking Satisfaction-Aware Map Agents through Behavior-Grounded Implicit Decision Factors Work / office

June 15, 2026 · 10

1. PDAGENT-BENCH: Characterizing, Grounding, and Architecting LLM/VLM Agents for VLSI Physical Design Multimodal
2. GeoDisaster: Benchmarking Orchestrated Agents for Operational Disaster Geo-Intelligence Multi-agent
3. MetaSyn: A Benchmark for LLM Agents on Meta-Analysis Articles from Nature Portfolio General
4. LabOSBench: Benchmarking Computer Use Agents for Scientific Instrument Control GUI / computer use
5. MyPCBench: A Benchmark for Personally Intelligent Computer-Use Agents GUI / computer use
6. AgentFairBench: Do LLM Agents Discriminate When They Act? Multi-agent
7. FraudSMSWalker: Benchmarking Agentic Large Language Models for SMS-to-Webpage Fraud Detection Web & browser
8. CoffeeBench: Benchmarking Long-Horizon LLM Agents in Heterogeneous Multi-Agent Economies Multi-agent
9. MIRAGE: Auditing Anti-Muslim Bias in Frontier LLMs Across Reasoning, Agentic, and Time-Coupled Conditions General
10. Embedded Arena: Iterative Optimization via Hardware Feedback General

June 14, 2026 · 3

1. Tokengeist: Multi-Turn Attribution Tracing in Agentic Conversations General
2. SkillVetBench: LLM-as-Judge for Multi-Dimensional Security Risk Evaluation in Open-Source LLM Agent Skills Multi-agent
3. RetailBench: Benchmarking long horizon reasoning and coherent decision making of LLM agents in realistic retail environments General

June 13, 2026 · 7

1. ToolMenuBench: Benchmarking Tool-Menu Filtering Strategies for Reliable and Efficient LLM Agents General
2. Beyond Monolingual Deep Research: Evaluating Agents and Retrievers with Cross-Lingual BrowseComp-Plus General
3. LatentGym: A Testbed For Cross-Task Experiential Learning With Controllable Latent Structure General
4. CODA-BENCH: Can Code Agents Handle Data-Intensive Tasks? General
5. Can Agents Read the Room? Benchmarking Visual Social Intelligence in Multimodal Simulation Multimodal
6. Towards Verifiable Agentic Data Science: Solving Irregular TSQA Via Tool-Grounded Reasoning General
7. OSGuard: A Benchmark for Safety in Computer-Use Agents GUI / computer use

June 12, 2026 · 7

1. StreamMemBench: Streaming Evaluation of Agent Memory for Future-Oriented Assistance Dialogue
2. Running the Gauntlet: Re-evaluating the Capabilities of Agents Beyond Familiar Environments General
3. LessonBench-V1: A Benchmark Dataset for Evaluating AI Lesson Generation Agents General
4. SkillMutator: Benchmarking and Defending Language-and-Code Cross-modal Attacks on LLM Agent Skills General
5. Same-Origin Policy for Agentic Browsers Web & browser
6. Dialogue SWE-Bench: A Benchmark for Dialogue-Driven Coding Agents Coding / SWE
7. Hidden in Plain Sight: Benchmarking Agent Safety Against Decomposition Attacks with DECOMPBENCH Safety

June 11, 2026 · 9

1. EvoArena: Tracking Memory Evolution for Robust LLM Agents in Dynamic Environments General
2. EpiBench: Verifiable Evaluation of AI Agents on Epigenomics Analysis General
3. SEVRA-BENCH: Social Engineering of Vulnerabilities in Review Agents General
4. Who Pays the Price? Stakeholder-Centric Prompt Injection Benchmarking for Real-world Web Agents Web & browser
5. TerraBench: Can Agents Reason Over Heterogeneous Earth-System Data? General
6. EvoBrowseComp: Benchmarking Search Agents on Evolving Knowledge General
7. DailyReport: An Open-ended Benchmark for Evaluating Search Agents on Daily Search Tasks General
8. LoHoSearch: Benchmarking Long-Horizon Search Agents Beyond the Human Difficulty Ceiling General
9. GeoNatureAgent Benchmark: Benchmarking LLM Agents for Environmental Geospatial Analysis Across Frontier and Open-Weight Foundation Models General

June 10, 2026 · 6

1. Benchmarking AI Agents for Addressing Scientific Challenges Across Scales Science & research
2. WorkBench Revisited: Workplace Agents Two Years On Work / office
3. Claw-SWE-Bench: A Benchmark for Evaluating OpenClaw-style Agent Harnesses on Coding Tasks Coding / SWE
4. CORE-Bench: A Comprehensive Benchmark for Code Retrieval in the Era of Agentic Coding Coding / SWE
5. MedCTA: A Benchmark for Clinical Tool Agents Multimodal
6. DFAH-Bench: Benchmarking Observable Agent Instability in Financial Decision-Making General

June 9, 2026 · 9

1. AI Coding Agents Can Reproduce Social Science Findings Coding / SWE
2. ABC-Bench: An Agentic Bio-Capabilities Benchmark for Biosecurity Embodied / robotics
3. From Knowing to Acting: Benchmarking Self-Awareness Capability of LLM Agents General
4. T1-Bench: Benchmarking Multi-Scenario Agents in Real-World Domains General
5. Workflow-GYM: Towards Long-Horizon Evaluation of Computer-use Agentic tasks in Real-World Professional Fields Work / office
6. RedAct: Redacting Agent Capability Traces for Procedural Skill Protection General
7. FinSkillBench: Evaluating AI Agents and Domain Skills for Investment Management General
8. STAGE-Claw: Automated State-based Agent Benchmarking for Realistic Scenarios General
9. Right Family, Wrong Skill: Benchmarking Risk Exposure in Agent Skill Retrieval General

June 8, 2026 · 8

1. $τ$-Rec: A Verifiable Benchmark for Agentic Recommender Systems Dialogue
2. Bittensor Agent Arenas as a Trajectory Primitive: Distilling a Shopping Agent from ShoppingBench Subnet Traces General
3. OmniGameArena: A Unified UE5 Benchmark for VLM Game Agents with Improvement Dynamics Multimodal
4. iOSWorld: A Benchmark for Personally Intelligent Phone Agents Mobile
5. SpatialWorld: Benchmarking Interactive Spatial Reasoning of Multimodal Agents in Real-World Tasks Multimodal
6. H2HMem: A Multimodal Memory Benchmark for Agents in Human-Human Interactions Multimodal
7. WeaveBench: A Long-Horizon, Real-World Benchmark for Computer-Use Agents with Hybrid Interfaces GUI / computer use
8. Hardening Agent Benchmarks with Adversarial Hacker-Fixer Loops General

June 7, 2026 · 1

1. PerspectiveGap: A Benchmark for Multi-Agent Orchestration Prompting Multi-agent

June 5, 2026 · 8

1. The Cold-Start Safety Gap in LLM Agents Safety
2. Beyond Goodhart's Law: A Dynamic Benchmark for Evaluating Compliance in Multi-Agent Systems Multi-agent
3. QBugLM: An Agentic Benchmarking Framework for LLM-based Quantum Software Debugging Coding / SWE
4. SWE-Explore: Benchmarking How Coding Agents Explore Repositories Coding / SWE
5. MalSkillBench: A Runtime-Verified Benchmark of Malicious Agent Skills Safety
6. FinEvolveBench: A Benchmark for Self-Evolving Agents on Low-Repetition Tasks with Implicit Rewards General
7. SWE-Marathon: Can Agents Autonomously Complete Ultra-Long-Horizon Software Work? Coding / SWE
8. What Your Posts Reveal: A Benchmark and Agentic Framework for User-Level Privacy Leakage on Social Media Work / office

June 4, 2026 · 7

1. DragOn: A Benchmark and Dataset for Drag-Based GUI Interactions GUI / computer use
2. MacArena: Benchmarking Computer Use Agents on an Online macOS Environment GUI / computer use
3. HUSH-Bench: Measuring Memory-Use Boundaries for Sensitive History in Conversational Agents General
4. Asuka-Bench: Benchmarking Code Agents on Underspecified User Intent and Multi-Round Refinement Web & browser
5. When Tools Fail: Benchmarking Dynamic Replanning and Anomaly Recovery in LLM Agents General
6. AdaPlanBench: Evaluating Adaptive Planning in Large Language Model Agents under World and User Constraints General
7. TensorBench: Benchmarking Coding Agents on a Compiler-Based Tensor Framework Coding / SWE

June 3, 2026 · 7

1. Agents' Last Exam Work / office
2. SentinelBench: A Benchmark for Long-Running Monitoring Agents Web & browser
3. Benchmarking Living-Screen-Native GUI Agents on Short-Video Platforms GUI / computer use
4. LifeSide: Benchmarking Agents as Lifelong Digital Companions Multi-agent
5. Impostor: An Agent-Curated Benchmark for Realistic AIGC Manipulation Localization Multimodal
6. CyberGym-E2E: Scalable Real-World Benchmark for AI Agents' End-to-End Cybersecurity Capabilities General
7. DeployBench: Benchmarking LLM Agents for Research Artifact Deployment Coding / SWE

June 2, 2026 · 9

1. Benchmarking Visual State Tracking in Multimodal Video Understanding Multimodal
2. Hedge-Bench: Benchmarking Agents on Hard, Realistic Tasks Pertaining to Financial Reasoning General
3. RealClawBench: Live OpenClaw Benchmarks from Real Developer-Agent Sessions General
4. BigFinanceBench: A Workflow-Grounded Benchmark for Financial-Research Agents Science & research
5. Diagnosing Knowledge Gaps in LLM Tool Use: An Agentic Benchmark for Novel API Acquisition Tool use
6. LEAP: Supercharging LLMs for Formal Mathematics with Agentic Frameworks General
7. Agentic Relationship Harm: Benchmarking and Gating Relational Manipulation in AI Agents General
8. MedCUA-Bench: A Screenshot-Only Benchmark for Clinical Computer-Use Agents GUI / computer use
9. DeskCraft: Benchmarking Desktop Agents on Professional Workflows and Human-in-the-Loop Collaboration Work / office

June 1, 2026 · 11

1. $Ψ$-Bench: Evaluating Persona-Sensitive Influencing in Persuasive Dialogues Dialogue
2. MCP-Persona: Benchmarking LLM Agents on Real-World Personal Applications via Environment Simulation Tool use
3. K-BrowseComp: A Web Browsing Agent Benchmark Grounded in Korean Contexts General
4. SPADE-Bench: Evaluating Spontaneous Strategic Deception in Agents via Plan-Action Divergence General
5. TVIR: Building Deep Research Agents Towards Text-Visual Interleaved Report Generation Science & research
6. POIROT: Interrogating Agents for Failure Detection in Multi-Agent Systems Multi-agent
7. AgentRedBench: Dynamic Redteaming and Integration-Aware Defense for LLM Agents over SaaS Integrations General
8. AutoMedBench: Towards Medical AutoResearch with Agentic AI Models Multimodal
9. SMH-Bench: Benchmarking LLM Agents for Environment-Grounded Reasoning and Action in Smart Homes General
10. CRAB-Bench: Evaluating LLM Agents under Complex Task Dependencies and Human-aligned User Simulation General
11. RoleCDE:Benchmarking and Mitigating Role-Alignment Trade-offs in Role-Playing Agents General

May 2026

May 31, 2026 · 5

1. TimeSage-MT: A Multi-Turn Benchmark for Evaluating Agentic Time Series Reasoning Dialogue
2. DrugClaw and DrugAudit: A Primary-Source-Grounded Agent and Authority-Aware Benchmark for Drug-Information Question Answering Multi-agent
3. SABER: Benchmarking Operational Safety of LLM Coding Agents in Stateful Project Workspaces Coding / SWE
4. 3DCodeBench: Benchmarking Agentic Procedural 3D Modeling Via Code Multimodal
5. TravelEval: A Comprehensive Benchmarking Framework for Evaluating LLM-Powered Travel Planning Agents General

May 30, 2026 · 8

1. Benchmarking Security Risk Detection and Verification in Open Agentic Skill Ecosystems General
2. Momento: Evaluating Persistent Memory and Reasoning with Multi-Session Agentic Conversations General
3. SuperMemory-VQA: An Egocentric Visual Question-Answering Benchmark for Long-Horizon Memory General
4. I-WebGenBench : Evaluating Interactivity in LLM-Generated Scientific Web Applications Science & research
5. PersonaTrail: Benchmarking Personalized Web Agents through Browsing Trails Web & browser
6. ForeSci: Evaluating LLM Agents for Forward-Looking AI Research Judgment General
7. DEMM-Bench: A Cross-Regime Benchmark for Agent-Runtime Governance-Evidence Sufficiency General
8. "I Strongly Suspect This Website Is a Scam": Benchmarking PII Leakage and Detection without Defense in Autonomous Web Agents Web & browser

May 29, 2026 · 9

1. VESTA: Visual Exploration with Statistical Tool Agents Multimodal
2. ROGUE: Misaligned Agent Behavior Arising from Ordinary Computer Use GUI / computer use
3. SVI-Bench: A Dynamic Microworld for Strategic Video Intelligence Multi-agent
4. TadA-Bench: A Million-Variant Benchmark for Future-Round Discovery Toward Agentic Protein Engineering General
5. MineExplorer: Evaluating Open-World Exploration of MLLM Agents in Minecraft Multi-agent
6. BlueFin: Benchmarking LLM Agents on Financial Spreadsheets Work / office
7. VisualLeakBench: Reproducible Action-Boundary Propagation Failures in Vision-Language Agents Multimodal
8. PrivacyPeek: Auditing What LLM-Based Agents Acquire, Not Just What They Say General
9. MosaicLeaks:Privacy Risks in Querying-in-the-Open for Deep Research Agents Science & research

May 28, 2026 · 16

1. Crafter: A Multi-Agent Harness for Editable Scientific Figure Generation from Diverse Inputs Multi-agent
2. LongDS-Bench: On the Failure of Long-Horizon Agentic Data Analysis General
3. SpecBench: Evaluating Specification-Level Reasoning for Software Engineering LLM Agents Coding / SWE
4. ResearchClawBench: A Benchmark for End-to-End Autonomous Scientific Research Science & research
5. DirectorBench: Diagnosing Long-Form Video Generation with Personalized Multi-Agent Evaluation Multi-agent
6. HEART-Bench: Do LLM Agents Exhibit Human-like Psychology? General
7. Redundant or Necessary? A Benchmark for Detecting Redundant Steps in Agent Trajectories General
8. Notation Matters: A Benchmark Study of Token-Optimized Formats in Agentic AI Systems General
9. PTCG-Bench: Can LLM Agents Master Pokémon Trading Card Game? General
10. MINDGAMES: A Live Arena for Evaluating Social and Strategic Reasoning in Multi-Agent LLMs Multi-agent
11. Recovering Policy-Induced Errors: Benchmarking and Trajectory Synthesis for Robust GUI Agents GUI / computer use
12. WorldMemArena: Evaluating Multimodal Agent Memory Through Action-World Interaction Multimodal
13. CrossAlpha: An Annual-Report Benchmark for Cross-Market Factor Research (with LLM Agents) General
14. OpenClawBench: Benchmarking Process-side Anomalies in Real-world Agent Execution Trajectories General
15. BenchTrace: A Benchmark for Testing Reflection Ability and Controlled Evolution in LLM Agents General
16. Relevance as a Vulnerability: How Web Retrieval Degrades Safety Alignment in LLM Agents Safety

May 27, 2026 · 15

1. The Best-Laid SCHEMEs: Coordinated Sabotage and Monitoring in Multi-Agent Systems Multi-agent
2. LiveBrowseComp: Are Search Agents Searching, or Just Verifying What They Already Know? General
3. VeriTrip: A Verifiable Benchmark for Travel Planning Agents over Unstructured Web Corpora Multimodal
4. MaskClaw: Edge-Side Personalized Privacy Arbitration for GUI Agents with Behavior-Driven Skill Evolution Work / office
5. A Matter of TASTE: Improving Coverage and Difficulty of Agent Benchmarks General
6. Beyond One Path: Evaluating and Enhancing Divergent Thinking in Interactive LLM Agents General
7. From Knowing to Doing: A Memory-Controlled Benchmark for LLM Trading Agents on Stock Markets General
8. Plant, Persist, Trigger: Sleeper Attack on Large Language Model Agents General
9. Agentic Active Omni-Modal Perception for Multi-Hop Audio-Visual Reasoning General
10. OR-Space: A Full-Lifecycle Workspace Benchmark for Industrial Optimization Agents General
11. Ask Now, Use Later: Benchmarking the Proactivity Gap in Long-Lived LLM Agents General
12. KVoiceBench, KOpenAudioBench, and KMMAU: Agent-Driven Korean Speech Benchmarks for Evaluating SpeechLMs General
13. Harness-Bench: Measuring Harness Effects across Models in Realistic Agent Workflows General
14. Dr-CiK: A Testbed for Foresight-Driven Agents General
15. Towards Faithful Agentic XAI: A Verification Method and an Open-World Benchmark for Better Model Faithfulness General

May 26, 2026 · 9

1. AndroidDaily: A Verifiable Benchmark for Mobile GUI Agents on Real-World Closed-Source Applications GUI / computer use
2. AgenticVBench: Can AI Agents Complete Real-World Post-Production Tasks? Multimodal
3. DynaSchedBench: Calibrated Dynamic Scheduling Benchmarks and Observability Paradox in LLM-based Scheduling Agents General
4. ENPMR-Bench: Benchmarking Proactive Memory Retrieval for Emotional Support Agents General
5. VitaBench 2.0: Evaluating Personalized and Proactive Agents in Long-Term User Interactions General
6. Scaling, Benchmarking, and Reasoning of Vision-Language Agents for Mobile GUI Navigation Multimodal
7. Helicase: Uncertainty-Guided Supply Chain Knowledge Graph Construction with Autonomous Multi-Agent LLMs Multi-agent
8. LogDx-CI: Benchmarking Log Reduction Tools for LLM Root-Cause Diagnosis Coding / SWE
9. Verus-SpecGym: An Agentic Environment for Evaluating Specification Autoformalization Coding / SWE

May 25, 2026 · 5

1. JobBench: Aligning Agent Work With Human Will Work / office
2. Anchor: Mitigating Artifact Drift in Agent Benchmark Generation Work / office
3. Your Agents Are Aging Too: Agent Lifespan Engineering for Deployed Systems General
4. AgentHijack: Benchmarking Computer Use Agent Robustness to Common Environment Corruptions GUI / computer use
5. Personalize-then-Store: Benchmarking and Learning Personalized Memory for Long-horizon Agents General

May 24, 2026 · 1

1. GroupTravelBench: Benchmarking LLM Agents on Multi-Person Travel Planning Dialogue

May 23, 2026 · 1

1. MDIA: A Multi-Agent Diagnostic Intelligence Pipeline on HealthBench Professional Multi-agent

May 22, 2026 · 5

1. ContextEcho: A Benchmark for Persona Drift in Long Agentic-Coding Sessions General
2. VISTA: An End-to-End Benchmark for Visual Spec-to-Web-App Coding Agents Coding / SWE
3. AvalancheBench: Evaluating Enterprise Data Agents Through Latent World Recovery Coding / SWE
4. EvoCode-Bench: Evaluating Coding Agents in Multi-Turn Iterative Interactions Coding / SWE
5. PhotoFlow: Agentic 3D Virtual Photography Missions Multimodal

May 21, 2026 · 7

1. MBABench: Evaluating LLM Agents on End-to-End Spreadsheet Tasks in Finance General
2. Boiling the Frog: A Multi-Turn Benchmark for Agentic Safety Safety
3. Measuring Security Without Fooling Ourselves: Why Benchmarking Agents Is Hard General
4. SynAE: A Framework for Measuring the Quality of Synthetic Data for Tool-Calling Agent Evaluations Web & browser
5. TerminalWorld: Benchmarking Agents on Real-World Terminal Tasks General
6. AgroTools: A Benchmark for Tool-Augmented Multimodal Agents in Agriculture Multimodal
7. SGR-Bench: Benchmarking Search Agents on State-Gated Retrieval General

May 20, 2026 · 6

1. SMDD-Bench: Can LLMs Solve Real-World Small Molecule Drug Design Tasks? General
2. MemArena: An Ego-Centric Benchmark for On-Device Agentic Personal Memory Assistants at Scale Dialogue
3. What Twelve LLM Agent Benchmark Papers Disclose About Themselves: A Pilot Audit and an Open Scoring Schema General
4. SpecBench: Measuring Reward Hacking in Long-Horizon Coding Agents Coding / SWE
5. InferenceBench: A Benchmark for Open-Ended LLM Inference Optimization by AI Agents General
6. MemGym: a Long-Horizon Memory Environment for LLM Agents Web & browser

May 19, 2026 · 10

1. The Yes-Man Syndrome: Benchmarking Abstention in Embodied Robotic Agents Embodied / robotics
2. AgentAtlas: Beyond Outcome Leaderboards for LLM Agents General
3. WildRoadBench: A Wild Aerial Road-Damage Grounding Benchmark for Vision-Language Models and Autonomous Agents Multimodal
4. EngiAI: A Multi-Agent Framework and Benchmark Suite for LLM-Driven Engineering Design Multi-agent
5. Physics-in-the-Loop: A Hybrid Agentic Architecture for Validated CAD Engineering Design General
6. CutVerse: A Compositional GUI Agents Benchmark for Media Post-Production Editing GUI / computer use
7. Vision Harnessing Agent for Open Ad-hoc Segmentation GUI / computer use
8. HSCO-Bench: An Agent-Driven End-to-End Hardware-Software Co-design Benchmark for Systems-on-Chip Coding / SWE
9. PrefBench: Evaluating Zero-Shot LLM Agents in Hidden-Preference Personalized Pricing Negotiations General
10. RoboJailBench: Benchmarking Adversarial Attacks and Defenses in Embodied Robotic Agents Embodied / robotics

May 18, 2026 · 14

1. POLAR-Bench: A Diagnostic Benchmark for Privacy-Utility Trade-offs in LLM Agents General
2. DecisionBench: A Benchmark for Emergent Delegation in Long-Horizon Agentic Workflows Dialogue
3. SkillGenBench: Benchmarking Skill Generation Pipelines for LLM Agents Coding / SWE
4. Overeager Coding Agents: Measuring Out-of-Scope Actions on Benign Tasks Coding / SWE
5. MINTEval: Evaluating Memory under Multi-Target Interference in Long-Horizon Agent Systems Dialogue
6. STT-Arena: A More Realistic Environment for Tool-Using with Spatio-Temporal Dynamics General
7. EvoMemBench: Benchmarking Agent Memory from a Self-Evolving Perspective General
8. Duet instrumentation: An Agentic Approach to Improving Sensitivity in Cloud Service Benchmarking General
9. DocOS: Towards Proactive Document-Guided Actions in GUI Agents GUI / computer use
10. ProcCtrlBench: Evaluating Process-Level Defects and Control Preservation in LLM Coding Agents Coding / SWE
11. LivePI: More Realistic Benchmarking of Agents Against Indirect Prompt Injection Coding / SWE
12. SVFSearch: A Multimodal Knowledge-Intensive Benchmark for Short-Video Frame Search in the Gaming Vertical Domain Multimodal
13. BacktestBench: Benchmarking Large Language Models for Automated Quantitative Strategy Backtesting Coding / SWE
14. Evaluating Cognitive Age Alignment in Interactive AI Agents Multimodal

May 17, 2026 · 6

1. WebGameBench: Requirement-to-Application Evaluation for Coding Agents via Browser-Native Games Coding / SWE
2. Evaluating Deep Research Agents on Expert Consulting Work: A Benchmark with Verifiers, Rubrics, and Cognitive Traps Work / office
3. SaaSBench: Exploring the Boundaries of Coding Agents in Long-Horizon Enterprise SaaS Engineering Coding / SWE
4. FML-bench: A Controlled Study of AI Research Agent Strategies from the Perspective of Search Dynamics Science & research
5. ASPI: Seeking Ambiguity Clarification Amplifies Prompt Injection Vulnerability in LLM Agents General
6. ContractBench: Can LLM Agents Preserve Observation Contracts? General

May 16, 2026 · 4

1. 1GC-7RC: One Graphic Card -- Seven Research Challenges! How Good Are AI Agents at Doing Your Job? Coding / SWE
2. When JSON Is Not Enough: Semantic Reliability of Schema-Constrained LLM Ordering Agents General
3. TOBench: A Task-Oriented Omni-Modal Benchmark for Real-World Tool-Using Agents Work / office
4. AgentKernelArena: Generalization-Aware Benchmarking of GPU Kernel Optimization Agents Coding / SWE

May 15, 2026 · 9

1. CHI-Bench: Can AI Agents Automate End-to-End, Long-Horizon, Policy-Rich Healthcare Workflows? General
2. SLEIGHT-Bench: A Benchmark of Evasion Attacks Against Agent Monitors Coding / SWE
3. Confirming Correct, Missing the Rest: LLM Tutoring Agents Struggle Where Feedback Matters Most Dialogue
4. ShopGym: An Integrated Framework for Realistic Simulation and Scalable Benchmarking of E-Commerce Web Agents Web & browser
5. RoadmapBench: Evaluating Long-Horizon Agentic Software Development Across Version Upgrades Coding / SWE
6. SaaS-Bench: Can Computer-Use Agents Leverage Real-World SaaS to Solve Professional Workflows? Work / office
7. BioXArena: Benchmarking LLM Agents on Multi-Modal Biomedical Machine Learning Tasks General
8. SMMBench: A Benchmark for Source-Distributed Multimodal Agent Memory Multimodal
9. Rule2DRC: Benchmarking LLM Agents for DRC Script Synthesis with Execution-Guided Test Generation Coding / SWE

May 14, 2026 · 10

1. MemEye: A Visual-Centric Evaluation Framework for Multimodal Agent Memory Multimodal
2. $π$-Bench: Evaluating Proactive Personal Assistant Agents in Long-Horizon Workflows General
3. TwinRouterBench: Fast Static and Live Dynamic Evaluation for Realistic Agentic LLM Routing Coding / SWE
4. Cattle Trade: A Multi-Agent Benchmark for LLM Bluffing, Bidding, and Bargaining Multi-agent
5. When Robots Do the Chores: A Benchmark and Agent for Long-Horizon Household Task Execution Embodied / robotics
6. GroupMemBench: Benchmarking LLM Agent Memory in Multi-Party Conversations Work / office
7. SWE-Chain: Benchmarking Coding Agents on Chained Release-Level Package Upgrades Coding / SWE
8. Herculean: An Agentic Benchmark for Financial Intelligence General
9. TeachArena: Are Language Agents Ready for Realistic Teaching Work? Work / office
10. Auditing Agent Harness Safety Safety

May 13, 2026 · 15

1. ExploitBench: A Capability Ladder Benchmark for LLM Cybersecurity Agents General
2. ClawForge: Generating Executable Interactive Benchmarks for Command-Line Agents General
3. PolitNuggets: Benchmarking Agentic Discovery of Long-Tail Political Facts Multi-agent
4. EnergyAgentBench: Benchmarking LLM Agents on Live Energy Infrastructure Data General
5. PBT-Bench: Benchmarking AI Agents on Property-Based Testing General
6. Collider-Bench: Benchmarking AI Agents with Particle Physics Analysis Reproduction Coding / SWE
7. EVA-Bench: A New End-to-end Framework for Evaluating Voice Agents General
8. AgentTrap: Measuring Runtime Trust Failures in Third-Party Agent Skills General
9. Turning Intent into Specifications: A Benchmark and an Interactive User-Assistant Agent Coding / SWE
10. Is Agentic AI Ready for Real-World Hardware Engineering? A Deep Dive with Phoenix-bench Coding / SWE
11. RealICU: Do LLM Agents Understand Long-Context ICU Data? A Benchmark Beyond Behavior Imitation General
12. When Does Hierarchy Help? Benchmarking Agent Coordination in Event-Driven Industrial Scheduling Multi-agent
13. SWE-Cycle: Benchmarking Code Agents across the Complete Issue Resolution Cycle Coding / SWE
14. TERMS-Bench: Diagnosing LLM Negotiation Agents Beyond Deal Rate General
15. EcoGEO: Trajectory-Aware Evidence Ecosystems for Web-Enabled LLM Search Agents General

May 12, 2026 · 11

1. Neurodata Without Boredom: Benchmarking Agentic AI for Data Reuse Coding / SWE
2. Do Androids Dream of Breaking the Game? Systematically Auditing AI Agent Benchmarks with BenchJack Coding / SWE
3. Covering Human Action Space for Computer Use: Data Synthesis and Benchmark GUI / computer use
4. LongMemEval-V2: Evaluating Long-Term Agent Memory Toward Experienced Colleagues Coding / SWE
5. SkillSafetyBench: Evaluating Agent Safety under Skill-Facing Attack Surfaces Safety
6. RecRM-Bench: Benchmarking Multidimensional Reward Modeling for Agentic Recommender Systems General
7. MedMemoryBench: Benchmarking Agent Memory in Personalized Healthcare General
8. AgentDisCo: Towards Disentanglement and Collaboration in Open-ended Deep Research Agents Science & research
9. Can LLM Agents Respond to Disasters? Benchmarking Heterogeneous Geospatial Reasoning in Emergency Operations General
10. CTFusion: A CTF-based Benchmark for LLM Agent Evaluation General
11. An Empirical Study of Automating Agent Evaluation General

May 11, 2026 · 15

1. ABRA: Agent Benchmark for Radiology Applications General
2. RecoAtlas: From Semantic Plausibility to Set-Level Utility in LLM Recommendation Agents General
3. WildClawBench: A Benchmark for Real-World, Long-Horizon Agent Evaluation Multimodal
4. MDrive: Benchmarking Closed-Loop Cooperative Driving for End-to-End Multi-agent Systems Multi-agent
5. AssayBench: An Assay-Level Virtual Cell Benchmark for LLMs and Agents General
6. ComplexMCP: Evaluation of LLM Agents in Dynamic, Interdependent, and Large-Scale Tool Sandbox General
7. LITMUS: Benchmarking Behavioral Jailbreaks of LLM Agents in Real OS Environments Multi-agent
8. CrackMeBench: Binary Reverse Engineering for Agents Coding / SWE
9. Can Agent Benchmarks Support Their Scores? Evidence-Supported Bounds for Interactive-Agent Evaluation General
10. Agentic Performance at the Edge: Insights from Benchmarking General
11. Agent-ValueBench: A Comprehensive Benchmark for Evaluating Agent Values General
12. AgentRx: A Benchmark Study of LLM Agents for Multimodal Clinical Prediction Tasks Multimodal
13. TRACER: Verifiable Generative Provenance for Multimodal Tool-Using Agents Multimodal
14. EgoMemReason: A Memory-Driven Reasoning Benchmark for Long-Horizon Egocentric Video Understanding Embodied / robotics
15. EnactToM: An Evolving Benchmark for Functional Theory of Mind in Embodied Agents Embodied / robotics

May 10, 2026 · 10

1. CalBench: Evaluating Coordination-Privacy Trade-offs in Multi-Agent LLMs Multi-agent
2. An Executable Benchmarking Suite for Tool-Using Agents Coding / SWE
3. MOTOR-Bench: A Real-world Dataset and Multi-agent Framework for Zero-shot Human Mental State Understanding Multi-agent
4. Ambig-DS: A Benchmark for Task-Framing Ambiguity in Data-Science Agents General
5. MonitoringBench: Semi-Automated Red-Teaming for Agent Monitoring Coding / SWE
6. DeepTumorVQA: A Hierarchical 3D CT Benchmark for Stage-Wise Evaluation of Medical VLMs and Tool-Augmented Agents Multimodal
7. CodeClinic: Evaluating Automation of Coding Skills for Clinical Reasoning Agents Coding / SWE
8. Don't Click That: Teaching Web Agents to Resist Deceptive Interfaces Web & browser
9. MiroBench: Benchmarking Realism in Agentic Simulation of Real-world Discussions General
10. LLM Agents Already Know When to Call Tools -- Even Without Reasoning General

May 9, 2026 · 8

1. FORTIS: Benchmarking Over-Privilege in Agent Skills General
2. GAMBIT: A Three-Mode Benchmark for Adversarial Robustness in Multi-Agent LLM Collectives Multi-agent
3. MDGYM: Benchmarking AI Agents on Molecular Simulations Coding / SWE
4. OPT-BENCH: Evaluating the Iterative Self-Optimization of LLM Agents in Large-Scale Search Spaces General
5. When Agents Overtrust Environmental Evidence: An Extensible Agentic Framework for Benchmarking Evidence-Grounding Defects in LLM Agents General
6. Beyond the All-in-One Agent: Benchmarking Role-Specialized Multi-Agent Collaboration in Enterprise Workflows Work / office
7. AgentCollabBench: Diagnosing When Good Agents Make Bad Collaborators Coding / SWE
8. Source or It Didn't Happen: A Multi-Agent Framework for Citation Hallucination Detection Multi-agent

May 8, 2026 · 10

1. Measuring What Matters: Benchmarking Generative, Multimodal, and Agentic AI in Healthcare Multimodal
2. SWE Atlas: Benchmarking Coding Agents Beyond Issue Resolution Coding / SWE
3. AgentEscapeBench: Evaluating Out-of-Domain Tool-Grounded Reasoning in LLM Agents General
4. CyBiasBench: Benchmarking Bias in LLM Agents for Cyber-Attack Scenarios General
5. MMTB: Evaluating Terminal Agents on Multimedia-File Tasks General
6. InterLV-Search: Benchmarking Interleaved Multimodal Agentic Search Multimodal
7. Can Agents Price a Reaction? Evaluating LLMs on Chemical Cost Reasoning General
8. Learning Agent Routing From Early Experience General
9. SREGym: A Live Benchmark for AI SRE Agents with High-Fidelity Failure Scenarios General
10. TeamBench: Evaluating Agent Coordination under Enforced Role Separation General

May 7, 2026 · 8

1. Agentick: A Unified Benchmark for General Sequential Decision-Making Agents Multi-agent
2. STALE: Can LLM Agents Know When Their Memories Are No Longer Valid? General
3. Instrumental Choices: Measuring the Propensity of LLM Agents to Pursue Instrumental Behaviors General
4. More Than Can Be Said: A Benchmark and Framework for Pre-Question Scientific Ideation Science & research
5. MANTRA: Synthesizing SMT-Validated Compliance Benchmarks for Tool-Using LLM Agents General
6. BioMedArena: An Open-source Toolkit for Building and Evaluating Biomedical Deep Research Agents Science & research
7. Breaking, Stale, or Missing? Benchmarking Coding Agents on Project-Level Test Evolution Coding / SWE
8. SkillRet: A Large-Scale Benchmark for Skill Retrieval in LLM Agents General

May 6, 2026 · 2

1. Partial Evidence Bench: Benchmarking Authorization-Limited Evidence in Agentic Systems Work / office
2. SWE-WebDevBench: Evaluating Coding Agent Application Platforms as Virtual Software Agencies Coding / SWE

May 5, 2026 · 2

1. MOSAIC-Bench: Measuring Compositional Vulnerability Induction in Coding Agents Coding / SWE
2. Workspace-Bench 1.0: Benchmarking AI Agents on Workspace Tasks with Large-Scale File Dependencies General

May 4, 2026 · 2

1. DataClawBench: An Agent Benchmark for Exploratory Real-World Financial Data Analysis General
2. PhysicianBench: Evaluating LLM Agents in Real-World EHR Environments General

May 3, 2026 · 1

1. NeuroState-Bench: A Human-Calibrated Benchmark for Commitment Integrity in LLM Agent Profiles General

May 2, 2026 · 3

1. LiveFMBench: Unveiling the Power and Limits of Agentic Workflows in Specification Generation General
2. ESARBench: A Benchmark for Agentic UAV Embodied Search and Rescue Embodied / robotics
3. EO-Gym: A Multimodal, Interactive Environment for Earth Observation Agents Multimodal

May 1, 2026 · 2

1. Can Coding Agents Reproduce Findings in Computational Materials Science? Coding / SWE
2. Foresight Arena: An On-Chain Benchmark for Evaluating AI Forecasting Agents General

April 2026

April 30, 2026 · 4

1. Claw-Eval-Live: A Live Agent Benchmark for Evolving Real-World Workflows General
2. What Makes a Good Terminal-Agent Benchmark Task: A Guideline for Adversarial, Difficult, and Legible Evaluation Design General
3. WindowsWorld: A Process-Centric Benchmark of Autonomous GUI Agents in Professional Cross-Application Environments Work / office
4. Context as Prior: Bayesian-Inspired Intent Inference for Non-Speaking Agents with a Household Cat Testbed Embodied / robotics

April 29, 2026 · 3

1. ClawGym: A Scalable Framework for Building Effective Claw Agents General
2. SWE-Bench 5G: Benchmarking AI Coding Agents on Telecom Network Engineering Tasks Coding / SWE
3. LATTICE: Evaluating Decision Support Utility of Crypto Agents General

April 28, 2026 · 2

1. DV-World: Benchmarking Data Visualization Agents in Real-World Scenarios Work / office
2. AutoResearchBench: Benchmarking AI Agents on Complex Scientific Literature Discovery Science & research

April 27, 2026 · 4

1. Odysseys: Benchmarking Web Agents on Realistic Long Horizon Tasks Web & browser
2. BenchGuard: Who Guards the Benchmarks? Automated Auditing of LLM Agent Benchmarks General
3. GAIA-v2-LILT: Multilingual Adaptation of Agent Benchmark beyond Translation General
4. GAMMAF: A Common Framework for Graph-Based Anomaly Monitoring Benchmarking in LLM Multi-Agent Systems Multi-agent

April 26, 2026 · 4

1. MarketBench: Evaluating AI Agents as Market Participants Coding / SWE
2. Benchmarking Web Agent Safety under E-commerce Deceptive Interfaces Web & browser
3. ClawMark: A Living-World Benchmark for Multi-Turn, Multi-Day, Multimodal Coworker Agents Multimodal
4. PhysCodeBench: Benchmarking Physics-Aware Symbolic Simulation of 3D Scenes via Self-Corrective Multi-Agent Refinement Multi-agent

April 25, 2026 · 1

1. CUJBench: Benchmarking LLM-Agent on Cross-Modal Failure Diagnosis from Browser to Backend Web & browser

April 24, 2026 · 2

1. Seeing the Whole Elephant: A Benchmark for Failure Attribution in LLM-based Multi-Agent Systems Multi-agent
2. AgentSearchBench: A Benchmark for AI Agent Search in the Wild General

April 23, 2026 · 4

1. Do MLLMs Understand Pointing? Benchmarking and Enhancing Referential Reasoning in Egocentric Vision Multimodal
2. CI-Work: Benchmarking Contextual Integrity in Enterprise LLM Agents Work / office
3. PaperMind: Benchmarking Agentic Reasoning and Critique over Scientific Papers in Multimodal LLMs Science & research
4. IntrAgent: An LLM Agent for Content-Grounded Information Retrieval through Literature Review General

April 22, 2026 · 3

1. Beyond Pixels: Introspective and Interactive Grounding for Visualization Agents GUI / computer use
2. ActuBench: A Multi-Agent LLM Pipeline for Generation and Evaluation of Actuarial Reasoning Tasks Multi-agent
3. SkillLearnBench: Benchmarking Continual Learning Methods for Agent Skill Generation on Real-World Tasks General

April 21, 2026 · 4

1. From Recall to Forgetting: Benchmarking Long-Term Memory for Personalized Agents General
2. Cyber Defense Benchmark: Agentic Threat Hunting Evaluation for LLMs in SecOps General
3. Do Agents Dream of Root Shells? Partial-Credit Evaluation of LLM Agents in Capture the Flag Challenges General
4. AutomationBench General

April 20, 2026 · 5

1. Human-Guided Harm Recovery for Computer Use Agents GUI / computer use
2. ClawEnvKit: Automatic Environment Generation for Claw-Like Agents General
3. ComPASS: Towards Personalized Agentic Social Support via Tool-Augmented Companionship Dialogue
4. Agent-World: Scaling Real-World Environment Synthesis for Evolving General Agent Intelligence General
5. AJ-Bench: Benchmarking Agent-as-a-Judge for Environment-Aware Evaluation General

April 19, 2026 · 2

1. SkillFlow:Benchmarking Lifelong Skill Discovery and Evolution for Autonomous Agents General
2. BranchBench: Aligning Database Branching with Agentic Demands Coding / SWE

April 18, 2026 · 5

1. Systematic Capability Benchmarking of Frontier Large Language Models for Offensive Cyber Tasks Multi-agent
2. SeekerGym: A Benchmark for Reliable Information Seeking General
3. If Only My CGM Could Speak: A Privacy-Preserving Agent for Question Answering over Continuous Glucose Data General
4. Mini-BEHAVIOR-Gran: Revealing U-Shaped Effects of Instruction Granularity on Language-Guided Embodied Agents Embodied / robotics
5. PersonalHomeBench: Evaluating Agents in Personalized Smart Homes Multimodal

April 17, 2026 · 4

1. Agentic Large Language Models for Training-Free Neuro-Radiological Image Analysis Multi-agent
2. SocialGrid: A Benchmark for Planning and Social Reasoning in Embodied Multi-Agent Systems Multi-agent
3. MemEvoBench: Benchmarking Safety Risks from Memory Misevolution in LLM Agents Safety
4. GTA-2: Benchmarking General Tool Agents from Atomic Tool-Use to Open-Ended Workflows Tool use

April 16, 2026 · 6

1. MM-WebAgent: A Hierarchical Multimodal Web Agent for Webpage Generation Web & browser
2. CoopEval: Benchmarking Cooperation-Sustaining Mechanisms and LLM Agents in Social Dilemmas General
3. HarmfulSkillBench: How Do Harmful Skills Weaponize Your Agents? General
4. From Reactive to Proactive: Assessing the Proactivity of Voice Agents via ProVoice-Bench Multimodal
5. FedGUI: Benchmarking Federated GUI Agents across Heterogeneous Platforms, Devices, and Operating Systems GUI / computer use
6. HWE-Bench: Benchmarking LLM Agents on Real-World Hardware Bug Repair Tasks Coding / SWE

April 15, 2026 · 8

1. MARCA: A Checklist-Based Benchmark for Multilingual Web Search General
2. TREX: Automating LLM Fine-tuning via Agent-Driven Tree-based Exploration Multi-agent
3. ReviewGrounder: Improving Review Substantiveness with Rubric-Guided, Tool-Integrated Agents Multi-agent
4. HINTBench: Horizon-agent Intrinsic Non-attack Trajectory Benchmark General
5. GeoAgentBench: A Dynamic Execution Benchmark for Tool-Augmented Agents in Spatial Analysis Multimodal
6. AVID: A Benchmark for Omni-Modal Audio-Visual Inconsistency Understanding via Agent-Driven Construction General
7. RiskWebWorld: A Realistic Interactive Benchmark for GUI Agents in E-commerce Risk Management Work / office
8. MERRIN: A Benchmark for Multimodal Evidence Retrieval and Reasoning in Noisy Web Environments Multimodal

April 14, 2026 · 2

1. Frontier-Eng: Benchmarking Self-Evolving Agents on Real-World Engineering Tasks with Generative Optimization Coding / SWE
2. Policy-Invisible Violations in LLM-Based Agents General

April 13, 2026 · 10

1. Spatial Atlas: Compute-Grounded Reasoning for Spatial-Aware Research Agent Benchmarks Science & research
2. SIR-Bench: Evaluating Investigation Depth in Security Incident Response Agents General
3. PAC-BENCH: Evaluating Multi-Agent Collaboration under Privacy Constraints Multi-agent
4. PaperScope: A Multi-Modal Multi-Document Benchmark for Agentic Deep Research Across Massive Scientific Papers Science & research
5. BankerToolBench: Evaluating AI Agents in End-to-End Investment Banking Workflows Work / office
6. Evaluating LLM Agents on Automated Software Analysis Tasks General
7. CocoaBench: Evaluating Unified Digital Agents in the Wild Coding / SWE
8. EmbodiedGovBench: A Benchmark for Governance, Recovery, and Upgrade Safety in Embodied Agent Systems Safety
9. WebForge: Breaking the Realism-Reproducibility-Scalability Trilemma in Browser Agent Benchmark Web & browser
10. OccuBench: Evaluating AI Agents on Real-World Professional Tasks via Language Environment Simulation Work / office

April 12, 2026 · 3

1. The Blind Spot of Agent Safety: How Benign User Instructions Expose Critical Vulnerabilities in Computer-Use Agents GUI / computer use
2. Agent^2 RL-Bench: Can LLM Agents Engineer Agentic RL Post-Training? General
3. A Benchmark and Multi-Agent System for Instruction-driven Cinematic Video Compilation Multi-agent

April 11, 2026 · 3

1. STARS: Skill-Triggered Audit for Request-Conditioned Invocation Safety in Agent Systems Safety
2. The Amazing Agent Race: Strong Tool Users, Weak Navigators General
3. Taxonomy and Consistency Analysis of Safety Benchmarks for AI Agents Safety

April 10, 2026 · 8

1. HealthAdminBench: Evaluating Computer-Use Agents on Healthcare Administration Tasks GUI / computer use
2. COMPOSITE-Stem Multimodal
3. Pioneer Agent: Continual Improvement of Small Language Models in Production Coding / SWE
4. HiL-Bench (Human-in-Loop Benchmark): Do Agents Know When to Ask for Help? Coding / SWE
5. CORA: Conformal Risk-Controlled Agents for Safeguarded Mobile GUI Automation GUI / computer use
6. SEA-Eval: A Benchmark for Evaluating Self-Evolving Agents Beyond Episodic Assessment General
7. PilotBench: A Benchmark for General Aviation Agents with Safety Constraints Safety
8. Litmus (Re)Agent: A Benchmark and Agentic System for Predictive Evaluation of Multilingual Models General

April 9, 2026 · 5

1. ParseBench: A Document Parsing Benchmark for AI Agents GUI / computer use
2. ClawBench: Can AI Agents Complete Everyday Online Tasks? General
3. KnowU-Bench: Towards Interactive, Proactive, and Personalized Mobile Agent Evaluation Mobile
4. Aligning Agents via Planning: A Benchmark for Trajectory-Level Reward Modeling General
5. ACIArena: Toward Unified Evaluation for Agent Cascading Injection Multi-agent

April 8, 2026 · 4

1. GameWorld: Towards Standardized and Verifiable Evaluation of Multimodal Game Agents Multimodal
2. Where Did It Go Wrong? Process-Level Evaluation of Web Agents with Semantic State Tracking Web & browser
3. Diagon: A Programmable Testbed for AI-Agent Cognitive Labor Markets General
4. On Emotion-Sensitive Decision Making of Small Language Model Agents General

April 7, 2026 · 5

1. RealUserSim: Bridging the Reality Gap in Agent Benchmarking via Grounded User Simulation General
2. AgentCE-Bench: Agent Configurable Evaluation with Scalable Horizons and Controllable Difficulty under Lightweight Environments General
3. LUDOBENCH: Evaluating LLM Behavioural Decision-Making Through Spot-Based Board Game Scenarios in Ludo Multi-agent
4. EpiBench: Benchmarking Multi-turn Research Workflows for Multimodal Agents Multimodal
5. Market-Bench: Benchmarking Large Language Models on Economic and Trade Competition Multi-agent

April 6, 2026 · 5

1. ClawsBench: Evaluating Capability and Safety of LLM Productivity Agents in Simulated Workspaces Safety
2. GLANCE: A Global-Local Coordination Multi-Agent Framework for Music-Grounded Non-Linear Video Editing Multi-agent
3. Full-Duplex-Bench-v3: Benchmarking Tool Use for Full-Duplex Voice Agents Under Real-World Disfluency Tool use
4. CreativityBench: Evaluating Agent Creative Reasoning via Affordance-Based Tool Repurposing General
5. How Well Do Agentic Skills Work in the Wild: Benchmarking LLM Skill Usage in Realistic Settings General

April 5, 2026 · 5

1. SW-$A^2$-Bench: Benchmarking Autonomous Software Agent Generation for Agentic Web Coding / SWE
2. TimeSeek: Temporal Reliability of Agentic Forecasters General
3. LOCARD: An Agentic Framework for Blockchain Forensics General
4. ClawArena: Benchmarking AI Agents in Evolving Information Environments General
5. GeoBrowse: A Geolocation Benchmark for Agentic Tool Use with Expert-Annotated Reasoning Traces Tool use

April 4, 2026 · 1

1. Squeez: Task-Conditioned Tool-Output Pruning for Coding Agents Coding / SWE

April 3, 2026 · 7

1. BibTeX Citation Errors in Scientific Publishing Agents: Evaluation and Mitigation Science & research
2. A Systematic Security Evaluation of OpenClaw and Its Variants General
3. AgentHazard: A Benchmark for Evaluating Harmful Behavior in Computer-Use Agents GUI / computer use
4. OmniGUI: Benchmarking GUI Agents in Omni-Modal Smartphone Environments GUI / computer use
5. ESL-Bench: An Event-Driven Synthetic Longitudinal Benchmark for Health Agents General
6. NetAgentBench: A State-Centric Benchmark for Evaluating Agentic Network Configuration General
7. GBQA: A Game Benchmark for Evaluating LLMs as Quality Assurance Engineers Coding / SWE

April 2, 2026 · 5

1. ATBench: A Diverse and Realistic Agent Trajectory Benchmark for Safety Evaluation and Diagnosis Safety
2. MolClaw: An Autonomous Agent with Hierarchical Skills for Drug Molecule Evaluation, Screening, and Optimization General
3. Care-Conditioned Neuromodulation for Autonomy-Preserving Supportive Dialogue Agents Dialogue
4. REAP: Automatic Curation of Coding Agent Benchmarks from Interactive Production Usage Coding / SWE
5. ToolMisuseBench: An Offline Deterministic Benchmark for Tool Misuse and Recovery in Agentic Systems General

April 1, 2026 · 8

1. AgentSocialBench: Evaluating Privacy Risks in Human-Centered Agentic Social Networks Multi-agent
2. ClawSafety: "Safe" LLMs, Unsafe Agents Coding / SWE
3. HippoCamp: Benchmarking Contextual Agents on Personal Computers Multimodal
4. $\texttt{YC-Bench}$: Benchmarking AI Agents for Long-Term Planning and Consistent Execution General
5. Detecting Multi-Agent Collusion Through Multi-Agent Interpretability Multi-agent
6. When Users Change Their Mind: Evaluating Interruptible Agents in Long-Horizon Web Navigation Web & browser
7. Proactive Agent Research Environment: Simulating Active Users to Evaluate Proactive Assistants General
8. Do We Still Need GraphRAG? Benchmarking RAG and GraphRAG for Agentic Search Systems General

March 2026

March 31, 2026 · 7

1. Improvisational Games as a Benchmark for Social Intelligence of AI Agents: The Case of Connections General
2. LLM Agents Predict Social Media Reactions but Do Not Outperform Text Classifiers: Benchmarking Simulation Accuracy Using 120K+ Personas of 1511 Humans General
3. ATP-Bench: Towards Agentic Tool Planning for MLLM Interleaved Generation Multi-agent
4. ELT-Bench-Verified: Benchmark Quality Issues Underestimate AI Agent Capabilities General
5. PSPA-Bench: A Personalized Benchmark for Smartphone GUI Agent GUI / computer use
6. AEC-Bench: A Multimodal Benchmark for Agentic Systems in Architecture, Engineering, and Construction Multimodal
7. SciVisAgentBench: A Benchmark for Evaluating Scientific Data Analysis and Visualization Agents Science & research

March 30, 2026 · 4

1. Design Principles for the Construction of a Benchmark Evaluating Security Operation Capabilities of Multi-agent AI Systems Multi-agent
2. CirrusBench: Evaluating LLM-based Agents Beyond Correctness in Real-World Cloud Service Environments Dialogue
3. MiroEval: Benchmarking Multimodal Deep Research Agents in Process and Outcome Science & research
4. ViviDoc: Generating Interactive Documents through Human-Agent Collaboration Multi-agent

March 29, 2026 · 1

1. PRBench: End-to-end Paper Reproduction in Physics Research Coding / SWE

March 28, 2026 · 1

1. SkillTester: Benchmarking Utility and Security of Agent Skills General

March 27, 2026 · 3

1. Vision2Web: A Hierarchical Benchmark for Visual Website Development with Agent Verification Coding / SWE
2. RACE-Bench: A Reasoning-Augmented Benchmark for Repository-Level Code Agents on Feature Addition Coding / SWE
3. StressWeb: A Diagnostic Benchmark for Web Agent Robustness under Realistic Interaction Variability Web & browser

March 26, 2026 · 6

1. MemoryCD: Benchmarking Long-Context User Memory of LLM Agents for Lifelong Cross-Domain Personalization General
2. Doctorina MedBench: A Dialogue-Based Benchmark and Evaluation Framework for Agent-Based Medical AI Dialogue
3. Colon-Bench: An Agentic Workflow for Scalable Dense Lesion Annotation in Full-Procedure Colonoscopy Videos Multimodal
4. MolQuest: A Benchmark for Agentic Evaluation of Abductive Reasoning in Chemical Structure Elucidation General
5. WebTestBench: Evaluating Computer-Use Agents towards End-to-End Automated Web Testing GUI / computer use
6. FinMCP-Bench: Benchmarking LLM Agents for Real-World Financial Tool Use under the Model Context Protocol Tool use

March 25, 2026 · 4

1. SlopCodeBench: Benchmarking How Coding Agents Degrade Over Long-Horizon Iterative Tasks Coding / SWE
2. GameplayQA: A Benchmarking Framework for Decision-Dense POV-Synced Multi-Video Understanding of 3D Virtual Agents Multi-agent
3. BeliefShift: Benchmarking Temporal Belief Consistency and Opinion Drift in LLM Agents General
4. VehicleMemBench: An Executable Benchmark for Multi-User Long-Term Memory in In-Vehicle Agents Embodied / robotics

March 24, 2026 · 8

1. Efficient Benchmarking of AI Agents General
2. Can LLM Agents Be CFOs? Benchmarking Long-Horizon Resource Allocation in an Uncertain Enterprise Environment General
3. Code Review Agent Benchmark Coding / SWE
4. PERMA: Benchmarking Personalized Memory Agents via Event-Driven Preference and Realistic Task Environments General
5. PaperVoyager : Building Interactive Web with Visual Language Models General
6. Can LLM Agents Generate Real-World Evidence? Evaluating Observational Studies in Medical Databases General
7. LH-Bench: Skill-Grounded Evaluation of Long-Horizon Agents on Subjective Enterprise Tasks General
8. Benchmarking Multi-Agent LLM Architectures for Financial Document Processing: A Comparative Study of Orchestration Patterns, Cost-Accuracy Tradeoffs and Production Scaling Strategies Multi-agent

March 23, 2026 · 2

1. Ego2Web: A Web Agent Benchmark Grounded in Egocentric Videos Web & browser
2. CaP-X: A Framework for Benchmarking and Improving Coding Agents for Robot Manipulation Coding / SWE

March 22, 2026 · 1

1. WirelessBench: A Tolerance-Aware LLM Agent Benchmark for Wireless Network Intelligence General

March 21, 2026 · 1

1. Can AI Agents Answer Your Data Questions? A Benchmark for Data Agents General

March 20, 2026 · 2

1. LiveClawBench: Benchmarking LLM Agents on Complex, Real-World Assistant Tasks General
2. Skilled AI Agents for Embedded and IoT Systems Development General

March 19, 2026 · 2

1. Meanings and Measurements: Multi-Agent Probabilistic Grounding for Vision-Language Navigation Multi-agent
2. SODIUM: From Open Web Data to Queryable Databases Multi-agent

March 18, 2026 · 1

1. WebPII: Benchmarking Visual PII Detection for Computer-Use Agents GUI / computer use

March 17, 2026 · 4

1. CODMAS: A Dialectic Multi-Agent Collaborative Framework for Structured RTL Optimization Multi-agent
2. When the Specification Emerges: Benchmarking Faithfulness Loss in Long-Horizon Coding Agents Coding / SWE
3. RetailBench: Evaluating Long-Horizon Autonomous Decision-Making and Strategy Stability of LLM Agents in Realistic Retail Environments General
4. VisBrowse-Bench: Benchmarking Visual-Native Search for Multimodal Browsing Agents Multimodal

March 16, 2026 · 9

1. FormulaCode: Evaluating Agentic Optimization on Large Codebases Coding / SWE
2. CUBE: A Standard for Unifying Agent Benchmarks General
3. SWE-Skills-Bench: Do Agent Skills Actually Help in Real-World Software Engineering? Coding / SWE
4. CCTU: A Benchmark for Tool Use under Complex Constraints Tool use
5. GUI-CEval: A Hierarchical and Comprehensive Chinese Benchmark for Mobile GUI Agents GUI / computer use
6. VTC-Bench: Evaluating Agentic Multimodal Models via Compositional Visual Tool Chaining Multimodal
7. TelcoAgent-Bench: A Multilingual Benchmark for Telecom AI Agents General
8. Shopping Companion: Benchmarking and Training LLM Agents for Long-Horizon Preference-Grounded E-Commerce Tasks General
9. A Skill-augmented Agentic Framework and Benchmark for Multi-Video Understanding Multimodal

March 15, 2026 · 2

1. LongVidSearch: An Agentic Benchmark for Multi-hop Evidence Retrieval Planning in Long Videos General
2. AgentProcessBench: Diagnosing Step-Level Process Quality in Tool-Using Agents General

March 14, 2026 · 1

1. $τ$-Voice: Benchmarking Full-Duplex Voice Agents on Real-World Domains General

March 13, 2026 · 2

1. EnterpriseOps-Gym: Environments and Evaluations for Stateful Agentic Planning and Tool Use in Enterprise Settings Work / office
2. CTI-REALM: Benchmark to Evaluate Agent Performance on Security Detection Rule Generation Capabilities General

March 12, 2026 · 4

1. Strategic Navigation or Stochastic Search? How Agents and Humans Reason Over Document Collections Multimodal
2. HomeSafe-Bench: Evaluating Vision-Language Models on Unsafe Action Detection for Embodied Agents in Household Scenarios Embodied / robotics
3. You Told Me to Do It: Measuring Instructional Text-induced Private Data Leakage in LLM Agents GUI / computer use
4. When OpenClaw Meets Hospital: Toward an Agentic Operating System for Dynamic Clinical Workflows General

March 11, 2026 · 2

1. RewardHackingAgents: Benchmarking Evaluation Integrity for LLM ML-Engineering Agents General
2. Re-Evaluating EVMBench: Are AI Agents Ready for Smart Contract Security? General

March 10, 2026 · 4

1. CR-Bench: Evaluating the Real-World Utility of AI Code Review Agents Coding / SWE
2. MedMASLab: A Unified Orchestration Framework for Benchmarking Multimodal Medical Multi-Agent Systems Multi-agent
3. MA-EgoQA: Question Answering over Egocentric Videos from Multiple Embodied Agents Work / office
4. Real-Time Trust Verification for Safe Agentic Actions using TrustBench General

March 9, 2026 · 4

1. PostTrainBench: Can LLM Agents Automate LLM Post-Training? Coding / SWE
2. FinToolBench: Evaluating LLM Agents for Real-World Financial Tool Use Tool use
3. PIRA-Bench: A Transition from Reactive GUI Agents to GUI-based Proactive Intent Recommendation Agents GUI / computer use
4. \$OneMillion-Bench: How Far are Language Agents from Human Experts? Work / office

March 6, 2026 · 2

1. ResearchEnvBench: Benchmarking Agents on Environment Synthesis for Research Code Execution Coding / SWE
2. The World Won't Stay Still: Programmable Evolution for Agent Benchmarks General

March 5, 2026 · 4

1. TML-Bench: Benchmark for Data Science Agents on Tabular ML Tasks Coding / SWE
2. Tool-Genesis: A Task-Driven Tool Creation Benchmark for Self-Evolving Language Agent General
3. TimeWarp: Evaluating Web Agents by Revisiting the Past Web & browser
4. AgentSCOPE: Evaluating Contextual Privacy Across Agentic Workflows General

March 4, 2026 · 2

1. iAgentBench: Benchmarking Sensemaking Capabilities of Information-Seeking Agents on High-Traffic Topics General
2. AgentSelect: Benchmark for Narrative Query-to-Agent Recommendation General

March 3, 2026 · 2

1. Benchmarking Emergent Coordination in Large-Scale LLM Populations: An Evaluation Framework on the MoltBook Archive Multi-agent
2. LiveAgentBench: Comprehensive Benchmarking of Agentic Systems Across 104 Real-World Challenges General

March 2, 2026 · 5

1. TritonDFT: Automating DFT with a Multi-Agent Framework Multi-agent
2. Organizing, Orchestrating, and Benchmarking Agent Skills at Ecosystem Scale Coding / SWE
3. ZeroDayBench: Evaluating LLM Agents on Unseen Zero-Day Vulnerabilities for Cyberdefense Coding / SWE
4. LiveCultureBench: a Multi-Agent, Multi-Cultural Benchmark for Large Language Models in Dynamic Social Simulations Multi-agent
5. ASTRA-bench: Evaluating Tool-Use Agent Reasoning and Action Planning with Personal User Context Tool use

March 1, 2026 · 2

1. GroupGPT: A Token-efficient and Privacy-preserving Agentic Framework for Multi-User Chat Assistant Multimodal
2. Silo-Bench: A Scalable Environment for Evaluating Distributed Coordination in Multi-Agent LLM Systems Multi-agent

February 2026

February 28, 2026 · 5

1. MobiFlow: Real-World Mobile Agent Benchmarking through Trajectory Fusion Mobile
2. Theory of Code Space: Do Code Agents Understand Software Architecture? Coding / SWE
3. WirelessAgent++: Automated Agentic Workflow Design and Benchmarking for Wireless Networks General
4. Atomicity for Agents: Exposing, Exploiting, and Mitigating TOCTOU Vulnerabilities in Browser-Use Agents Web & browser
5. Cloud-OpsBench: A Reproducible Benchmark for Agentic Root Cause Analysis in Cloud Systems Multi-agent

February 27, 2026 · 3

1. TraderBench: How Robust Are AI Agents in Adversarial Capital Markets? General
2. HotelQuEST: Balancing Quality and Efficiency in Agentic Search General
3. ClarEval: A Benchmark for Evaluating Clarification Skills of Code Agents under Ambiguous Instructions Coding / SWE

February 26, 2026 · 4

1. OmniGAIA: Towards Native Omni-Modal AI Agents Multimodal
2. AMA-Bench: Evaluating Long-Horizon Memory for Agentic Applications Dialogue
3. SUPERGLASSES: Benchmarking Vision Language Models as Intelligent Agents for AI Smart Glasses Multimodal
4. MobilityBench: A Benchmark for Evaluating Route-Planning Agents in Real-World Mobility Scenarios General

February 24, 2026 · 2

1. A Benchmark for Deep Information Synthesis General
2. Turing Test on Screen: A Benchmark for Mobile GUI Agent Humanization GUI / computer use

February 23, 2026 · 3

1. Skill-Inject: Measuring Agent Vulnerability to Skill File Attacks Web & browser
2. ISO-Bench: Can Coding Agents Optimize Real-World Inference Workloads? Coding / SWE
3. CIBER: A Comprehensive Benchmark for Security Evaluation of Code Interpreter Agents General

February 22, 2026 · 3

1. AgenticRAGTracer: A Hop-Aware Benchmark for Diagnosing Multi-Step Retrieval Reasoning in Agentic RAG General
2. Evaluation and Benchmarking Suite for Financial Large Language Models and Agents General
3. Benchmark Test-Time Scaling of General LLM Agents General

February 20, 2026 · 2

1. EvoDiagram: Agentic Editable Diagram Creation via Design Expertise Evolution Multi-agent
2. Are LLMs Ready for Neural-integrated Mechanistic Modeling? A Benchmark and Agentic Framework General

February 19, 2026 · 1

1. Persona2Web: Benchmarking Personalized Web Agents for Contextual Reasoning with User History Web & browser

February 18, 2026 · 3

1. LLM4Cov: Execution-Aware Agentic Learning for High-coverage Testbench Generation General
2. MemoryArena: Benchmarking Agent Memory in Interdependent Multi-Session Agentic Tasks General
3. EnterpriseBench Corecraft: Training Generalizable Agents on High-Fidelity RL Environments Work / office

February 17, 2026 · 1

1. Lifelong Scalable Multi-Agent Realistic Testbed and A Comprehensive Study on Design Choices in Lifelong AGV Fleet Management Systems Multi-agent

February 16, 2026 · 3

1. OpaqueToolsBench: Learning Nuances of Tool Behavior Through Interaction General
2. ResearchGym: Evaluating Language Model Agents on Real-World AI Research Coding / SWE
3. Resp-Agent: An Agent-Based System for Multimodal Respiratory Sound Generation and Disease Diagnosis Multimodal

February 15, 2026 · 2

1. LongCLI-Bench: A Preliminary Benchmark and Study for Long-horizon Agentic Programming in Command-Line Interfaces Coding / SWE
2. AD-Bench: A Real-World, Trajectory-Aware Advertising Analytics Benchmark for LLM Agents General

February 13, 2026 · 9

1. Fix Before Search: Benchmarking Agentic Query Visual Pre-processing in Multimodal Retrieval-augmented Generation Multimodal
2. Unsafer in Many Turns: Benchmarking and Defending Multi-Turn Safety Risks in Tool-Using Agents Safety
3. TraceBack: Multi-Agent Decomposition for Fine-Grained Table Attribution Multi-agent
4. MoralityGym: A Benchmark for Evaluating Hierarchical Moral Alignment in Sequential Decision-Making Agents General
5. SciAgentGym: Benchmarking Multi-Step Scientific Tool-use in LLM Agents Science & research
6. BrowseComp-$V^3$: A Visual, Vertical, and Verifiable Benchmark for Multimodal Browsing Agents Multimodal
7. Benchmarking LLM Tool-Use in the Wild Tool use
8. SkillsBench: Benchmarking How Well Agent Skills Work Across Diverse Tasks General
9. AI Agents for Inventory Control: Human-LLM-OR Complementarity General

February 12, 2026 · 4

1. AgenticShop: Benchmarking Agentic Product Curation for Personalized Web Shopping General
2. Gaia2: Benchmarking LLM Agents on Dynamic and Asynchronous Environments General
3. AmbiBench: Benchmarking Mobile GUI Agents Beyond One-Shot Instructions in the Wild GUI / computer use
4. AgentLeak: A Benchmark for Internal-Channel Privacy Leakage in Multi-Agent LLM Systems Multi-agent

February 11, 2026 · 10

1. ReplicatorBench: Benchmarking LLM Agents for Replicability in Social and Behavioral Sciences General
2. AgentNoiseBench: Benchmarking Robustness of Tool-Using LLM Agents Under Noisy Condition General
3. Evaluating Memory Structure in LLM Agents General
4. FeatureBench: Benchmarking Agentic Coding for Complex Feature Development Coding / SWE
5. Agent-Diff: Benchmarking LLM Agents on Enterprise API Tasks via Code Execution with State-Diff-Based Evaluation General
6. See, Plan, Snap: Evaluating Multimodal GUI Agents in Scratch GUI / computer use
7. DeepImageSearch: Benchmarking Multimodal Agents for Context-Aware Image Retrieval in Visual Histories Multimodal
8. Locomo-Plus: Beyond-Factual Cognitive Memory Evaluation Framework for LLM Agents Dialogue
9. ISD-Agent-Bench: A Comprehensive Benchmark for Evaluating LLM-based Instructional Design Agents General
10. The Landscape of Prompt Injection Threats in LLM Agents: From Taxonomy to Analysis General

February 10, 2026 · 6

1. DECKBench: Benchmarking Multi-Agent Frameworks for Academic Slide Generation and Editing Multi-agent
2. KernelCraft: Benchmarking for Agentic Close-to-Metal Kernel Generation on Emerging Hardware Coding / SWE
3. SWE-Bench Mobile: Can Large Language Model Agents Develop Industry-Level Mobile Applications? Coding / SWE
4. SWE-AGI: Benchmarking Specification-Driven Software Construction with MoonBit in the Era of Autonomous Agents Coding / SWE
5. Beyond Closed-Pool Video Retrieval: A Benchmark and Agent Framework for Real-World Video Search and Moment Localization General
6. LingxiDiagBench: A Multi-Agent Framework for Benchmarking LLMs in Chinese Psychiatric Consultation and Diagnosis Multi-agent

February 9, 2026 · 2

1. From Assistant to Double Agent: Formalizing and Benchmarking Attacks on OpenClaw for Personalized Local AI Agent General
2. scBench: Evaluating AI Agents on Single-Cell RNA-seq Analysis Coding / SWE

February 8, 2026 · 2

1. CTFExplorer: Evaluating LLM Offensive Agents Through Multi-Target Web CTF Benchmarking Multi-agent
2. LOCA-bench: Benchmarking Language Agents Under Controllable and Extreme Context Growth General

February 6, 2026 · 2

1. VenusBench-Mobile: A Challenging and User-Centric Benchmark for Mobile GUI Agents with Capability Diagnostics GUI / computer use
2. AIRS-Bench: a Suite of Tasks for Frontier AI Research Science Agents General

February 5, 2026 · 6

1. SAGE: Benchmarking and Improving Retrieval for Deep Research Agents Science & research
2. ContextBench: A Benchmark for Context Retrieval in Coding Agents Coding / SWE
3. OdysseyArena: Benchmarking Large Language Models For Long-Horizon, Active and Inductive Interactions General
4. Beyond single-channel agentic benchmarking General
5. PieArena: Ranking and Profiling Language Agents in Realistic Negotiation Scenarios Multi-agent
6. TemporalBench: A Benchmark for Evaluating LLM-Based Agents on Contextual and Event-Informed Time Series Tasks General

February 4, 2026 · 3

1. LABBench2: An Improved Benchmark for AI Systems Performing Biology Research General
2. ProAgentBench: Evaluating LLM Agents for Proactive Assistance with Real-World Data Multimodal
3. What's in a Benchmark? The Case of SWE-Bench in Automated Program Repair Coding / SWE

February 3, 2026 · 6

1. MemGUI-Bench: Benchmarking Memory of Mobile GUI Agents in Dynamic Environments GUI / computer use
2. LPS-Bench: Benchmarking Safety Awareness of Computer-Use Agents in Long-Horizon Planning under Benign and Adversarial Scenarios GUI / computer use
3. TAME: A Trustworthy Test-Time Evolution of Agent Memory with Systematic Benchmarking General
4. FinMTM: A Multi-Turn Multimodal Benchmark for Financial Reasoning and Agent Evaluation Multimodal
5. Risky-Bench: Probing Agentic Safety Risks under Real-World Deployment Safety
6. Declarative by Design, Assistable Only by Convention: Benchmarking Multi-Agent Frameworks for AI-Assistability Multi-agent

February 2, 2026 · 6

1. FIRE-Bench: Evaluating AI Agents on the Rediscovery of Scientific Insights Science & research
2. Drift-Bench: Diagnosing Cooperative Breakdowns in LLM Agents under Input Faults via Multi-Turn Interaction General
3. OmniCode: A Benchmark for Evaluating Software Engineering Agents Coding / SWE
4. ES-MemEval: Benchmarking Conversational Agents on Personalized Long-Term Emotional Support Dialogue
5. TRIP-Bench: A Benchmark for Long-Horizon Interactive Agents in Real-World Scenarios General
6. ProjDevBench: Benchmarking AI Coding Agents on End-to-End Project Development Coding / SWE

February 1, 2026 · 1

1. Aggregation Queries over Unstructured Text: Benchmark and Agentic Method General

January 2026

January 31, 2026 · 3

1. MCP-Atlas: A Large-Scale Benchmark for Tool-Use Competency with Real MCP Servers Tool use
2. HumanStudy-Bench: Towards AI Agent Design for Participant Simulation General
3. Benchmarking Agents in Insurance Underwriting Environments General

January 30, 2026 · 2

1. DETOUR: An Interactive Benchmark for Dual-Agent Search and Reasoning General
2. BeSafe-Bench: Unveiling Behavioral Safety Risks of Situated Agents in Functional Environments Safety

January 29, 2026 · 4

1. World of Workflows: A Benchmark for Bringing World Models to Enterprise Systems Work / office
2. CAR-bench: Evaluating the Consistency and Limit-Awareness of LLM Agents under Real-World Uncertainty Dialogue
3. BioAgent Bench: An AI Agent Evaluation Suite for Bioinformatics General
4. From Digital to Physical: Digital Agents as Autonomous Coaches for Physical Intelligence Embodied / robotics

January 28, 2026 · 8

1. AgentLongBench: A Controllable Long Benchmark For Long-Contexts Agents via Environment Rollouts General
2. OS-Marathon: Benchmarking Computer-Use Agents on Long-Horizon Repetitive Tasks GUI / computer use
3. Agent Benchmarks Fail Public Sector Requirements General
4. AgentIF-OneDay: A Task-level Instruction-Following Benchmark for General AI Agents in Daily Scenarios Dialogue
5. MobileBench-OL: A Comprehensive Chinese Benchmark for Evaluating Mobile GUI Agents in Real-World Environment GUI / computer use
6. Less is More: Benchmarking LLM Based Recommendation Agents General
7. PredictionMarketBench: A SWE-bench-Style Framework for Backtesting Trading Agents on Prediction Markets Coding / SWE
8. IDE-Bench: Evaluating Large Language Models as IDE Agents on Real-World Software Engineering Tasks Coding / SWE

January 27, 2026 · 2

1. DevOps-Gym: Benchmarking AI Agents in Software DevOps Cycle Coding / SWE
2. Automated Safety Benchmarking: A Multi-agent Pipeline for LVLMs Multi-agent

January 26, 2026 · 4

1. TEA-Bench: A Systematic Benchmarking of Tool-enhanced Emotional Support Dialogue Agent Dialogue
2. GUIGuard-Bench: Toward a General Evaluation for Privacy-Preserving GUI Agents GUI / computer use
3. DeepPlanning: Benchmarking Long-Horizon Agentic Planning with Verifiable Constraints General
4. MalURLBench: A Benchmark Evaluating Agents' Vulnerabilities When Processing Web URLs Web & browser

January 25, 2026 · 2

1. When Personalization Legitimizes Risks: Uncovering Safety Vulnerabilities in Personalized Dialogue Agents Safety
2. EntWorld: A Holistic Environment and Benchmark for Verifiable Enterprise GUI Agents Work / office

January 23, 2026 · 1

1. EMemBench: Interactive Benchmarking of Episodic Memory for VLM Agents Multimodal

January 20, 2026 · 3

1. APEX-Agents General
2. LifeAgentBench: A Multi-dimensional Benchmark and Agent for Personal Health Assistants in Digital Health General
3. OP-Bench: Benchmarking Over-Personalization for Memory-Augmented Personalized Conversational Agents Dialogue

January 19, 2026 · 2

1. CooperBench: Why Coding Agents Cannot be Your Teammates Yet Coding / SWE
2. MedConsultBench: A Full-Cycle, Fine-Grained, Process-Aware Benchmark for Medical Consultation Agents General

January 18, 2026 · 1

1. MMDeepResearch-Bench: A Benchmark for Multimodal Deep Research Agents Science & research

January 17, 2026 · 3

1. PEARL: Self-Evolving Assistant for Time Management with Reinforcement Learning General
2. Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces Web & browser
3. ATOD: An Evaluation Framework and Benchmark for Agentic Task-Oriented Dialogue Systems Dialogue

January 16, 2026 · 3

1. AstroReason-Bench: Evaluating Unified Agentic Planning across Heterogeneous Space Planning Problems General
2. ABC-Bench: Benchmarking Agentic Backend Coding in Real-World Development Coding / SWE
3. AgencyBench: Benchmarking the Frontiers of Autonomous Agents in 1M-Token Real-World Contexts General

January 15, 2026 · 2

1. DR-Arena: an Automated Evaluation Framework for Deep Research Agents Science & research
2. OctoBench: Benchmarking Scaffold-Aware Instruction Following in Repository-Grounded Agentic Coding Coding / SWE

January 14, 2026 · 1

1. AI-NativeBench: An Open-Source White-Box Agentic Benchmark Suite for AI-Native Systems Coding / SWE

January 13, 2026 · 9

1. ART: Action-based Reasoning Task Benchmarking for Medical AI Agents General
2. Advancing ESG Intelligence: An Expert-level Agent and Comprehensive Benchmark for Sustainable Finance Multi-agent
3. DeepResearch Bench II: Diagnosing Deep Research Agents via Rubrics from Expert Report Science & research
4. M3-BENCH: Process-Aware Evaluation of LLM Agents' Social Behaviors in Mixed-Motive Games General
5. Mem2ActBench: A Benchmark for Evaluating Long-Term Memory Utilization in Task-Oriented Autonomous Agents General
6. MPCI-Bench: A Benchmark for Multimodal Pairwise Contextual Integrity Evaluation of Language Model Agents Multimodal
7. Evaluating Implicit Regulatory Compliance in LLM Tool Invocation via Logic-Guided Synthesis General
8. The Agent's First Day: Benchmarking Learning, Exploration, and Scheduling in the Workplace Scenarios Work / office
9. MirrorBench: A Benchmark to Evaluate Conversational User-Proxy Agents for Human-Likeness General

January 12, 2026 · 2

1. Proof of Time: A Benchmark for Evaluating Scientific Idea Judgments Science & research
2. FOCAL: A Novel Benchmarking Technique for Multi-modal Agents General

January 11, 2026 · 2

1. Sola-Visibility-ISPM: Benchmarking Agentic AI for Identity Security Posture Management Visibility General
2. AgentHallu: Benchmarking Automated Hallucination Attribution of LLM-based Agents General

January 10, 2026 · 2

1. One Interaction Is Worth a Thousand Guesses: Benchmarking the Interactive Capabilities of Deep Research Agents Science & research
2. SafePro: Evaluating the Safety of Professional-Level AI Agents Work / office

January 9, 2026 · 3

1. Beyond BeautifulSoup: Benchmarking LLM-Powered Web Scraping for Everyday Users Web & browser
2. TowerMind: A Tower Defence Game Learning Environment and Benchmark for LLM as Agents Multimodal
3. FinVault: Benchmarking Financial Agent Safety in Execution-Grounded Environments Safety

January 8, 2026 · 2

1. Arabic Prompts with English Tools: A Benchmark General
2. FinDeepForecast: A Live Multi-Agent System for Benchmarking Deep Research Agents in Financial Forecasting Multi-agent

January 7, 2026 · 3

1. From Laboratory to Real-World Applications: Benchmarking Agentic Code Reasoning at the Repository Level Coding / SWE
2. DeepSynth-Eval: Objectively Evaluating Information Consolidation in Deep Survey Writing General
3. Mem-Gallery: Benchmarking Multimodal Long-Term Conversational Memory for MLLM Agents Multimodal

January 6, 2026 · 2

1. SastBench: A Benchmark for Testing Agentic SAST Triage General
2. M3MAD-Bench: Multi-Dimensional Evaluation of Multi-Agent Debate Across Domains and Modalities Multi-agent

January 5, 2026 · 2

1. LongDA: Benchmarking LLM Agents for Long-Document Data Analysis General
2. Project Ariadne: A Structural Causal Framework for Auditing Faithfulness in LLM Agents General

January 4, 2026 · 1

1. KGCE: Knowledge-Augmented Dual-Graph Evaluator for Cross-Platform Educational Agent Benchmarking with Multimodal Language Models Multimodal

January 3, 2026 · 1

1. ReliabilityBench: Evaluating LLM Agent Reliability Under Production-Like Stress Conditions General

January 2, 2026 · 2

1. FinRetrieval: A Benchmark for Financial Data Retrieval by AI Agents General
2. Beyond IVR: Benchmarking Customer Support LLM Agents for Business-Adherence Dialogue

January 1, 2026 · 2

1. $α^3$-Bench: A Unified Benchmark of Safety, Robustness, and Efficiency for LLM-Based UAV Agents over 6G Networks Safety
2. FlashInfer-Bench: Building the Virtuous Cycle for AI-driven LLM Systems General