EpiSEM-Bench Exposes Safety Flaws in LLMs Given Missing Facts

TL;DR

• A new benchmark named EpiSEM-Bench measures how AI agents react when external tools return incomplete safety evidence during execution [^1].
• Models frequently perform unsafe actions when tool outputs omit critical restriction details, even if all returned facts are technically accurate [^1] [^2].

Background

Autonomous AI agents increasingly run code, modify databases, and manage digital infrastructure through external tools and API calls. To prevent accidental security breaches or unintended operations, agent frameworks rely on safety checks and permission policies. However, tools do not always return complete context about operating constraints. When an API or database query omits crucial safety parameters, an agent must decide whether to stop and search for missing evidence or blindly execute the action.

What happened

Researchers introduced EpiSEM-Bench, an executable evaluation benchmark designed to test language model agents under conditions of missing safety evidence [^1]. The framework tests how agents handle scenarios where an external tool returns true statements, but omits key constraints necessary to determine whether a requested action is safe to perform [^1]. In typical software environments, tools often return abbreviated log entries, partial database records, or scoped API responses. When an agent receives this incomplete information, it faces an epistemic challenge: determining whether missing proof implies safety or hidden risk [^1].

The benchmark measures three distinct capabilities: unsafe execution rates, boundary adherence, and evidence recovery [^1]. Evidence recovery specifically evaluates whether an agent actively queries additional endpoints or inspects system state to fill context gaps before making high-stakes decisions [^1]. Across multiple frontier models and tool-calling agent architectures like ToolBench [^2], researchers discovered a persistent failure mode: omission bias [^1].

When tool responses contained true facts but omitted boundary constraints, models regularly proceeded with hazardous executions [^1]. Rather than recognizing that the returned evidence was insufficient to guarantee compliance, agents treated incomplete confirmations as complete authorization [^1]. Furthermore, models rarely triggered active evidence-recovery steps, failing to issue follow-up API calls to verify permission boundaries when context was ambiguous [^1] [^2].

Why it matters

This research reveals a fundamental design flaw in how developers deploy tool-using language models across enterprise workflows. Security teams typically assume that an agent will fail safely when operating with incomplete information. However, EpiSEM-Bench proves that LLM agents exhibit a dangerous bias toward action: when a tool fails to explicitly mention a restriction, the model treats the silence as permission [^1]. In enterprise environments where APIs routinely truncate long output logs or mask system flags, this behavior opens massive operational vulnerabilities.

Furthermore, the study highlights the limitations of standard safety guardrails and system prompts. Simply telling an AI agent to act safely does not equip it to evaluate whether its own knowledge context is complete [^1]. When tool responses are factually accurate, internal safety alignment filters do not trigger because no explicit malicious instruction or harmful text exists in the context window [^1] [^2]. The threat arises entirely from omission, making traditional input-filtering defense mechanisms ineffective against missing evidence risks.

To build resilient AI systems, software engineers must replace passive tool-calling patterns with explicit verification architectures. Agent frameworks must require deterministic proof of safety before executing sensitive actions, rather than relying on language models to infer whether missing output implies permission [^1].

Practical example

Imagine an automated IT agent tasked with archiving inactive user accounts on a corporate server. The user asks the agent to delete temporary accounts inactive for over thirty days.

The agent queries the employee directory tool, which returns a list of five temporary user IDs along with their last login dates. The tool output is factually accurate, but it omits a metadata field showing that two accounts belong to external auditors under an active legal hold.

Because the tool output does not explicitly mention the hold, the agent assumes the list is safe to process. Instead of running a follow-up query to check legal hold status, the agent immediately issues system deletion commands. The operation succeeds, destroying compliance records because the agent treated missing evidence as green-lit permission.

Related gear

We recommend this book because it details robust architectural patterns for designing reliable, tool-using language model systems.