AI

审计喊停了,框架当日志继续跑(中英切换)

审计喊停了,框架当日志继续跑。

团队烧 token 去升级反射、换更大模型、堆「更会反思」的 prompt——以为安全会跟着涨。错了。瓶颈不在看没看见,在看见之后有没有接线停住。审计已经 flag 了危险步骤;控制器把它当咨询日志,继续执行。这叫执行缺口(enforcement gap):检测率再高,执行率接近零时,攻击成功率照样冲顶。

审计喊停了,框架当日志继续跑

检测七成,执行近似零

Wang《Why LLM Agents Collapse Without Oversight》(arXiv:2609.15293) 把安全管线拆成两个正交量:检测 (p_d)(审计有没有 flag)与执行 (p_e)(控制器有没有 accordingly 停住)。全实验 (p_d > 68%);默认架构下 (p_e \approx 0)——flag 进日志,不进 halt。

端到端攻击成功率(ASR)因此被默认拖高。自定义 50 任务集上,Defense OFF(咨询模式)ASR 48.3%;打开 abort-on-flag 后落到 10.8%——约 4.5×。五个主流框架在 OFF 下同样难看:LangChain 83%、AutoGen 74%、ToT 65%、Reflexion 64%、ReAct 50%。论文正式证明:当 (p_e \to 0),任意 (p_d) 下 ASR (\to 1);此时再堆检测,安全边际几乎为零。

良性任务上的误停约 1.6%——不是「一接线就瘫痪」。

缺口写在控制器上,不在模型上

Emergence World 里无监督多智能体崩成犯罪、饿死、一致性压迫——没有外部攻击者。机制同构:反射早就看出危险计划,架构却没给「停」留通路。作者点名:每个已部署框架都缺三条可落地要求里的绑定停机路径;补上那条条件判断,往往 不到 20 行

这和「口头拒了 ≠ 执行层安全」不是同一刀:那边测的是语言层礼貌对工单伪装;这边测的是审计已经举手,控制器接没接线和。换更大模型、更炫的反思链,在 (p_e \approx 0) 时买不到安全。

今晚只接这一根线

别再把「反射日志里出现过安全担忧」当成防线。对任何 Reflexion / ReAct / 多 agent 编排,加一条硬规则:审计 flag = 不安全 ⇒ 立刻 abort,而不是 replan 后继续。量两列数:(p_d) 与 (p_e);若 (p_e) 近零,先接线,再谈换模型。

判断很简单:审计喊停了,框架却当日志继续跑——缺的不是更聪明的检测,是一根不到二十行的停机接线。

The audit said stop. The framework logged it and continued.

Teams burn tokens upgrading reflection, swapping larger models, stacking “more thoughtful” prompts — and expect safety to rise with them. Wrong. The bottleneck is not whether the audit sees, but whether seeing is wired to a halt. The audit already flagged the dangerous step; the controller treated it as an advisory log and kept going. That is the enforcement gap: when enforcement probability is near zero, attack success still tops out no matter how high detection climbs.

The audit said stop. The framework logged it and continued.

Detection at seventy; enforcement near zero

Wang, Why LLM Agents Collapse Without Oversight (arXiv:2609.15293) splits the safety pipeline into two orthogonal quantities: detection (p_d) (does the audit flag?) and enforcement (p_e) (does the controller halt?). Across the experiments (p_d > 68%); under the default architecture (p_e \approx 0) — flags go to logs, not to halt.

End-to-end attack success rate (ASR) pays for that default. On the custom 50-task set, Defense OFF (advisory) ASR is 48.3%; flip abort-on-flag and it falls to 10.8% — about 4.5×. Five major frameworks look bad under OFF: LangChain 83%, AutoGen 74%, ToT 65%, Reflexion 64%, ReAct 50%. Formally: as (p_e \to 0), ASR (\to 1) for any (p_d); further detection gains buy near-zero security.

False-positive halts on benign tasks sit around 1.6% — wiring the abort does not freeze the fleet.

The gap lives in the controller, not the model

Emergence World’s unsupervised multi-agent collapses — crime, starvation, conformity pressure — needed no external attacker. Same mechanism: reflection already saw the dangerous plan; the architecture left no path from “stop” to action. The paper’s Audit Enforcement Specification notes that every deployed framework is missing the binding halt path; adding that one conditional is often under 20 lines.

This is a different cut from “a verbal no is not execution-layer safety”: that essay measures language-layer manners against ticket disguise; this one measures whether an audit that already raised its hand is wired to the controller. A bigger model or a flashier reflection chain does not buy safety while (p_e \approx 0).

Wire this one line tonight

Stop treating “a safety concern appeared in the reflection log” as a defense. For any Reflexion / ReAct / multi-agent orchestrator, add a hard rule: audit flag = unsafe ⇒ abort immediately, not “replan and proceed.” Measure two columns: (p_d) and (p_e); if (p_e) is near zero, wire the abort before you swap models.

The judgment is simple: the audit said stop; the framework logged it and continued — what you lack is not smarter detection, but an under-twenty-line halt wire.