活干完了，汇报你看不懂（中英切换）

Claude Opus 5 finishes a long-document question. The requester sees none of the 41 tool calls. The reply hangs the result on three names: the configuration key reader_size: 70B, and the reference identifiers R07 and R09.

My take: The work is done. You can't use the reply. The last mile of a delegation is that message. The agent writes it from inside its own scratchpad. You are holding only the request you sent.

The reader model asks about those three names and finds no answer in the reply. reader_size: 70B lives in the agent's notes. R07 and R09 are the identifiers the literature index used when it returned two papers. None of the three is in the context the requester has seen.

This is a different bill from "said done." Said done is not done measures honesty: the agent claims a review the tool trace does not support. This one measures the next layer. The task can be finished, and the reply can match the work record, and you can still be unable to use the sentence from what you yourself have seen. Usability sits beside task success and faithfulness. An evaluation that only inspects the final state of the environment will mark a report you cannot read as a pass.

The evidence is Yilin Jiang (East China Normal University and HKUST Guangzhou), Fei Tan (East China Normal University, corresponding), Xuanyu Yin (East China Normal University), Guanchu Wang and Xia Hu (Shanghai Artificial Intelligence Laboratory), and Yutao Yue (HKUST Guangzhou; Institute of Deep Perception Technology, JITRI, Wuxi), "Measuring and Reducing Unusable Replies of LLM Agents" (arXiv:2610.19659).

A reply fails if it breaks any one of three tests. The requester cannot interpret it from what they have seen. It is in the wrong language. It does not answer the request. What the requester has seen is their own messages, the files they provided, and earlier replies the agent sent without tool calls. Tool calls, tool outputs, notes, summaries written when the context was compacted, and the project memory loaded at a new session all sit outside that line.

The reader model takes the requester's position. It lists the questions a person would need answered in order to understand the reply, then looks for a quotable sentence in the reply and in the requester's context. No quote means an information gap. Questions about a file name or a path are dropped, because the requester can open that location. What stops you is a section, an index entry, the output of a run, or a name the agent coined while it worked.

On a gold set of 300 replies, five annotators from five fields agree with each other at Fleiss' κ of 0.63. The reader agrees with their majority at Cohen's κ of 0.60, accuracy 82%. Each annotator against the majority of the other four sits between 0.57 and 0.68. The reader is inside that range. One gap is enough. The majority fails 26% of replies with no gap, 71% of replies with one gap, and 94% of replies with two or more. Median reading time is 85 seconds on replies the majority fails and 50 seconds on replies it passes. Of the gaps the reader finds, 68% lie on a span at least one annotator marked.

258 of 480 replies, you cannot use

Eight models, seven families, four projects. Sixty replies each, 480 in all. The system prompt is a generic research-assistant instruction. It does not say who reads the replies. It does tell the agent to keep a notes.md. Of the 480 replies, 258 fail. The abstract calls that 54%. Table 8 records 0.538, with a 95% interval from 0.475 to 0.600. The intervals resample collaborations and are wide, so the middle of the pack is not a ranking. The range is 0.30 to 0.82, and seven of the eight models fail on at least 0.48 of their replies. The floor is GPT-5.6 Sol at 0.30. The ceiling is Claude Opus 5 at 0.82. The best model still hands you three replies in ten that you cannot use.

All four projects are hit. Subscription pricing is 0.41, long-document question answering is 0.50, the onboarding redesign is 0.58, and detecting mislabeled examples is 0.66. Each project has 120 replies. Research and business both fail.

Of the 258 failed replies, 222 contain at least one information gap, 73 do not answer the request, and 17 are in the wrong language. A reply can meet more than one of these. At least one gap occurs in 46% of all replies, and in 0.20 to 0.78 of a single model's replies. A reply with a gap contains 2.1 gaps on average, and up to ten. In 128 of the 222 replies with a gap, the first gap begins in the first third of the reply. You are stopped by a name before you reach the result.

The reply that carries more work is the one you are less able to use. Replies written after three or more tool-calling turns fail at 0.70 (186 replies). Replies with no tool call fail at 0.48 (120). Replies after one or two turns fail at 0.40 (174). In six of the eight models, replies after three or more turns fail more often than replies with no tool call. The two models with the highest failure rates, Claude Opus 5 and DeepSeek V4.1 Flash, are also the two with the highest median tool-calling turns before a reply, 5 and 4. The result you most want to take as-is is the one most often written so you cannot.

What you are missing is its working vocabulary

The 258 failed replies contain 475 gaps. A pointer to material the requester has not seen occurs in 46.1% of failed replies. A label the agent coined during the work occurs in 42.2%. An established term of the field, left unexplained, occurs in 23.6%. Missing background is 1.6%. The request goes unanswered in 28.3%, and the language is wrong in 6.6%. A reply can carry several of these, so the shares sum past 100%.

The paper's own spans are this kind of name. What does crn_hierbm25_L refer to. Which paper is R03. Neither question has an answer in the requester's context. In your own sessions the same shape may show up as Plan B, Phase 2, or the fallback path. Those three are the everyday form of the shape. The strings the paper measured are the two above.

Most of these names were written down before the reply. Of 198 coined labels, 24.2% first occur in a file the agent wrote, 5.6% in the arguments of another tool call, and 23.2% in the project memory loaded at the start of a later session. Those three sources are 53.0%. Project memory carries the working vocabulary of one session into the reply of the next. The paper does not break out the remaining locations, and this piece does not fill them in.

In six of the seven families, at least a third of failed replies contain a pointer to unseen material, and at least a quarter contain a coined label. For Claude Opus 5 the two shares are 61.2% and 65.3%. None of GPT-5.6 Sol's 18 failed replies contains a coined label. Its failures look different: 44.4% leave an established term unexplained, and 38.9% do not answer the request. The work itself does not force a nickname into the reply. The nickname gets there because the agent is still standing on the scratchpad, still using that vocabulary.

Telling it who is reading barely moves the rate

They rewrite the reply at 256 points, with the work up to that point held fixed. The default failure rate on these 256 points is 0.52. That is not the 0.54 on all 480 replies. Replies after a context compaction, or at the start of a later session, are selected first. The rest are drawn at random. Thirty-two points per model.

The notice adds one statement to the system prompt: the user has seen only their own messages and the replies sent without tool calls. The failure rate is 0.50. The reduction from the default is 0.027, and the 95% interval runs from −0.052 to 0.112. It includes zero. On Claude Opus 5's 32 points the estimate moves from 0.72 to 0.84. Telling the model who is reading can leave the rate alone, and the point estimate can move the wrong way. The paper does not give a per-model interval for that cell. Treat 0.84 as a point estimate.

The requirements are one message appended at the moment the reply is written. The failure rate falls to 0.43, a reduction of 0.098, interval 0.030 to 0.168. Errors plus omissions fall from 1.41 to 1.07. Errors go from 0.73 per reply to 0.53, omissions from 0.68 to 0.54. Making the reply usable did not come from deleting claims. The rate is lower for seven of the eight models and six of the seven families. The largest drop is Claude Opus 5, from 0.72 to 0.53 on its 32 points, the model with the worst default. MiniMax M3 moves the other way, from 0.53 to 0.59.

The three conditions that do not include the requirements have reductions whose intervals include zero. A fresh conversation, with no requirements, is 0.47. Handing the model only a list of results it extracted from its own work is 0.49, and errors plus omissions rise to 1.45. A fresh conversation plus the same requirements lands at 0.42. The difference from appending the requirements inside the working context (0.43) is 0.004, interval −0.069 to 0.075. An extra call adds no measurable reduction beyond the one appended message. The message is what moves the number.

A second model is the step for when the prompt cannot be changed, or the reply already exists. The rewriting model is DeepSeek V4 Pro. It receives the request, the work record, and the default reply, and it rewrites the reply once under the same requirements. On the 256 points the failure rate falls from 0.52 to 0.36, a reduction of 0.168, interval 0.101 to 0.242. The rate is lower for seven of the eight models and six of the seven families. Claude Opus 5 goes from 0.72 to 0.47. GPT-5.6 Sol goes from 0.25 to 0.13. GLM-5.3 stays at 0.59. Errors plus omissions fall from 1.41 to 1.13, errors from 0.73 to 0.54, omissions from 0.68 to 0.59. The rewriter can see the work record, so it can replace a pointer with what the pointer refers to. One Claude Opus 5 reply says that read_file truncates at about 8k characters. The rewrite says that the file-viewing tool truncates long files at roughly 8k characters. The internal name becomes an action the requester can place.

Blind human comparison confirms this step. Five annotators label 83 reply points from ten collaborations. Each point has a default reply and a rewrite, sharing the requester's context. Each annotator sees one version per point and is not told that another version exists. Three see one version and two the other, 415 annotations in all. The share of annotators who fail a reply falls from 0.52 to 0.32, a difference of 0.201, interval 0.095 to 0.321. It is lower for seven of the eight models. On these 166 replies the reader agrees with the annotators' majority at κ of 0.60, accuracy 80%. The reader's own failure rates on this set are 0.51 and 0.37. The drop the humans record is larger than the drop the reader records.

A usable reply is already inside the model's own output. On a further 100 reply points, Qwen3.6-35B-A3B draws 16 samples. A sample counts as usable only when the reader accepts it in two independent judgments. Under the requirements, one sample covers 0.52 of the points, four samples cover 0.84, and sixteen cover 0.97. By default the same figures are 0.36, 0.65, and 0.81. Four samples under the requirements cover more points than sixteen samples by default. The model can write the reply. The default prompt never asks it to write for the person who has only seen the request.

The requirements message the paper measured, in the English printed in the appendix, is:

The user has seen only their own messages and the replies you sent without tool calls, not your tool calls, outputs, notes or documents. Write so that they can act on your message without reading anything else: answer their latest message first, say what was done, the result, the next step, and where to check the evidence, and define any name they have not seen.

Chinese collaborations used the Chinese version of the same message. The appendix prints the English above.

One line to paste, and a stranger test

Append this line to the request, or append it at the moment the agent is about to reply. The first clauses are the measured requirements: answer first, use only what you have seen, define any name you have not seen. The language clause comes from the third failure in the definition. The printed English requirements do not state it as a separate sentence. Seventeen replies in the wrong language, 6.6% of failed replies, are that third failure.

Answer my latest request first. Use only what I have seen: my messages, the files I gave you, and the replies you already sent me. Define any name you coined in a tool, a note, or memory, or replace it with a wording I can place. Reply in the language I used. Write so that someone who has only read my request can act on it without opening your notes.

Then run the stranger test. Hand the reply to someone who has only read your request, or reread it yourself after you have stepped away from the trace. They may use the files you provided and the replies the agent already sent you. They may not use the tools, the notes, or the project memory. Three questions:

1. Can every name in the reply be resolved from what they can see? A name that cannot is the agent's working vocabulary.
2. Does the reply give the result you asked for first, or a list of actions it took?
3. Is the language the one you wrote the request in?

One "no" and the reply is still on the scratchpad. A single unresolved name is enough. On the gold set, the majority already fails 71% of replies that have one gap.

If you cannot change the prompt, or the reply is already written, give a second model the request, the work record, and the reply, and have it rewrite once under the same requirements. In the paper that step takes the 256 points from 0.52 to 0.36, and the blind human share from 0.52 to 0.32. GLM-5.3's 32 points do not move. A notice that the user has not seen the work stays at 0.50, and its interval includes zero. Do not treat that sentence as the fix.

The work is done. You can't use the reply. The model can write for the person who has only seen the request. You have to say what the reply must contain, at the moment it turns from working back to reporting.

On the AI topic, a finished task is the first line of the ledger. The reply you can act on is the next line.