AI

陌生人的 PR 有人审,Agent 的反而没人看(中英切换)

想象两个 PR 同时出现在你的开源仓库里。都来自没有写权限的陌生账号,都改了几个业务文件,顺手动了一下 .github/workflows/ 里的一个 yml。唯一的区别是:一个是人写的,另一个的描述里写着“Generated by Codex”,或者作者头像是 Copilot。

你会更仔细地看哪一个?

大多数人的直觉是:当然是 Agent 那个,机器写的更不放心。数据说的正好相反。

我的判断:我们对 Agent 的 PR,审得比对陌生人还松。最该看的那一类,也就是连 CI 一起改的 PR,恰恰最容易没人看就合进去,而它跑出来的绿勾,是由它刚改过的那条流水线发的。这不是 Agent 的问题,是维护者把“Agent 写的”默认读成了“机器生成的杂活”。

证据来自 Mohammad Wathiq Soualhi(独立研究者)和 Basem Suleiman(新南威尔士大学)的 《Co-Modifying Application Code and Deployment Infrastructure: A Causal Study of AI Agent Pull Requests》(arXiv:2610.12849,2026-10-09)。他们从两个公开数据集(AIDev v5 和 MOSAIC-3M)里取了 26,999 个 PR,全部来自 100 星以上的 GitHub 仓库,创建于 2025 年 6 到 7 月,覆盖 Codex、Claude Code、Copilot、Devin、Cursor、Jules 六种 Agent,外加一组人写的对照。

同样是陌生人,署名 Agent 就少审

最干净的一组比较是这样的:只看没有任何写权限的外部贡献者,一共 2,232 个 PR。这些人跟仓库没有组织关系,按理说维护者对他们一视同仁地警惕。

结果,人写的 PR 有 82.7% 得到了外部审查;Agent 写的只有 63.6%。控制住改动文件数、代码行数、有没有带测试、仓库星数之后,署名 Agent 让 PR 被人审的几率(odds)降低了 63.7%(OR = 0.363,95% CI 0.293–0.451)。

同样是陌生人,署名不同

这一组专门排除了“维护者自己用 Agent 给自己提 PR,所以不审”的解释。作者的结论很直白:审不审,跟着署名走,不跟补丁复杂度走,也不跟权限走。

最该审的那一类,最容易漏

论文管“同时改业务代码和部署基础设施”的 PR 叫双域 PR(dual-scope)。它们只占 Agent PR 的 4.5% 左右,但里面 七成以上动的是 CI/CD 流水线;只改一个基础设施文件的时候,绝大多数改的就是 CI 配置。

人写的 PR,加不加 CI 改动,被正式审查的比例几乎不变:72.2% 对 73.4%。Agent 写的就飘了:在 MOSAIC-3M 里,从只改代码时的 52.5% 掉到连 CI 一起改时的 36.2%;在 AIDev v5 里反而略升,47.5% 到 53.1%。两个数据集方向不一致,MOSAIC 这组的 p 值是 0.052,样本也只有 80 对,不能当铁证。

但下面这个对比在同一个数据集里,很难绕过去:连 CI 一起改的 PR,人写的有 19.0% 没人评论就合了;Agent 写的,56.2%。 放到两个数据集一起看,这类 Agent PR 有 43% 到 56% 零外部评论、零审查就合并,中位耗时 26 到 31 分钟。

绿勾是它自己发的

为什么偏偏是 CI 改动最要命?因为它让“测试通过”这件事失去了意义。

绿勾是它自己发的

一个 PR 改了 .github/workflows/test.yml,CI 就按改过之后的那份配置去跑。它可以少跑一个测试套件,可以把某一步改成 continue-on-error,可以把 lint 的路径缩小。然后它全绿了。论文把这叫做循环验证失效:被检查的东西就是检查规则本身,绿勾提供的独立保证是零。CI 安全领域早就给这个起了名字,叫投毒流水线执行(Poisoned Pipeline Execution):不受信任的贡献改写 workflow 定义,在构建机里执行任意代码。

把这两件事叠在一起:一个外部账号,用 Agent 提了一个顺手改 CI 的 PR,维护者看到“又是 Agent 的 PR”,看一眼绿勾,半小时内合了。这条路径不需要任何人坏心眼,只需要大家都很忙。

我猜,是杂活把人训练成了只看绿勾

论文没有解释维护者为什么对 Agent 更松,下面是我的推测,不是论文结论。

同一篇论文里有个数字:Agent 的 PR 里有 17.7% 到 25.4% 是纯杂活,不碰业务代码也不碰基础设施,只改文档、测试或 lockfile;在 AIDev 的杂活 PR 里,README.md 一个文件就出现在 44.7% 的 PR 里。维护者每天被几十个“Agent 帮你更新了 README”“Agent 升了一个依赖”淹没,很自然地学会了一种处理方式:是 Agent 的,扫一眼,绿了就合。

问题是,这个习惯是在低风险的 PR 上练出来的,却被原样用在了改流水线的 PR 上。审查的力度应该跟着改了哪些路径走,现在却跟着谁署的名走。

能马上做的一步

第一步:查一下你的仓库已经合进去了多少“改了 CI、却没人审”的 PR。 装好 gh 和 jq,在仓库目录里跑:

gh pr list --state merged --limit 300 \
  --json number,title,author,mergedAt,files,reviews \
| jq -r '.[]
  | select(any(.files[]; .path | test("^\\.github/workflows/|(^|/)Dockerfile$|\\.tf$|^(terraform|k8s|deploy)/")))
  | .author.login as $a
  | select([.reviews[] | select(.author.login != $a)] | length == 0)
  | "#\(.number)\t\($a)\t\(.mergedAt[:10])\t\(.title)"'

它会列出最近 300 个已合并 PR 里,动过 workflow、Dockerfile 或 Terraform/k8s/部署目录,却没有任何非作者审查记录的那些。路径正则按你的仓库改。

第二步:让审查跟着路径走,而不是跟着署名走。 加一个 .github/CODEOWNERS:

/.github/workflows/  @your-org/infra
/.github/CODEOWNERS  @your-org/infra
Dockerfile           @your-org/infra
/terraform/          @your-org/infra

再在分支保护规则或 ruleset 里打开“Require review from Code Owners”。这样不管 PR 是谁写的、署了什么名,只要碰了这些路径,就必须有指定的人点头;只改业务代码的 PR 照常走,不拖慢。

最后给自己定一条规矩:改了 CI 的 PR,它自己跑出来的绿勾不算证据。 要看 diff 里 workflow 改了什么,必要时用主分支上的 workflow 再跑一遍。

一句话:我们担心 Agent 写的代码不可靠,却在实际操作中给了它比陌生人更大的信任。别再按署名决定看不看,按它改了哪里决定。

Picture two PRs landing in your open-source repo at the same time. Both come from strangers with no write access, both touch a few application files, and both quietly edit a yml under .github/workflows/. The only difference: one was written by a person, the other says "Generated by Codex" in the description, or its author avatar is Copilot.

Which one do you read more carefully?

Most people's instinct: the agent one, obviously, machine-written code is less trustworthy. The data says the opposite.

My take: we review agent PRs more loosely than we review strangers. The class that most needs review, PRs that also change CI, is exactly the class most likely to merge unread, and its green check comes from the pipeline it just rewrote. This isn't an agent problem. It's maintainers reading "written by an agent" as "machine-generated chore".

The evidence is Mohammad Wathiq Soualhi (independent researcher) and Basem Suleiman (UNSW), "Co-Modifying Application Code and Deployment Infrastructure: A Causal Study of AI Agent Pull Requests" (arXiv:2610.12849, 9 Oct 2026). They take 26,999 PRs from two public datasets (AIDev v5 and MOSAIC-3M), all from GitHub repos with more than 100 stars, created in June–July 2025, covering six agents (Codex, Claude Code, Copilot, Devin, Cursor, Jules) plus a matched set of human-written PRs.

Same stranger, agent byline, less review

The cleanest comparison: look only at external contributors with zero write access, 2,232 PRs. These people have no organisational tie to the repo, so in principle maintainers should be equally wary of all of them.

Human-written PRs got external review 82.7% of the time. Agent-written ones, 63.6%. Controlling for files changed, lines changed, whether tests were included and repo stars, an agent byline cut the odds of human review by 63.7% (OR = 0.363, 95% CI 0.293–0.451).

Same stranger, different byline

This group specifically rules out "maintainers use agents to open their own PRs, so they skip review". The authors put it plainly: whether a PR gets reviewed follows attribution, not patch complexity and not privilege.

The class that most needs review slips most

The paper calls PRs that change both application code and deployment infrastructure dual-scope PRs. They're only about 4.5% of agent PRs, but more than 70% of them touch CI/CD pipelines, and when only one infrastructure file changes, it's overwhelmingly a CI config.

For human authors, adding a CI change barely moves the formal review rate: 72.2% vs 73.4%. For agents it wobbles: in MOSAIC-3M it drops from 52.5% (code only) to 36.2% (code plus CI); in AIDev v5 it rises slightly, 47.5% to 53.1%. The two datasets disagree on direction, and the MOSAIC result has p = 0.052 on just 80 pairs, so it's not proof on its own.

But this comparison, within a single dataset, is hard to explain away: of PRs that also change CI, 19.0% of human-written ones merged with no review comments; for agent-written ones it was 56.2%. Across both datasets, 43% to 56% of these agent PRs merge with zero external comments and zero reviews, in a median of 26 to 31 minutes.

The green check grades itself

Why are CI changes the dangerous ones? Because they make "tests passed" meaningless.

The green check grades itself

A PR that edits .github/workflows/test.yml gets validated by the edited version. It can skip a test suite, mark a step continue-on-error, or narrow the lint paths. And then it's green. The paper calls this a circular verification failure: the thing being checked is the checker, so the green status gives zero independent assurance. CI security already has a name for it, Poisoned Pipeline Execution: untrusted contributions rewrite workflow definitions to execute arbitrary code inside build runners.

Put the two together: an outside account opens an agent PR that tweaks CI on the side, a maintainer sees "another agent PR", glances at the green check, and merges within half an hour. Nobody on that path needs bad intent. Everyone just needs to be busy.

My guess: chores trained people to read only the check

The paper doesn't explain why maintainers go easier on agents. What follows is my speculation, not the paper's finding.

The same paper has this number: 17.7% to 25.4% of agent PRs are pure chores, touching neither application code nor infrastructure, only docs, tests or lockfiles; among AIDev's chore PRs, README.md alone appears in 44.7%. Maintainers drowning in "agent updated your README" and "agent bumped a dependency" naturally learn a routine: if it's an agent, skim, merge when green.

The trouble is that the habit was learned on low-risk PRs and gets applied unchanged to PRs that rewrite the pipeline. Review effort should follow which paths changed. Right now it follows whose name is on it.

One step you can run now

Step one: find out how many "changed CI, nobody reviewed" PRs your repo has already merged. With gh and jq installed, run this inside the repo:

gh pr list --state merged --limit 300 \
  --json number,title,author,mergedAt,files,reviews \
| jq -r '.[]
  | select(any(.files[]; .path | test("^\\.github/workflows/|(^|/)Dockerfile$|\\.tf$|^(terraform|k8s|deploy)/")))
  | .author.login as $a
  | select([.reviews[] | select(.author.login != $a)] | length == 0)
  | "#\(.number)\t\($a)\t\(.mergedAt[:10])\t\(.title)"'

It lists merged PRs among the last 300 that touched workflows, Dockerfiles or Terraform/k8s/deploy directories with no review from anyone but the author. Adjust the path regex to your repo.

Step two: make review follow paths, not bylines. Add a .github/CODEOWNERS:

/.github/workflows/  @your-org/infra
/.github/CODEOWNERS  @your-org/infra
Dockerfile           @your-org/infra
/terraform/          @your-org/infra

Then turn on "Require review from Code Owners" in your branch protection or ruleset. Whoever wrote the PR, whatever name is on it, touching those paths needs a named person's approval; application-only PRs flow as before.

And one rule for yourself: a PR that changes CI doesn't get to count its own green check as evidence. Read what the workflow diff changed, and if needed rerun with the workflow from the main branch.

In one line: we worry agent code is unreliable, then in practice hand it more trust than a stranger. Stop deciding what to read by who signed it; decide by what it touched.