Summaries > AI > Ai Hero > Fixing the PR Bottleneck â€” Matt Pocock, AIHero...

Fixing The Pr Bottleneck â€” Matt Pocock, Ai Hero

https://www.youtube.com/watch?v=LlgiOCmFG_w

TLDR AI can speed up software development, but teams need automated checks, a dedicated AI code reviewer, and focused human review to prevent low-quality PRs from overwhelming reviewers. Make PRs easy to assess by explaining their risks and impact, reserve the most scrutiny for irreversible changes, and use recurring review issues to improve standards and automation.

Key Insights

Start with broad, inexpensive automated checks

Run tests, linting, type checks, and code-quality tools on every change to catch common problems early. These checks are relatively cheap and help agents produce work that is easier to review. However, a green CI result does not prove that code is safe or ready to merge. Treat automated checks as the first quality filter, not the final approval.

Make tests verify behavior, not implementation details

Check that tests exercise real behavior through the code’s public interfaces rather than merely repeating the implementation. Tautological tests, brittle source-code inspections, and mocks that cannot expose production failures can create false confidence. Deep modules—with complex behavior behind simple interfaces—make it easier to write meaningful tests. Review test quality as well as test results.

Separate implementation from standards-focused review

Let an implementation agent focus on exploring, changing, and debugging the code, then use a separate review agent to inspect the finished diff. Give the reviewer a team-maintained `coding standards.md` file and relevant project documentation so it can apply consistent expectations in its own context. This two-part workflow helps prevent standards from being overlooked amid implementation work. Ask the reviewer to fix straightforward issues directly and raise comments when it needs clarification.

Build an automated reviewer around your team’s needs

A team-specific code reviewer can apply your conventions and project knowledge more usefully than a generic service that may produce noisy or poorly targeted feedback. Incorporate coding standards and documentation into its guidance, and use review as an iterative step rather than expecting perfect code on the first attempt. Automated review should complement checks and human judgment, not replace them. The goal is to reduce repetitive feedback and make each pull request easier to assess.

Make pull requests easy for humans to evaluate

Explain what a change does and why, using concise summaries, pseudocode, diagrams, or other visual aids when they make the design clearer. Call out the change’s blast radius and potential consequences in a brief “merge danger” summary. Distinguish reversible decisions from one-way-door changes that are difficult or costly to undo. Highlight risks such as data loss, expensive migrations, or unintended mass communications so reviewers can focus their attention where it matters most.

Reserve careful human review for consequential decisions

Use automated checks and review to handle routine issues, then direct human attention toward risky or irreversible changes. Human reviewers should assess not only the code but also whether the workflow and agent environment contributed to recurring problems. This layered approach can make reviews faster while preserving careful judgment for decisions with significant consequences. Reversible changes can generally receive lighter scrutiny than one-way-door decisions.

Turn repeated review issues into improvements

When the same problems appear across agent sessions or pull requests, investigate the process instead of fixing each instance in isolation. Use a retrospective to identify whether the solution is a new automated check, an update to `coding standards.md`, or clearer navigation guidance for agents. Improving these shared resources helps prevent repeat mistakes and reduces future review effort. Treat each review as an opportunity to make the software workflow more reliable.

Questions & Answers

Why are safeguards needed when AI agents produce more code and pull requests?

Without safeguards, agents can create a flood of low-quality pull requests that people cannot review effectively. Automated checks, automated review, and human review help preserve quality and reduce the effort people need to spend.

What can automated checks catch, and why is passing CI not enough?

Tests, linting, type checking, and code-quality metrics can catch many issues cheaply. But passing CI does not prove code is ready to merge: tests can restate the implementation, inspect brittle implementation details, or use mocks that hide errors or cannot fail.

How can codebase design make tests more reliable?

Use deep modules that hide complex behavior behind simple interfaces. Tests can then exercise those interfaces instead of depending on internal implementation details, making them less brittle and more meaningful.

Why use a separate code-review agent, and how should it be used?

A separate reviewer has its own context window to inspect the diff, explore relevant code, and apply team standards without overloading the implementation agent. Have the implementation agent write the code first, then have the reviewer check and refactor it against a customizable `coding standards.md` file.

How should teams build an automated code reviewer?

Tailor the reviewer to the team by providing its coding standards and documentation, rather than relying on a generic third-party reviewer that may be noisy or overly specialized. The reviewer should generally commit fixes instead of leaving lengthy comments, while raising questions when needed.

What makes a pull request easier and safer for humans to review?

Explain what the change does and why, and make its risk clear. Distinguish reversible “two-way door” changes from risky “one-way door” decisions, and summarize the change’s “merge danger,” including its blast radius and consequences such as data loss, costly migrations, or unintended mass communications. Concise pseudocode, diagrams, and visual summaries can help.

What should human review evaluate besides the code itself?

Human review should also examine the process and agent environment that produced the code. A retro skill can analyze agent sessions, pull requests, and reviews, then suggest automated checks, updates to `coding standards.md`, or clearer navigation pointers to prevent recurring problems.

How should teams prioritize human review effort?

Layer automated checks and reviews to make human review faster, and reserve the most careful attention for risky, irreversible one-way-door decisions rather than spending equal effort on every reversible choice.

Summary of Timestamps

AI agents can greatly increase the amount of software work a team produces, but without safeguards they may generate a flood of low-quality pull requests that no one has time to review. The speaker recommends three layers of protection—automated checks, automated review, and human review—to preserve quality while helping teams move faster.

Automated checks such as tests, linting, type checking, and code-quality metrics are inexpensive and should be used broadly. However, passing CI does not prove a change is ready to merge: tests can simply repeat the implementation, depend on fragile source-code structure, or use mocks that conceal real failures.

Better codebase design can make tests more meaningful. Deep modules hide complex behavior behind simple interfaces, so tests can exercise what a module actually does instead of depending on its internals. Shared design concepts—such as locality, leverage, and seams—can also help teams discuss and improve their code consistently.

Keep implementation and review as separate agent tasks. An implementation agent already needs to explore the code, make changes, and debug them; adding every coding standard to its instructions can overload its context. Instead, a separate review agent can inspect the diff, explore relevant code, and apply guidance from a customizable `coding standards.md` file.

Use an iterative workflow: first implement the change, then have a reviewer agent identify and fix issues so the code meets team standards. This resembles red-green-refactor. A team-specific reviewer, guided by internal documentation and standards, can be more useful than a generic review service that may be too noisy or too narrowly focused.

The automated reviewer should generally commit straightforward fixes rather than leave long lists of comments that create extra work for human reviewers. It can ask questions when needed, but teams should not expect the implementation agent to produce perfect code in one attempt; checks and review are part of an ongoing improvement cycle.

Make pull requests easy for people to assess. Explain what changed and why, and describe the change’s risk or “merge danger.” In particular, distinguish reversible two-way-door decisions from risky one-way-door decisions, and call out potential consequences such as data loss, costly migrations, or unintended mass communications.

Use concise explanations and visual aids—such as pseudocode, diagrams, or summaries—to show how a change works. The speaker points to the “show me” skill in Dex Hadley’s Human Layer skills repository as an example of a tool for making changes clearer to reviewers.

Human review should examine not just the code, but also the process and agent environment that produced it. A retrospective skill can analyze agent sessions, pull requests, and reviews, then suggest new automated checks, updates to `coding standards.md`, or clearer navigation pointers to prevent recurring problems.

The overall goal is to reserve careful human attention for high-risk, hard-to-reverse decisions while letting automated checks and reviews handle routine issues. These tools can also reveal problems that are otherwise difficult to diagnose, including inefficient tool use and bloated instruction files or skills.

The speaker closes by noting that the PR skill is still in progress and will be released soon. They mention being at dot dev/skills, shipping version 1.3 that week, and invite attendees to continue the conversation in the lobby.