If AI writes the code, what should I review?

I'm looking for other ways to maintain quality, without giving up on understanding what's changing. Part of that means deciding earlier what the software should do and what must remain true, then checking whether the tests actually verify those expectations.

I start with a specification: the expected behavior, the constraints, and the properties that must hold. I also decide which risks deserve the most attention. An agent can then implement a solution and write tests against those expectations.

To avoid relying on one agent's reasoning for both the code and its tests, I can give the same specification to a second agent. Before seeing the first agent's code or tests, it works out its own expectations and test cases. Only then does it compare its work with the implementation and existing tests, looking for differences and missing cases. I can also use different models to bring in different approaches, although that doesn't guarantee they won't make the same mistakes.

In my projects, I also challenge some tests with targeted, automated scenarios. They deliberately introduce defects into the software, then run the tests that are supposed to catch them. If a test still passes, I investigate why. This lets me evaluate some of those checks without having to review every test individually.

Imagine a program that's supposed to draw a blue square. An agent writes the program and a test to check the result. To challenge that test, an automated scenario changes the program so it draws a red square instead. If the test still passes, it missed the mistake. If it fails because the color changed, at least I know it can catch that specific defect.

None of this guarantees bug-free software. Agents can share the same wrong assumptions, and tests can miss important cases. I also need to make sure the reported checks actually ran. I still review code and make technical decisions, especially around interfaces, architecture, and critical algorithms. How much review and validation I do depends on the risk.

Agents let me explore more cases, and I'm trying to make those checks more varied and more reliable. But running them takes time. In my current setup, a full validation run takes around 25 minutes. The next challenge is figuring out how to reduce that cost without weakening the guarantees that matter.