AI code review: a product tour
Will this engineer catch what an AI coding agent got wrong before it merges?
You’re Jordan Lee
Candidate
Jordan Lee reviews a pull request that an AI coding agent wrote. Then you see what the hiring team sees: the answer key beside Jordan’s work, a replay of the review, and the actions behind each level.
11 steps · about 2 minutes
Read the tour as text
Will they catch what an AI agent got wrong?
Jordan reviews a pull request written by an AI coding agent. It arrives with the author’s claims, a review bot’s comments and checks Jordan can run.
Can they point at the exact line?
Jordan reads the API docs, then leaves a blocking comment on the exact line where the change goes wrong. Each comment is anchored to a line, which is how it is matched to a known defect later.
Do they trust the review bot?
The review bot is sometimes wrong. Agreeing with a wrong bot comment counts against the review, and so does dismissing a right one.
Do they take the author’s word for it?
Jordan asks the author why the cursor code changed. The author answers confidently, and here the answer is wrong.
Do they check?
Jordan runs the pagination check instead of taking the answer on trust, and it fails.
What happens after they request changes?
Jordan comments on the skipped rows, then requests changes and says why. The author pushes one revision, and the review continues on the new code.
Do they verify the fix before approving?
Jordan reruns every check on the revision, sees them pass and approves. The final verdict ends the review.
Which problems did they find?
You see each planted defect next to the comment that found it, the severity Jordan gave it, and whether Jordan checked the evidence first.
What did they do, and in what order?
Replay steps through the review as Jordan saw it, with the time between actions.
How is each level set?
In hiring, rules set what they can: the line, the checks run, bot triage, the verdict. For each defect found, you answer whether the comment explains it, with an AI reading and a word check beside it as suggestions. Levels that rest on that wait for your answer.
Who decides?
A person. Time away and pastes are shown as the browser reported them and are never scored. You read the evidence, add a note and decide.