When Your Code Reviewer Runs Out of Credits

It was one of those moments where a tool quits exactly when you need it most. A project under time pressure, a stack of pull requests — and my automated code reviewer answered every single one with the same line:

we couldn't start this review because you've reached your PR review rate limit.

The prepaid credits were gone. In practice that meant pull requests were merging unreviewed. In a hot phase, that is precisely the risk you don't want to carry.

Three obvious reactions: buy more credits, review by hand, or merge unreviewed. Expensive, slow, or reckless. So Claude and I built a fourth one — a switch.

The idea: the outage is machine-readable

The key was hiding in the error itself. The reviewer doesn't post its rate-limit notice as prose; it embeds a stable, hidden marker in the comment:

<!-- This is an auto-generated comment: rate limited by coderabbit.ai -->

That isn't a bug. It's a signal. If the outage is machine-readable, I can react to it deterministically — no polling, no guessing at timeouts. A GitHub workflow listens for new comments and only fires when exactly that marker arrives from the reviewer bot:

on:
  issue_comment:
    types: [created]

jobs:
  claude-review:
    if: >
      github.event.issue.pull_request &&
      github.event.comment.user.login == 'coderabbitai[bot]' &&
      contains(github.event.comment.body, 'rate limited by coderabbit.ai')
    # ... then: Claude reviews the PR

When the primary reviewer runs normally, the marker isn't there — and nothing happens. No double review, no doubled spend. A real switch: one reviewer takes over when the other drops out.

The trap you learn exactly once

issue_comment workflows always run with the file version from the default branch, not the one on the PR. So the switch only takes effect after it's merged to main — and then for every future PR. One more sharp edge: in these events the PR number lives in github.event.issue.number, not pull_request.number (which is null here). Two small stones that otherwise cost you half an hour.

From one repo to the whole portfolio

A switch in one repository is nice. But the reviewer's credits were exhausted everywhere. So: roll it out across every active repo.

Instead of clicking through 14 times, everything ran through the GitHub API — create a branch, write the file, open a PR, fully remote:

gh api repos/$SLUG/git/refs -f ref="refs/heads/ci/claude-review-switch" -f sha="$BASE_SHA"
echo "$PAYLOAD" | gh api -X PUT "repos/$SLUG/contents/$WF_PATH" --input -
gh pr create -R "$SLUG" --base "$DEF" --head "ci/claude-review-switch" --title "…"

One prerequisite: the gh token needs the workflow scope, otherwise GitHub refuses to write workflow files. One run, fourteen repositories.

"Just copy the secret" — you can't

Here came the first real lesson. Every workflow needs a token. My reflex was to "copy" the secret over from the repo where it already worked. You can't. GitHub Actions secrets are write-only — you can set them, never read them back. For good reason.

The clean solution: generate the token once, store it in a password manager, and pull it per repo straight from there at rollout time:

gh secret set CLAUDE_CODE_OAUTH_TOKEN -R "$SLUG" \
  --body "$(op item get 'claude-code/oauth - ci' --vault 'Development Keys' \
              --fields CLAUDE_CODE_OAUTH_TOKEN --reveal)"

The token never lands in a terminal history, a chat, or a log — it flows straight from the vault into the secret.

The detour that mattered more than the goal

Thirteen of fourteen PRs merged straight away. One stuck — blocked by branch protection because a required check called lint-and-test was red. The easy path would have been an admin override. The right path was to find out why.

The failing step: npm audit --audit-level=moderate. Three transitive vulnerabilities had piled up — one "high", two "moderate". None of it had anything to do with my change; the check had been red for days and was blocking every PR.

A single npm audit fix (no --force, all non-breaking) resolved all three. Verified locally end to end — npm ci, audit, lint, test (103/103 green) — then merged. Only then, the switch.

That's the real lesson: an npm audit gate is good, but it accumulates silent security debt until some entirely unrelated PR trips over it. Whoever waves it through with an admin override fixes nothing — they hand the problem to the next person.

What stays

Automate the detection, not the manual fallback. The marker turns "I should check on that" into a deterministic event.
Secrets have a source, not a backup. Write-only forces clean token hygiene — the vault as the single truth.
Root cause before convenience. The blocked PR wasn't an obstacle. It was a hint.

I didn't build this alone — Claude was in the loop the whole way: research, script, verification, rollout. Exactly the kind of work an agent shines at: many small, correct steps across many repos, without anyone's focus running out halfway through.