ratemyserver · Release review for APIs, workers, and pipelines
Your backend works when everything goes right. Now make it survive retries, races, dependency failures, restarts, and real users — with five evidence lanes that cannot substitute for each other.
Issue list
BLOCKER · F-001A tenant can read another tenant's report — private data crosses the ownership boundary.
HIGH · F-003A duplicated webhook delivery applies the same credit twice — there is no durable uniqueness on the effect.
To verify
UNKNOWN · U-001Backups exist but restoration has never been exercised — recoverability is not yet proved.
Evidence lanes
Scope
Only what has a product consequence. Style preferences and fashionable architecture are not findings.
Authentication, object ownership, role policy, tenant isolation, secrets, and privacy boundaries.
Request, job, stream, and batch contracts — including whether reported success is actually true.
Transactions, idempotency, durable uniqueness, ordering, leases, acknowledgement, replay, reconciliation.
Timeouts, retry budgets, dependency failure, admission control, queue bounds, saturation, backpressure.
Graceful shutdown, restart, deploy, migration, rollback, telemetry, backup, restore, incident recovery.
Cross-component journeys: request → commit/outbox → queue → worker → external effect.
With and without
A capable agent already reviews code. These are the behaviors that appear only with the Skill loaded.
Without it
It reads the auth middleware, sees a session check, and calls the boundary fine.
With it
It reproduces an authenticated Alpha caller reading a Beta report by ID because the lookup carries no tenant predicate — with an own-tenant read kept as the clean control.
Without it
It suggests “add idempotency” as a best practice.
With it
It reproduces the effect committing and then timing out, and shows retry creating a second effect for the same business identity.
Without it
Green tests and a healthy golden path read as production-ready.
With it
It reproduces blocked in-flight work outliving the join grace and leaving a job in processing, then connects startup recovery to that state without claiming data loss it did not observe.
Both columns come from the Skill's own test cases — what it requires of itself, not a score
anyone measured. The paired runs are written down but have not been executed, so if you want the number rather than the
promise, run them: evals/execution_cases.json.
How it runs
It never silently defaults to the strictest review — or the friendliest.
Service mode matched to what the service actually owns
Quick check / Strict review / Launch gate / Real stakes / Life-or-death
Hard gates block the release regardless of the weighted average. Accepting the risk does not turn an active gate into a technical pass.
Findings advance to verified-fixed only on fresh passing evidence from an independent retest that
also audits the fix code itself.
The five
Each is a standalone plugin. Install only the layer you are shipping.
Install
Pick one method per client and scope. The first audit runs read-only — your sandbox, permissions, and approvals stay the security boundary.
/plugin marketplace add \ AmsonntagChow/ratemyserver /plugin install \ ratemyserver@amsonntagchow-ratemyserver
codex plugin marketplace add \ AmsonntagChow/ratemyserver codex plugin add \ ratemyserver@ratemyserver
npx skills add \ AmsonntagChow/ratemyserver \ --skill ratemyserver