tools(sentry): anchor the measurement windows on the gate's rollout instant (#178)
A window that spans the 2026-08-14 14:22Z production rollout contains devices that could not possibly have run the gate. Its rate is neither a baseline nor a result, and it prints identically to both. This was not hypothetical: a `post=08-14T07:00Z..now` window (84% of it pre-rollout) was run against this script and reported opencode-mobile *rising* to 4.44/h. `noise-gate-report.mjs` shipped with the same defect built into its default: `post = now-7d..now` straddles the rollout on every run before 08-21, diluting the after-rate toward baseline - biased toward grading the gate as ineffective on exactly the dates the ticket schedules its reads (08-17, 08-21). - sentry-volume-report: `--since-rollout` reads the instant from the release history table in docs/playstore.md (production versionCode >= 150, earliest such release, so a later v0.4.15 does not restart the window) and splits there. Every window is labelled [pre]/[post]/[mixed]; mixed prints how much of it predates the gate, a young post window prints its uptake age, and an unparseable table reports "unknown" rather than assuming post. - noise-gate-report: defaults post to the rollout instant, returns UNGRADED for a mixed/unknown post window, and pins the baseline to the documented post-box-bot-fix window instead of a 7d lookback that dragged ~22k/mo of already-fixed box-bot volume into the org outlook (it read "MISSES by 18,612" for a dead reason; now 628/mo, clears). - before_send == 0 is now reported as expected in a pre/mixed/young window and as a failure only after 24h+ of gated production. Re-probed every server-side lever with a WRITE-scoped token so none of the answers is a permissions artifact, and corrected the record in docs/analytics.md: per-key rate limit returns 200 and silently drops the field; error-message filters return 400 "You do not have that feature enabled" (a plan gate, not absence - it is the one lever that would reach never-updating installs); spike protection is not 403-unavailable, it is already enabled everywhere and simply does not fire on sustained baseline volume. Co-authored-by: engineer <engineer@macbookpro.lan>
This commit is contained in:
17
.github/workflows/publish-play-store.yml
vendored
17
.github/workflows/publish-play-store.yml
vendored
@@ -133,13 +133,16 @@ jobs:
|
||||
run: ./gradlew bundleRelease
|
||||
|
||||
- name: Verify the Sentry noise gate is in the artifact
|
||||
# AGE-105: the org error quota is defended ONLY by the client-side gate
|
||||
# (every server-side lever on this Sentry plan was checked and is dead:
|
||||
# per-key rate limit silently no-ops with a 200, custom inbound filters
|
||||
# absent, spike protection 403). If a build ships without the gate — or
|
||||
# without a DSN, which makes Sentry a silent no-op — the org goes back
|
||||
# over quota, and while it is over quota Sentry stores nothing, so the
|
||||
# regression is invisible in Sentry itself until the monthly reset.
|
||||
# AGE-105: the org error quota is defended ONLY by the client-side gate.
|
||||
# Every server-side lever on this Sentry plan was probed and is dead
|
||||
# (docs/analytics.md): per-key rate limit answers 200 and silently drops
|
||||
# the field, custom error-message filters answer 400 "You do not have
|
||||
# that feature enabled", and spike protection is already on everywhere
|
||||
# but only catches spikes, not this steady baseline. If a build ships
|
||||
# without the gate — or without a DSN, which makes Sentry a silent
|
||||
# no-op — the org goes back over quota, and while it is over quota
|
||||
# Sentry stores nothing, so the regression is invisible in Sentry itself
|
||||
# until the monthly reset.
|
||||
# Grep the shipped Hermes bundle instead. Verified to discriminate:
|
||||
# v0.4.14 passes, pre-gate v0.4.13 fails.
|
||||
run: node scripts/verify-release-bundle.mjs android/app/build/outputs/bundle/release/app-release.aab
|
||||
|
||||
@@ -83,6 +83,6 @@ jobs:
|
||||
echo ''
|
||||
echo '### Raw org volume'
|
||||
echo '```'
|
||||
node scripts/sentry-volume-report.mjs --by-reason || true
|
||||
node scripts/sentry-volume-report.mjs --by-reason --since-rollout || true
|
||||
echo '```'
|
||||
} >> "$GITHUB_STEP_SUMMARY"
|
||||
|
||||
Reference in New Issue
Block a user