tools(sentry): org-wide volume report + measure on submitted, not accepted (#172)

* tools(sentry): add org-wide volume report and fix the metric we measure on

The AGE-105 gate is a measured number, so it needs a repeatable query. It also
needed a correction: `accepted` is the wrong headline. The org is over its error
quota, so Sentry rejects nearly everything and `accepted` reads ~0 for every
project - a blown org and a fixed one look identical on that column. The demand
metric is `submitted` = accepted + rate_limited.

scripts/sentry-volume-report.mjs takes named --window ranges and prints
per-project submitted / accepted / rate_limited / client_discard plus the
per-hour and projected per-month rate, so before/after comparisons run the exact
same query instead of being re-derived by hand each time.

Records the pre-rollout baseline in docs/analytics.md: opencode-mobile at
4.71/h (3,441/mo), 87% of the org's post-box-bot demand, from two windows that
agree to within 0.2%.

Co-Authored-By: Paperclip <noreply@paperclip.ing>

* test(sentry): pin the noise gate against 90d of real production events

The gate's unit tests prove it behaves as specified. Nothing proved the spec
was aimed at the right targets. Replaying the actual 90d census of the
opencode-mobile Sentry project (648 events, 11 issues) through the gate's own
precedence shows 96.9% hard-dropped as transport noise and every observed crash
class (OOM, ANR, IllegalStateException) still allowlisted -> ~87 events/month
against a 1,500/month target.

Also records two findings from measuring the org directly:

* The error quota resets on the 4th. The 5,000-event month opened 2026-08-04
  and was spent by 08-08; the org has accepted zero errors since. 2026-09-04 is
  the date the gate has to hold by, and it is why 'submitted' is the metric.
* Server-side levers are unavailable on this plan. A per-key rate limit PUT
  returns HTTP 200 and silently discards the value (verified for three window
  sizes), custom inbound filters are absent, spike protection 403s. The client
  gate is the only control that exists, so its coverage is the whole margin.

Refs AGE-105

Co-Authored-By: Paperclip <noreply@paperclip.ing>

* tools(sentry): split client_discard by reason so gate drops aren't confused with quota backoff

Raw client_discard cannot show whether the noise gate works. Today 100% of
opencode-mobile's client_discard is ratelimit_backoff -- the SDK backing off a
429 because the ORG is over quota -- which rises when things get WORSE. Gate
drops land in a different reason: @sentry/core records before_send when
beforeSend returns null.

- stats_v2 now groups by reason as well as project/outcome
- the before_send vs ratelimit_backoff split always prints; --by-reason adds
  the full per-project reason table
- before_send > 0 is install-share-independent, so it proves the gate is live
  on real devices days before a monthly rate can bend
- documents that release-level segmentation is impossible while over quota:
  rate_limited events are never stored, so release tags stop (last value
  0.4.12, 2026-08-08). Version share comes from Play, not Sentry.

* ci(sentry): block a Play release whose bundle lost the noise gate

The AGE-105 quota fix is entirely client-side (every server-side lever on
this plan is dead), so the gate being *in the shipped binary* is the whole
safety margin. That is also the one thing Sentry cannot tell us: while the
org is over quota nothing is stored, release tags stop dead at 0.4.12, and
a release:0.4.14 query returns empty in a way that reads like success.

Grep the Hermes bundle inside the AAB instead, before the Play upload step:
the gate's reason codes, the transport drop-list regex, the
noise.dropped_since_last tag only applyNoiseGate() writes, and a baked-in
DSN (a release built without EXPO_PUBLIC_SENTRY_DSN makes Sentry a silent
no-op). Verified to discriminate on real artifacts - the v0.4.14 build now
on Play production passes, pre-gate v0.4.13 fails all six markers.

Also records the rejected alternative: persisting gate state across cold
starts pays off only under ~94 active devices (2,633 session envelopes/7d
vs a 6h cooldown), and the install base is above that.

---------

Co-authored-by: engineer <engineer@macbookpro.lan>
Co-authored-by: Paperclip <noreply@paperclip.ing>

scripts/verify-release-bundle.mjs

@@ -0,0 +1,137 @@
+#!/usr/bin/env node
+// Verify that a release AAB actually ships the Sentry noise gate (AGE-105).
+//
+// Why this exists: the entire quota fix is client-side. Sentry's server-side
+// levers were checked and are all dead on this plan (per-key rate limit returns
+// HTTP 200 and silently drops the field; custom inbound filters absent; spike
+// protection 403). So if a build ever ships without the gate — a Metro entry
+// change, a refactor that stops importing sentry-noise, a release built without
+// EXPO_PUBLIC_SENTRY_DSN — the org quietly goes back over quota and nobody
+// finds out until the monthly reset.
+//
+// It also can't be caught by watching Sentry: while the org is over quota,
+// events are rejected at ingest and never stored, so release tags stop
+// updating (opencode-mobile's stop at 0.4.12 / 2026-08-08 while clients keep
+// submitting ~4.7/h). A `release:0.4.14` query returns EMPTY, which reads like
+// "no errors from the new build" but actually means "no data at all".
+//
+// The binary itself is the only same-day evidence. Hermes bytecode keeps string
+// literals, so the gate's own reason strings and the drop-list regex are
+// greppable in base/assets/index.android.bundle.
+//
+// Usage:
+//   node scripts/verify-release-bundle.mjs android/app/build/outputs/bundle/release/app-release.aab
+//
+// Optional env:
+//   EXPO_PUBLIC_SENTRY_DSN  when set, the DSN baked into the bundle must point
+//                           at the same project id (never printed).
+//
+// Read-only, dependency-free (uses the `unzip` CLI, present on the runner).
+
+import { execFileSync } from "node:child_process"
+import { existsSync } from "node:fs"
+
+/** Literals that only exist because the noise gate is in the bundle graph.
+ *  Each is unique to our code, not the Sentry SDK. */
+export const GATE_MARKERS = [
+  // NoiseGate reason codes (src/lib/sentry-noise.ts)
+  "transport-noise",
+  "new-fingerprint-cap",
+  "hourly-cap",
+  "always-send",
+  // the transport drop-list regex source — the single biggest volume cut
+  "connect (?:timeout",
+  // applyNoiseGate() in src/lib/sentry.ts, i.e. the gate is wired to beforeSend
+  "noise.dropped_since_last",
+]
+
+const DSN_RE = /https:\/\/[0-9a-f]{16,64}@[a-z0-9.-]*ingest[a-z0-9.-]*\/(\d+)/g
+
+/** Pure check over the raw JS/Hermes bundle bytes. Returns a list of problems;
+ *  empty means the artifact is good. Never returns the DSN itself. */
+export function checkBundle(bundle, opts = {}) {
+  const text = Buffer.isBuffer(bundle) ? bundle.toString("latin1") : String(bundle)
+  const problems = []
+
+  const missing = GATE_MARKERS.filter((marker) => !text.includes(marker))
+  if (missing.length) {
+    problems.push(`noise gate missing from bundle — absent markers: ${missing.join(", ")}`)
+  }
+
+  const projectIds = [...text.matchAll(DSN_RE)].map((m) => m[1])
+  if (projectIds.length === 0) {
+    // No DSN => Sentry.init() is a no-op => the gate never runs and no telemetry
+    // arrives at all. A release build must never be in this state.
+    problems.push("no Sentry DSN baked into the bundle — telemetry would be a silent no-op")
+  } else if (opts.expectedProjectId && !projectIds.includes(String(opts.expectedProjectId))) {
+    problems.push(
+      `bundled DSN points at project ${projectIds.join("/")}, expected ${opts.expectedProjectId}`,
+    )
+  }
+
+  return problems
+}
+
+/** Extract the project id from a DSN without leaking the key. */
+export function dsnProjectId(dsn) {
+  if (!dsn) return null
+  const m = /\/(\d+)\/?$/.exec(String(dsn).trim())
+  return m ? m[1] : null
+}
+
+export function checkAppConfig(configJson, expectedVersion) {
+  if (!expectedVersion) return []
+  let version
+  try {
+    version = JSON.parse(configJson)?.version
+  } catch {
+    return ["base/assets/app.config is not valid JSON"]
+  }
+  return version === expectedVersion ? [] : [`bundle app.config version ${version}, expected ${expectedVersion}`]
+}
+
+function unzipEntry(archive, entry) {
+  return execFileSync("unzip", ["-p", archive, entry], { maxBuffer: 256 * 1024 * 1024 })
+}
+
+function main() {
+  const aab = process.argv[2]
+  if (!aab || !existsSync(aab)) {
+    console.error("usage: node scripts/verify-release-bundle.mjs <path-to.aab|.apk>")
+    process.exit(2)
+  }
+
+  let bundle
+  try {
+    bundle = unzipEntry(aab, "base/assets/index.android.bundle")
+  } catch {
+    console.error(`FAIL ${aab}: no base/assets/index.android.bundle inside the archive`)
+    process.exit(1)
+  }
+
+  const problems = checkBundle(bundle, { expectedProjectId: dsnProjectId(process.env.EXPO_PUBLIC_SENTRY_DSN) })
+
+  let version = null
+  try {
+    const config = unzipEntry(aab, "base/assets/app.config").toString("utf8")
+    version = JSON.parse(config)?.version ?? null
+  } catch {
+    problems.push("could not read base/assets/app.config")
+  }
+
+  console.log(`artifact: ${aab}`)
+  console.log(`bundle:   ${(bundle.length / 1024 / 1024).toFixed(2)} MiB, app version ${version ?? "unknown"}`)
+  for (const marker of GATE_MARKERS) {
+    const ok = bundle.toString("latin1").includes(marker)
+    console.log(`  ${ok ? "ok  " : "MISS"} ${marker}`)
+  }
+
+  if (problems.length) {
+    console.error("\nFAIL — this build must not go to Play:")
+    for (const p of problems) console.error(`  - ${p}`)
+    process.exit(1)
+  }
+  console.log("\nOK — Sentry noise gate is present and pointed at the expected project.")
+}
+
+if (import.meta.url === `file://${process.argv[1]}`) main()