4d64700b1b
A window that spans the 2026-08-14 14:22Z production rollout contains devices that could not possibly have run the gate. Its rate is neither a baseline nor a result, and it prints identically to both. This was not hypothetical: a `post=08-14T07:00Z..now` window (84% of it pre-rollout) was run against this script and reported opencode-mobile *rising* to 4.44/h. `noise-gate-report.mjs` shipped with the same defect built into its default: `post = now-7d..now` straddles the rollout on every run before 08-21, diluting the after-rate toward baseline - biased toward grading the gate as ineffective on exactly the dates the ticket schedules its reads (08-17, 08-21). - sentry-volume-report: `--since-rollout` reads the instant from the release history table in docs/playstore.md (production versionCode >= 150, earliest such release, so a later v0.4.15 does not restart the window) and splits there. Every window is labelled [pre]/[post]/[mixed]; mixed prints how much of it predates the gate, a young post window prints its uptake age, and an unparseable table reports "unknown" rather than assuming post. - noise-gate-report: defaults post to the rollout instant, returns UNGRADED for a mixed/unknown post window, and pins the baseline to the documented post-box-bot-fix window instead of a 7d lookback that dragged ~22k/mo of already-fixed box-bot volume into the org outlook (it read "MISSES by 18,612" for a dead reason; now 628/mo, clears). - before_send == 0 is now reported as expected in a pre/mixed/young window and as a failure only after 24h+ of gated production. Re-probed every server-side lever with a WRITE-scoped token so none of the answers is a permissions artifact, and corrected the record in docs/analytics.md: per-key rate limit returns 200 and silently drops the field; error-message filters return 400 "You do not have that feature enabled" (a plan gate, not absence - it is the one lever that would reach never-updating installs); spike protection is not 403-unavailable, it is already enabled everywhere and simply does not fire on sustained baseline volume. Co-authored-by: engineer <engineer@macbookpro.lan>
89 lines
3.4 KiB
YAML
89 lines
3.4 KiB
YAML
name: Sentry noise-gate report
|
|
|
|
# Grades the AGE-105 noise gate against the number it promised, with install-base
|
|
# uptake folded in — see scripts/noise-gate-report.mjs for why a raw event count
|
|
# cannot grade a gate that ships inside an app binary.
|
|
#
|
|
# It lives in CI rather than on a laptop for one blunt reason: neither credential
|
|
# it needs (Sentry org read token, Play service account) exists outside GitHub
|
|
# Secrets, so an agent picking up this measurement locally is stuck. Dispatch this
|
|
# instead and read the run summary:
|
|
#
|
|
# gh workflow run "Sentry noise-gate report" -f post=2026-08-21T00:00:00Z..now
|
|
# gh run watch <id> && gh run view <id> # table is in the job summary
|
|
#
|
|
# The scheduled Monday run exists so the trend is recorded even if nobody asks.
|
|
|
|
on:
|
|
workflow_dispatch:
|
|
inputs:
|
|
pre:
|
|
description: "Baseline window START..END (rate before the rollout reached devices)"
|
|
required: false
|
|
# Post-box-bot-fix, pre-mobile-rollout. The only clean baseline that
|
|
# measures mobile alone: AGE-55 silenced openclaw-box-bot at 06:19Z and
|
|
# v0.4.14 reached production at 14:22Z on the same day.
|
|
default: "2026-08-14T07:00:00Z..2026-08-14T14:00:00Z"
|
|
post:
|
|
description: "Measured window START..END (default: trailing 7d)"
|
|
required: false
|
|
default: ""
|
|
project:
|
|
description: "Sentry project slug"
|
|
required: false
|
|
default: "opencode-mobile"
|
|
schedule:
|
|
# Mondays 15:00 UTC. Weekly, not daily: a window under ~24h ranks sources but
|
|
# cannot certify a monthly rate, and Play vitals land with a multi-day lag.
|
|
- cron: "0 15 * * 1"
|
|
|
|
permissions:
|
|
contents: read
|
|
|
|
concurrency:
|
|
group: sentry-noise-gate-report-${{ github.ref }}
|
|
cancel-in-progress: false
|
|
|
|
jobs:
|
|
report:
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 10
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
|
|
- uses: actions/setup-node@v6
|
|
with:
|
|
node-version: 24
|
|
|
|
# The grading model is the part that can be wrong silently, so it is tested
|
|
# before it is trusted — in the same job that publishes the number.
|
|
- name: Test the grading model
|
|
run: node --test scripts/noise-gate-report.test.mjs
|
|
|
|
- name: Report
|
|
env:
|
|
SENTRY_AUTH_TOKEN: ${{ secrets.SENTRY_PRODUCT_INTELLIGENCE_TOKEN || secrets.SENTRY_AUTH_TOKEN }}
|
|
SENTRY_ORG: ${{ secrets.SENTRY_ORG || 'vibetechnologies' }}
|
|
GOOGLE_PLAY_SERVICE_ACCOUNT_JSON: ${{ secrets.PLAY_STORE_SERVICE_ACCOUNT_JSON }}
|
|
run: |
|
|
set -euo pipefail
|
|
args=(--pre "${{ inputs.pre || '2026-08-14T07:00:00Z..2026-08-14T14:00:00Z' }}")
|
|
args+=(--project "${{ inputs.project || 'opencode-mobile' }}")
|
|
if [ -n "${{ inputs.post }}" ]; then args+=(--post "${{ inputs.post }}"); fi
|
|
node scripts/noise-gate-report.mjs "${args[@]}"
|
|
|
|
- name: Raw org volume (per project, per outcome, with client_discard reasons)
|
|
if: always()
|
|
env:
|
|
SENTRY_AUTH_TOKEN: ${{ secrets.SENTRY_PRODUCT_INTELLIGENCE_TOKEN || secrets.SENTRY_AUTH_TOKEN }}
|
|
SENTRY_ORG: ${{ secrets.SENTRY_ORG || 'vibetechnologies' }}
|
|
run: |
|
|
set -euo pipefail
|
|
{
|
|
echo ''
|
|
echo '### Raw org volume'
|
|
echo '```'
|
|
node scripts/sentry-volume-report.mjs --by-reason --since-rollout || true
|
|
echo '```'
|
|
} >> "$GITHUB_STEP_SUMMARY"
|