Files
opencode-mobile/distribution/waitlist-signup-path-coverage.md
Den 98233d351f measure: how much of the install base has no in-app waitlist signup path (#164)
AGE-61 asked for a number: what share of installs is still on a build older
than v0.4.8, where the ONLY waitlist path is a mailto: to a human inbox that
nothing reconciles back into the store (20 of 21 signups lost, 2026-08-03 →
2026-08-13). Answering it by hand is how it stays unanswered next quarter, so
this is a script, not a screenshot.

- scripts/play-version-share.mjs: Play Developer Reporting API
  (crashRateMetricSet -> distinctUsers by versionCode) for the auto-updating
  channel, plus --github for lifetime release-APK downloads per tag, which is
  the only per-version signal the sideload channel emits. Mints its own token
  from the service account we already ship to CI; no new deps, no new secret.
  Play versionCodes are run_number+100, NOT the gradle ones — the mapping is
  derived from the publish runs and documented inline (139 = v0.4.8).
- distribution/waitlist-signup-path-coverage.md: the measured answer.

The answer, 2026-08-14: Play is 0% stale (single reported versionCode 142 =
v0.4.10, ~90-100 daily users); the sideload channel is 25.9% stale (436 of
1682 lifetime APK downloads predate v0.4.8) and can never auto-update. There
is no iOS listing and no IzzyOnDroid presence, so nothing else contributes.

Two consequences worth stating plainly: the hourly mailto reconciler is
permanent infrastructure, not a stopgap; and shipping updates does NOT close
the leak, because on current builds the fallback still fires on timeout/5xx/
offline (src/lib/waitlist.ts:shouldFallbackToMailto).

Co-authored-by: engineer <engineer@macbookpro.lan>
Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-08-14 00:42:42 -07:00

3.3 KiB
Raw Blame History

Waitlist signup-path coverage — how much of the install base can't sign up in-app

Measured 2026-08-14 for AGE-61. Re-run with node scripts/play-version-share.mjs --github.

The question

The in-app OpenCode Connect waitlist form posts to POST /api/beta-signup (Brevo list 4). That code path first shipped in v0.4.8 (commit 0fdfb54, 2026-07-18). Every older build has exactly one path: open a mailto: to support@agentlabs.cc, which lands in a human inbox and nowhere near the waitlist store. 20 of 21 signups between 2026-08-03 and 2026-08-13 were lost that way. So: how many people are still on a build with no working signup path?

Answer, by channel

Channel Population measured Pre-v0.4.8 share
Google Play (auto-updates) ~90–100 daily distinct users, 2026-07-31 → 2026-08-13 0% — Play reports a single versionCode, 142 (v0.4.10). Any older build is below Play's reporting floor (<10 users/day).
Sideload — GitHub release APKs (never auto-update) 1,682 lifetime APK downloads across all releases 25.9% (436 downloads) are pre-v0.4.8 builds.

There is no iOS App Store listing (itunes.apple.com/lookup?bundleId=cc.agentlabs.opencode returns 0 results) and the app is not on IzzyOnDroid, so those channels contribute nothing.

What that means

  1. Play is not the problem. Auto-update did its job: the measurable Android active base is effectively 100% on a build that can sign up through the API.
  2. The stale cohort is sideload-only and permanent. ~436 devices pulled an APK that predates the signup API. Nothing will ever update them; they will keep emitting mailto signups until the owner manually re-downloads. The hourly reconciler (VibeBrowserProductPage/.github/workflows/waitlist-mailto-reconcile.yml) is what keeps those signups from being lost, and it is not a temporary measure.
  3. Stale builds are not the only source of mailto signups. In current builds the fallback also fires on network error, timeout (8s) and 5xx (src/lib/waitlist.ts:shouldFallbackToMailto). A user on v0.4.12 with flaky mobile data takes the same lossy path. Any plan that assumes "ship an update and the leak closes" is wrong.

Method / reproducing

GOOGLE_PLAY_SERVICE_ACCOUNT_JSON="$(cat play-store-key.json)" \
  node scripts/play-version-share.mjs --days 14 --github
  • Play numbers come from the Play Developer Reporting API, crashRateMetricSet → distinctUsers grouped by versionCode. That is Play's own vitals denominator: users who actually opened the app. It only covers devices with usage-and-diagnostics sharing on, and Play rounds it — which is why we quote a share, never an absolute install count.
  • Play versionCodes are not the ones in android/app/build.gradle: the publish workflow overwrites them with github.run_number + 100. The mapping in the script was derived from the successful runs of publish-play-store.yml (versionCode 139 = v0.4.8 = first build with the API path).
  • The service account is the same PLAY_STORE_SERVICE_ACCOUNT_JSON already used to publish; the script needs only the playdeveloperreporting scope and mints its own token, no extra deps.