feat: add daily product intelligence and versioned site assets (#64)

Adds privacy-safe aggregate product intelligence, reviewed/versioned website assets, and a dispatch-only rollout until the dedicated Sentry token is verified. Independent review blockers were fixed in 8bc47e4; app checks, website production build, Android CI, and iOS CI are green.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This commit is contained in:
Den
2026-07-15 11:59:40 -07:00
committed by GitHub
parent cbe00c3222
commit 5c14ce0a5d
46 changed files with 4539 additions and 1 deletions

View File

@@ -119,3 +119,76 @@ git push origin main --tags
- Multi-user / shared sessions.
- Voice input beyond what the OS dictation keyboard provides.
- iPad / tablet-optimised layout (phone-first only; tablet works but is not designed for).
## 8. Continuous Product Intelligence
### Problem
At 1,000 downloads, acquisition, reliability, feedback, and visual-conversion
signals are fragmented across Sentry, GitHub, Google Play, and Vercel. The app
has opted-in diagnostics but no daily operating loop that turns aggregate,
privacy-safe evidence into a small, prioritized queue. The public Vercel site
also deploys from an untracked Next.js working directory, so its screenshots
cannot be traced to a reviewed source commit.
### Scope
- A daily GitHub Action, triggered only by `schedule` and maintainer
`workflow_dispatch`, that publishes an aggregate-only product-intelligence
report to the Actions summary and artifact.
- Rollout starts with maintainer dispatch only. Enable the daily schedule after
the dedicated read-only Sentry token succeeds against production.
- A material-signal issue upsert: create or update a GitHub issue only when a
defined threshold is crossed, never one issue per uneventful day.
- A metric contract covering acquisition, activation, reliability, retention,
feedback, and visual conversion, with explicit data sources and gaps.
- Sentry diagnostic payloads that match the consent copy: no server address,
port, URL, credential, code, prompt, session title, or directory leaves the
device.
- Versioned Play screenshots, checksums, provenance, and intended website
placement in this repository.
- A versioned Next.js source for the Vercel site, replacing its untracked
working copy before future visual changes.
### Non-Goals
- Automatic code changes, merges, releases, deployments, or Play Console edits
from a daily report.
- Sending raw Sentry issue text, review body text, reviewer names, device
details, or user-generated content to a GitHub issue or Actions artifact.
- Treating a download count as active-user, activation, or retention evidence.
- Adding product-usage analytics without a separate explicit consent,
disclosure, and Data Safety review.
### Metric Contract
| Decision | Metric | Current source | Gap / next step |
| --- | --- | --- | --- |
| Is acquisition growing? | Play listing visitors, installs, uninstall rate, release APK downloads, repo views, stars | Play Console; GitHub Releases and Traffic APIs | Automate GitHub now; add read-only Play reporting only after access is verified. |
| Do new users reach value? | Connection success, time to first connection, sessions loaded, first prompt, first response | Sentry records opted-in failures only | Design explicit opt-in aggregate activation measurement before implementation. |
| Is app reliable? | Crash-free sessions, unresolved/new/regressed Sentry issues, error volume by release, connection classification, CUA pass rate | Sentry Release Health / Issues; GitHub Actions | Daily report initially aggregates Sentry Issues only; add Release Health after its aggregate query is verified. |
| Do users return? | 7-day / 30-day returning active installations | None | Do not infer this. Evaluate a privacy-reviewed aggregate telemetry design. |
| What should be fixed next? | Low-rating review themes, GitHub issue themes, issue age, CI failures | Android Publisher reviews script; GitHub Issues; Actions | Repair review ingestion and dedupe before enabling it. |
| Does site convert? | Unique visitors, install/beta CTA clicks, screenshot engagement, listing conversion | Vercel Analytics; Play Console | Define aggregate event names and exports; do not fingerprint visitors. |
### Acceptance Criteria
- [ ] A manual or scheduled daily run creates or updates exactly one report for
its UTC day in its Actions summary/artifact, exposes source freshness, and
distinguishes unavailable data from zero.
- [ ] The first report populates only GitHub Traffic, Releases, Issues, and
Actions plus Sentry Issues aggregates. Release Health, activation, retention,
Play, review-theme, and site-conversion rows appear as `deferred` with their
reason, never zero or blank.
- [ ] Scheduled runs create a GitHub issue only for a documented material
threshold; otherwise they update no public issue and require no daily manual
review.
- [ ] No Sentry payload of any kind - event, breadcrumb, tag, context,
transaction/span name, or `serverName` - contains a server host, port, URL,
credential, prompt, code, session title, or directory.
- [ ] Seven full-resolution Play screenshots are versioned with source URL,
retrieval date, SHA-256, dimensions, and website placement.
- [ ] `https://opencode.agentlabs.cc` serves the approved screenshots from
versioned source after its Vercel deployment.
- [ ] Material signals become implementation issues only after a maintainer
reproduces them in the affected user channel.

View File

@@ -182,3 +182,74 @@ A test for this would feed a basic-auth URL into a fake event and assert the scr
- **No fallback UI for SSE disconnect.** Today the user sees the existing chat with a (small) banner; a more deliberate "Reconnecting…" affordance would help.
- **Silent `.catch(() => null)` in stores.** Intentional today (these are non-critical fetches), but should be revisited once we have proper severity tiers for breadcrumbs.
- **PRD analytics.** No usage analytics; only crash telemetry. A future opt-in product-analytics provider could close that loop without compromising the privacy posture.
## 10. Continuous Product Intelligence
```text
Maintainer manual dispatch
|
v
scripts/product-intelligence.mjs
| GitHub REST: releases, traffic, issues, Actions
| Sentry REST: aggregate unresolved/new issue health
| (future) Android Publisher: aggregate review and Play signals
|
v
sanitized Markdown report + JSON evidence
|
v
Actions summary + artifact per UTC day
|
v
material threshold -> deduplicated GitHub issue -> reviewed PR
```
### Components
| Component | Location | Purpose |
| --- | --- | --- |
| Product-intelligence workflow | `.github/workflows/product-intelligence.yml` | Starts with maintainer dispatch only. Enable its cron after the dedicated Sentry token passes a production run. Uses `actions: read`, `contents: read`, and `issues: write`. |
| Collector | `scripts/product-intelligence.mjs` | Validates sources, collects aggregate signals, renders a sanitized report, and fails visibly when a source is unavailable. |
| Material issue upsert | Workflow `actions/github-script` | Locates a stable signal fingerprint and creates or updates an issue only after a material threshold. |
| Sentry privacy implementation | `src/lib/sentry.ts`, `src/lib/scrub.ts` | Separate P1 prerequisite: remove user-controlled server data from events, breadcrumbs, tags, contexts, and span names before an event leaves device. |
| Screenshot source of truth | `distribution/reference-screenshots/play-v1/manifest.json` | Records Play asset provenance, checksum, dimensions, and page placement. |
| Vercel website | `website/` | Versioned Next.js source deployed to existing `opencode-mobile-site` project. |
### Data and Privacy Boundaries
- The initial collector contains numeric aggregates only. It must not publish
Sentry issue titles, exception text, culprits, request URL, device metadata,
raw review content, author names, or user-generated text.
- The material-issue upsert independently reconstructs its body from an
allowlisted UTC date and allowlisted signal names. It rejects malformed
report data and never uses collector-provided Markdown, URLs, or Sentry
response fields.
- A missing credential or failed request is reported as unavailable and exits
non-zero; it must never become a success-shaped zero.
- The first report supports only GitHub and Sentry aggregates. Every other
metric-contract row is rendered as `deferred` with its source/access reason.
- The first delivery introduces no new analytics SDK or user identifier.
- Sentry remains explicitly opt-in. The consent UI, Settings UI, privacy
policy, and runtime scrubber must agree on what leaves the device.
- Future activation and retention metrics require a separate product decision
that updates consent language and Play Data Safety before code ships.
- The `SENTRY_PRODUCT_INTELLIGENCE_TOKEN` repository secret is a read-only,
single-project Sentry token with only `project:read`, `event:read`, and
`org:read` scopes; it is distinct from release-upload credentials. Any
future Android Publisher credential is read-only and limited to
reporting/reviews.
### Rollout
1. Provision the dedicated read-only Sentry repository secret, then manually
run the daily workflow against production credentials and verify a
sanitized Actions summary and artifact for its UTC date.
2. Enable the cron, then observe one scheduled run and verify it creates a new UTC-day artifact but
no public issue unless a documented material threshold is crossed.
Back-to-back manual and scheduled runs must update at most one open
material-signal issue.
3. Merge the Sentry privacy boundary before exposing Sentry detail links.
4. Import and manifest Play screenshots, deploy versioned web source, and
verify every screenshot loads at the public domain.
5. Enable Android Publisher review/Play data collection only after
least-privilege credentials and real-data validation are complete.