feat: add daily product intelligence and versioned site assets (#64)
Adds privacy-safe aggregate product intelligence, reviewed/versioned website assets, and a dispatch-only rollout until the dedicated Sentry token is verified. Independent review blockers were fixed in 8bc47e4; app checks, website production build, Android CI, and iOS CI are green. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This commit is contained in:
73
docs/prd.md
73
docs/prd.md
@@ -119,3 +119,76 @@ git push origin main --tags
|
||||
- Multi-user / shared sessions.
|
||||
- Voice input beyond what the OS dictation keyboard provides.
|
||||
- iPad / tablet-optimised layout (phone-first only; tablet works but is not designed for).
|
||||
|
||||
## 8. Continuous Product Intelligence
|
||||
|
||||
### Problem
|
||||
|
||||
At 1,000 downloads, acquisition, reliability, feedback, and visual-conversion
|
||||
signals are fragmented across Sentry, GitHub, Google Play, and Vercel. The app
|
||||
has opted-in diagnostics but no daily operating loop that turns aggregate,
|
||||
privacy-safe evidence into a small, prioritized queue. The public Vercel site
|
||||
also deploys from an untracked Next.js working directory, so its screenshots
|
||||
cannot be traced to a reviewed source commit.
|
||||
|
||||
### Scope
|
||||
|
||||
- A daily GitHub Action, triggered only by `schedule` and maintainer
|
||||
`workflow_dispatch`, that publishes an aggregate-only product-intelligence
|
||||
report to the Actions summary and artifact.
|
||||
- Rollout starts with maintainer dispatch only. Enable the daily schedule after
|
||||
the dedicated read-only Sentry token succeeds against production.
|
||||
- A material-signal issue upsert: create or update a GitHub issue only when a
|
||||
defined threshold is crossed, never one issue per uneventful day.
|
||||
- A metric contract covering acquisition, activation, reliability, retention,
|
||||
feedback, and visual conversion, with explicit data sources and gaps.
|
||||
- Sentry diagnostic payloads that match the consent copy: no server address,
|
||||
port, URL, credential, code, prompt, session title, or directory leaves the
|
||||
device.
|
||||
- Versioned Play screenshots, checksums, provenance, and intended website
|
||||
placement in this repository.
|
||||
- A versioned Next.js source for the Vercel site, replacing its untracked
|
||||
working copy before future visual changes.
|
||||
|
||||
### Non-Goals
|
||||
|
||||
- Automatic code changes, merges, releases, deployments, or Play Console edits
|
||||
from a daily report.
|
||||
- Sending raw Sentry issue text, review body text, reviewer names, device
|
||||
details, or user-generated content to a GitHub issue or Actions artifact.
|
||||
- Treating a download count as active-user, activation, or retention evidence.
|
||||
- Adding product-usage analytics without a separate explicit consent,
|
||||
disclosure, and Data Safety review.
|
||||
|
||||
### Metric Contract
|
||||
|
||||
| Decision | Metric | Current source | Gap / next step |
|
||||
| --- | --- | --- | --- |
|
||||
| Is acquisition growing? | Play listing visitors, installs, uninstall rate, release APK downloads, repo views, stars | Play Console; GitHub Releases and Traffic APIs | Automate GitHub now; add read-only Play reporting only after access is verified. |
|
||||
| Do new users reach value? | Connection success, time to first connection, sessions loaded, first prompt, first response | Sentry records opted-in failures only | Design explicit opt-in aggregate activation measurement before implementation. |
|
||||
| Is app reliable? | Crash-free sessions, unresolved/new/regressed Sentry issues, error volume by release, connection classification, CUA pass rate | Sentry Release Health / Issues; GitHub Actions | Daily report initially aggregates Sentry Issues only; add Release Health after its aggregate query is verified. |
|
||||
| Do users return? | 7-day / 30-day returning active installations | None | Do not infer this. Evaluate a privacy-reviewed aggregate telemetry design. |
|
||||
| What should be fixed next? | Low-rating review themes, GitHub issue themes, issue age, CI failures | Android Publisher reviews script; GitHub Issues; Actions | Repair review ingestion and dedupe before enabling it. |
|
||||
| Does site convert? | Unique visitors, install/beta CTA clicks, screenshot engagement, listing conversion | Vercel Analytics; Play Console | Define aggregate event names and exports; do not fingerprint visitors. |
|
||||
|
||||
### Acceptance Criteria
|
||||
|
||||
- [ ] A manual or scheduled daily run creates or updates exactly one report for
|
||||
its UTC day in its Actions summary/artifact, exposes source freshness, and
|
||||
distinguishes unavailable data from zero.
|
||||
- [ ] The first report populates only GitHub Traffic, Releases, Issues, and
|
||||
Actions plus Sentry Issues aggregates. Release Health, activation, retention,
|
||||
Play, review-theme, and site-conversion rows appear as `deferred` with their
|
||||
reason, never zero or blank.
|
||||
- [ ] Scheduled runs create a GitHub issue only for a documented material
|
||||
threshold; otherwise they update no public issue and require no daily manual
|
||||
review.
|
||||
- [ ] No Sentry payload of any kind - event, breadcrumb, tag, context,
|
||||
transaction/span name, or `serverName` - contains a server host, port, URL,
|
||||
credential, prompt, code, session title, or directory.
|
||||
- [ ] Seven full-resolution Play screenshots are versioned with source URL,
|
||||
retrieval date, SHA-256, dimensions, and website placement.
|
||||
- [ ] `https://opencode.agentlabs.cc` serves the approved screenshots from
|
||||
versioned source after its Vercel deployment.
|
||||
- [ ] Material signals become implementation issues only after a maintainer
|
||||
reproduces them in the affected user channel.
|
||||
|
||||
71
docs/tdd.md
71
docs/tdd.md
@@ -182,3 +182,74 @@ A test for this would feed a basic-auth URL into a fake event and assert the scr
|
||||
- **No fallback UI for SSE disconnect.** Today the user sees the existing chat with a (small) banner; a more deliberate "Reconnecting…" affordance would help.
|
||||
- **Silent `.catch(() => null)` in stores.** Intentional today (these are non-critical fetches), but should be revisited once we have proper severity tiers for breadcrumbs.
|
||||
- **PRD analytics.** No usage analytics; only crash telemetry. A future opt-in product-analytics provider could close that loop without compromising the privacy posture.
|
||||
|
||||
## 10. Continuous Product Intelligence
|
||||
|
||||
```text
|
||||
Maintainer manual dispatch
|
||||
|
|
||||
v
|
||||
scripts/product-intelligence.mjs
|
||||
| GitHub REST: releases, traffic, issues, Actions
|
||||
| Sentry REST: aggregate unresolved/new issue health
|
||||
| (future) Android Publisher: aggregate review and Play signals
|
||||
|
|
||||
v
|
||||
sanitized Markdown report + JSON evidence
|
||||
|
|
||||
v
|
||||
Actions summary + artifact per UTC day
|
||||
|
|
||||
v
|
||||
material threshold -> deduplicated GitHub issue -> reviewed PR
|
||||
```
|
||||
|
||||
### Components
|
||||
|
||||
| Component | Location | Purpose |
|
||||
| --- | --- | --- |
|
||||
| Product-intelligence workflow | `.github/workflows/product-intelligence.yml` | Starts with maintainer dispatch only. Enable its cron after the dedicated Sentry token passes a production run. Uses `actions: read`, `contents: read`, and `issues: write`. |
|
||||
| Collector | `scripts/product-intelligence.mjs` | Validates sources, collects aggregate signals, renders a sanitized report, and fails visibly when a source is unavailable. |
|
||||
| Material issue upsert | Workflow `actions/github-script` | Locates a stable signal fingerprint and creates or updates an issue only after a material threshold. |
|
||||
| Sentry privacy implementation | `src/lib/sentry.ts`, `src/lib/scrub.ts` | Separate P1 prerequisite: remove user-controlled server data from events, breadcrumbs, tags, contexts, and span names before an event leaves device. |
|
||||
| Screenshot source of truth | `distribution/reference-screenshots/play-v1/manifest.json` | Records Play asset provenance, checksum, dimensions, and page placement. |
|
||||
| Vercel website | `website/` | Versioned Next.js source deployed to existing `opencode-mobile-site` project. |
|
||||
|
||||
### Data and Privacy Boundaries
|
||||
|
||||
- The initial collector contains numeric aggregates only. It must not publish
|
||||
Sentry issue titles, exception text, culprits, request URL, device metadata,
|
||||
raw review content, author names, or user-generated text.
|
||||
- The material-issue upsert independently reconstructs its body from an
|
||||
allowlisted UTC date and allowlisted signal names. It rejects malformed
|
||||
report data and never uses collector-provided Markdown, URLs, or Sentry
|
||||
response fields.
|
||||
- A missing credential or failed request is reported as unavailable and exits
|
||||
non-zero; it must never become a success-shaped zero.
|
||||
- The first report supports only GitHub and Sentry aggregates. Every other
|
||||
metric-contract row is rendered as `deferred` with its source/access reason.
|
||||
- The first delivery introduces no new analytics SDK or user identifier.
|
||||
- Sentry remains explicitly opt-in. The consent UI, Settings UI, privacy
|
||||
policy, and runtime scrubber must agree on what leaves the device.
|
||||
- Future activation and retention metrics require a separate product decision
|
||||
that updates consent language and Play Data Safety before code ships.
|
||||
- The `SENTRY_PRODUCT_INTELLIGENCE_TOKEN` repository secret is a read-only,
|
||||
single-project Sentry token with only `project:read`, `event:read`, and
|
||||
`org:read` scopes; it is distinct from release-upload credentials. Any
|
||||
future Android Publisher credential is read-only and limited to
|
||||
reporting/reviews.
|
||||
|
||||
### Rollout
|
||||
|
||||
1. Provision the dedicated read-only Sentry repository secret, then manually
|
||||
run the daily workflow against production credentials and verify a
|
||||
sanitized Actions summary and artifact for its UTC date.
|
||||
2. Enable the cron, then observe one scheduled run and verify it creates a new UTC-day artifact but
|
||||
no public issue unless a documented material threshold is crossed.
|
||||
Back-to-back manual and scheduled runs must update at most one open
|
||||
material-signal issue.
|
||||
3. Merge the Sentry privacy boundary before exposing Sentry detail links.
|
||||
4. Import and manifest Play screenshots, deploy versioned web source, and
|
||||
verify every screenshot loads at the public domain.
|
||||
5. Enable Android Publisher review/Play data collection only after
|
||||
least-privilege credentials and real-data validation are complete.
|
||||
|
||||
Reference in New Issue
Block a user