Files
opencode-mobile/.github/workflows/activation-e2e.yml
Den 2da0fdf109 test: E2E coverage for directory picker, all-sessions, variant picker. Refs #46 #48 #47 #49 #57. (#82)
* test: E2E coverage for directory picker, all-sessions, variant picker

Extend the Maestro suite for the features merged into main today:
DirectoryBrowserSheet's server-folder picker, the directory-less
all-sessions-across-projects list (+ the #46/#48 open-across-project
regression), and VariantPicker's reasoning-effort chip.

- tests/fixtures/mock-opencode-server.ts: GET /file (directory-scoped via
  the x-opencode-directory header) with a small fake tree, GET /project
  for the "Server Projects" section, POST /session honoring the directory
  header, GET /session/:id (needed to open a session from the all-sessions
  list), GET /provider variants for VariantPicker, and an optional
  --seed-sessions mode that pre-populates two sessions across two
  directories. --fail-auth mode is untouched.
- .maestro/flows/directory-picker.yaml, all-sessions.yaml,
  variant-picker.yaml: three new flows, run in the same emulator session
  as the existing activation flows.
- Additive testIDs on DirectoryBrowserSheet, the "Browse Folders" row,
  session list rows, the variant chip, and VariantPicker rows.
- .github/workflows/activation-e2e.yml: two more mock server instances
  (4098 seeded, 4099 fresh) and three more maestro test steps.

Verified: tsc --noEmit clean, all 108 existing unit tests pass, every new
mock endpoint curled against its real shape read from the app code, YAML
validated. No Android emulator available locally to run the Maestro flows
themselves.

* test(mock): enforce per-directory session scoping so #46/#48 coverage can fail

Review finding (HIGH): GET /session/:id and /session/:id/message ignored
x-opencode-directory, so all-sessions.yaml could not fail if the directory
threading fix regressed. The mock now mirrors the real server's per-directory
workspace scoping:

- GET /session/:id and GET /session/:id/message 404 unless the request's
  x-opencode-directory (or DEFAULT_DIRECTORY when absent) matches the stored
  session's directory.
- GET /session without ?roots=true is scoped to the request's directory;
  loadSessions()'s directory-less roots=true call still returns everything.
- Document the port-4099 shared-state coupling between directory-picker and
  variant-picker flows, and why all-sessions.yaml now has teeth (flow comment).

Curl-verified: correct header 200, wrong/no header 404, scoped vs roots
listing, create-then-open paths for all three flows, --fail-auth untouched.
tsc clean, 108/108 unit tests pass.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NJKAQ6HAikWGQK7PGZ5Y4E

* fix(e2e): widen connect-handshake wait past client's own 30s timeout

Run 29546383612 (7cbd3a6, first real emulator execution of these flows)
failed on activation-positive.yaml: "Assert that id: connection-status-dot
is visible" timed out after the flow's 20s extendedWaitUntil, right after
tapOn connect-submit-button.

The mock server itself is fast (verified locally: health + project/current
+ path respond in ~30ms total), so this isn't a mock fidelity gap. But
Quick Connect's testConnection()/addConnection() path chains up to 3
fetches (health, then project.current + path.get in parallel), and each
individual fetch is capped by src/lib/sdk.ts REQUEST_TIMEOUT_MS = 30_000 —
strictly longer than the 20s the flow was willing to wait. A first-attempt
emulator-to-host (10.0.2.2) connection that's merely slow to establish,
rather than outright failing, would blow past the test's wait before the
app's own client-side timeout even fires.

Bump the connect -> connection-status-dot / "Connection Failed" waits from
20000 to 40000 across all 5 flows that share this pattern
(activation-positive, activation-negative-401, all-sessions,
directory-picker, variant-picker) so the wait is never shorter than the
code path it's gating on. Assertions are unchanged — still requires the
real dot / real error text, just with a timeout that isn't racing the
client.

Verified locally: typecheck clean, all 108 unit tests pass, YAML parses,
mock server confirmed fast under direct curl. Emulator behavior itself
(whether 40s consistently clears it) is unverified until the next CI run.

* fix(e2e): use adb reverse + 127.0.0.1 instead of 10.0.2.2; capture logcat/maestro debug

Root cause of the activation-e2e failure (connect step timed out, ~0 requests
reaching the mock): the 10.0.2.2 host alias is unreliable under the headless
emulator-runner — the app's http://10.0.2.2:4096/global/health never completed,
so connection-status-dot never rendered.

- run-e2e-flows.sh: single script (fixes cd-per-line fragility) that adb-reverses
  each mock port (4096-4099) into the emulator's localhost, runs every flow with
  --debug-output, and dumps logcat on exit.
- All flows now connect to 127.0.0.1:<port> (the adb reverse target).
- Upload maestro-debug (UI hierarchy on failure) + logcat as artifacts so future
  failures are diagnosable instead of blind.

* fix(e2e): connect via 127.0.0.1:PORT in IP field, stop typing into port input

Root cause of every activation-e2e connect failure (proven by the app's own
logcat diagnostic: '[diag] probe start http://127.0.0.1:40966 ... server
unreachable'): the port field defaults to useState("4096"), and the flow's
eraseText + inputText "4096" raced the controlled number-pad input, leaving
"40966" — nothing listens there, so connect always failed. This was never a
10.0.2.2 / adb reverse issue.

Fix: buildUrl already extracts host:port from the IP field, so enter
127.0.0.1:<port> there and remove the flaky port-field steps entirely.
pastedPort overrides the default port state, so each flow's port is
deterministic (4096 positive / 4097 negative / 4098 all-sessions / 4099
directory+variant).

---------

Co-authored-by: engineer <engineer@gray-knight-m1.local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-17 02:55:25 -07:00

187 lines
8.0 KiB
YAML

name: Activation E2E (Maestro)
# Deterministic regression coverage for the activation flow (first open ->
# telemetry consent -> server URL entry -> connect -> send first message ->
# receive reply), including the connect-time-401 negative case tied to the
# 0%-7-day-retention / GitHub issue #76 investigation.
#
# Also covers newer surfaces merged after the initial activation suite:
# DirectoryBrowserSheet's server-folder picker, the directory-less
# "all sessions across all projects" list (+ the #46/#48 open-across-project
# regression), and VariantPicker's reasoning-effort chip. See
# .maestro/flows/directory-picker.yaml, all-sessions.yaml, variant-picker.yaml.
#
# Runs against tests/fixtures/mock-opencode-server.ts (a small dependency-free
# HTTP+SSE stub matching the REAL client protocol read from src/lib/sdk.ts —
# NOT a live opencode server, NOT a WebSocket), so the suite is fast and fully
# self-contained: no external server, no LLM provider, no network flakiness.
#
# This is intentionally separate from cua-smoke.yml (the existing
# vision-driven CUA harness): that one needs a live opencode server + an Azure
# LLM and is exploratory/non-deterministic by design, so it isn't suited to
# tight regression assertions like "a 401 must show a visible error."
on:
push:
branches: [main]
paths:
- "app/**"
- "src/**"
- "tests/fixtures/**"
- ".maestro/**"
- ".github/workflows/activation-e2e.yml"
pull_request:
branches: [main]
paths:
- "app/**"
- "src/**"
- "tests/fixtures/**"
- ".maestro/**"
- ".github/workflows/activation-e2e.yml"
workflow_dispatch: {}
jobs:
activation-e2e:
runs-on: ubuntu-latest
# Non-blocking until the suite has its first green run: the positive flow
# still fails its final reply assertion (mode B in issue #90). Remove once
# #90 is closed so regressions block PRs again.
continue-on-error: true
# Same npm install + expo prebuild + assembleRelease + emulator pipeline as
# cua-smoke.yml, which budgets 60 min (emulator-boot-timeout alone is 10 min).
# Typical runs finish well under this; the ceiling just avoids flaky kills
# on cold Gradle caches.
timeout-minutes: 60
steps:
- uses: actions/checkout@v6
- uses: actions/setup-node@v6
with:
node-version: 24
cache: npm
- uses: actions/setup-java@v5
with:
distribution: temurin
java-version: 17
- name: Setup Android SDK
uses: android-actions/setup-android@v4
- name: Add emulator to PATH
run: echo "$ANDROID_HOME/emulator" >> $GITHUB_PATH
- name: Enable KVM
run: |
echo 'KERNEL=="kvm", GROUP="kvm", MODE="0666", OPTIONS+="static_node=kvm"' | sudo tee /etc/udev/rules.d/99-kvm4all.rules
sudo udevadm control --reload-rules
sudo udevadm trigger --name-match=kvm
- name: Cache Gradle
uses: actions/cache@v5
with:
path: |
~/.gradle/caches
~/.gradle/wrapper
android/.gradle
key: ${{ runner.os }}-gradle-${{ hashFiles('android/**/*.gradle*', 'android/gradle/wrapper/gradle-wrapper.properties') }}
restore-keys: |
${{ runner.os }}-gradle-
- name: Purge stale generated sources
# Same mitigation as cua-smoke.yml (whose Gradle cache entries this job
# shares — the cache key/restore-keys are identical): the restore-keys
# prefix fallback can restore a generated autolinking tree from a
# previous package id, making compileReleaseJavaWithJavac fail against
# the old package (ai.opencode.mobile vs cc.agentlabs.opencode). Delete
# generated sources so prebuild + Gradle regenerate them.
run: rm -rf android/app/build/generated android/build/generated android/app/build/intermediates
- name: Install Maestro CLI
run: |
curl -Ls "https://get.maestro.mobile.dev" | bash
echo "$HOME/.maestro/bin" >> $GITHUB_PATH
- name: Install dependencies & build APK
env:
SENTRY_DISABLE_AUTO_UPLOAD: "true"
run: |
npm install --legacy-peer-deps
npx expo prebuild --platform android --no-install
keytool -genkey -v -keystore android/app/debug.keystore -storepass android -alias androiddebugkey -keypass android -keyalg RSA -keysize 2048 -validity 10000 -dname "CN=Android Debug,O=Android,C=US"
cd android && ./gradlew assembleRelease
- name: Start mock opencode servers
run: |
set -x
mkdir -p artifacts/screenshots
# Normal mode on 4096 (positive flow) and --fail-auth on 4097 (negative flow).
# 4098 is normal mode + --seed-sessions (two pre-populated sessions in two
# different directories, for all-sessions.yaml). 4099 is a fresh normal-mode
# instance shared by directory-picker.yaml and variant-picker.yaml (its fake
# file tree / provider variants / project list don't affect each other).
# All bind 0.0.0.0 so the emulator can reach them via 10.0.2.2.
nohup node tests/fixtures/mock-opencode-server.ts --port 4096 > /tmp/mock-4096.log 2>&1 &
nohup node tests/fixtures/mock-opencode-server.ts --port 4097 --fail-auth > /tmp/mock-4097.log 2>&1 &
nohup node tests/fixtures/mock-opencode-server.ts --port 4098 --seed-sessions > /tmp/mock-4098.log 2>&1 &
nohup node tests/fixtures/mock-opencode-server.ts --port 4099 > /tmp/mock-4099.log 2>&1 &
for port in 4096 4097 4098 4099; do
for i in $(seq 1 30); do
if curl -sf --connect-timeout 1 -m 3 "http://127.0.0.1:${port}/global/health" > /dev/null 2>&1; then
echo "mock server on ${port} responded (may be 401, that's expected on 4097)"
break
fi
# 4097 always 401s -> curl -sf treats that as failure, so also accept "connection made"
if curl -s --connect-timeout 1 -m 3 -o /dev/null -w '%{http_code}' "http://127.0.0.1:${port}/global/health" 2>/dev/null | grep -qE '^[0-9]+$'; then
echo "mock server on ${port} is up (got an HTTP response)"
break
fi
if [ "$i" = "30" ]; then
echo "::error::mock server on port ${port} did not come up in 30s"
cat "/tmp/mock-${port}.log" || true
exit 1
fi
sleep 1
done
done
- name: Run activation E2E flows on emulator
uses: reactivecircus/android-emulator-runner@v2
with:
api-level: 28
arch: x86_64
target: default
disable-animations: true
emulator-boot-timeout: 600
emulator-options: -no-window -no-audio -no-boot-anim -gpu swiftshader_indirect -no-snapshot
# One script file, not one `sh -c` per line — run-e2e-flows.sh does the
# adb reverse port-forwarding, runs every flow, and captures logcat on
# exit. (Running maestro line-by-line here broke `cd` persistence and
# left no diagnostics.)
script: bash scripts/run-e2e-flows.sh
- name: Mock server logs
if: always()
run: |
echo "--- 4096 (normal) ---"; cat /tmp/mock-4096.log || true
echo "--- 4097 (fail-auth) ---"; cat /tmp/mock-4097.log || true
echo "--- 4098 (seed-sessions) ---"; cat /tmp/mock-4098.log || true
echo "--- 4099 (normal, directory-picker + variant-picker) ---"; cat /tmp/mock-4099.log || true
- name: Upload screenshots
if: always()
uses: actions/upload-artifact@v7
with:
name: activation-e2e-screenshots-${{ github.run_number }}
path: artifacts/screenshots/*.png
if-no-files-found: warn
- name: Upload maestro debug + logcat
if: always()
uses: actions/upload-artifact@v7
with:
name: activation-e2e-debug-${{ github.run_number }}
path: |
artifacts/diag/**
if-no-files-found: warn