test(activation): add Maestro E2E coverage for the activation flow
Adds deterministic end-to-end coverage for first-open -> telemetry consent -> server URL entry -> connect -> send first message -> receive reply, targeting the 0%-7-day-retention investigation (GitHub issue #76). - tests/fixtures/mock-opencode-server.ts: dependency-free HTTP+SSE stub matching the REAL client protocol (src/lib/sdk.ts) — REST + a single long-lived GET /global/event SSE stream, no WebSocket. Supports a --fail-auth mode that 401s every request to exercise the connect-time auth-failure class. - .maestro/flows/activation-positive.yaml: consent -> quick connect -> new session -> send message -> assert streamed reply renders, with a screenshot at every step (positive-S1..S8). - .maestro/flows/activation-negative-401.yaml: same setup against the --fail-auth server, asserts Quick Connect's existing "Connection Failed" alert is shown (not silently swallowed) and that the connection is not saved. Flags in comments that Advanced-mode Save (handleAdvancedSave) still has no testConnection() check and is a known, uncovered gap. - testID props added (no restructuring) to the screens/components the flows drive: TelemetryConsentModal, connection/add.tsx, tabs/index.tsx, session/[id].tsx, MessageBubble. - .github/workflows/activation-e2e.yml: new CI job — Android emulator via reactivecircus/android-emulator-runner, builds the debug-signed APK, starts both mock server instances, runs both Maestro flows, uploads screenshots via actions/upload-artifact. Kept separate from the existing vision-driven cua-smoke.yml, which needs a live server + LLM and isn't suited to tight deterministic regression assertions. - .gitignore: Maestro takeScreenshot output is never committed. Verified locally: mock server exercised standalone via curl (health, project/current, path, session create, SSE event ordering, message persistence) in both normal and --fail-auth modes; both Maestro flow files validated as well-formed YAML; tsc --noEmit clean on all changed files. No lint script exists in this repo (N/A). Full emulator execution was not run — no Android SDK/emulator available in this environment. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NJKAQ6HAikWGQK7PGZ5Y4E
This commit is contained in:
68
.maestro/flows/activation-negative-401.yaml
Normal file
68
.maestro/flows/activation-negative-401.yaml
Normal file
@@ -0,0 +1,68 @@
|
||||
appId: cc.agentlabs.opencode
|
||||
name: Activation - negative path (connect-time 401 must surface a visible error)
|
||||
---
|
||||
# Negative activation flow — regression test for the GitHub issue #76 failure
|
||||
# class: a connect-time 401 from the server must produce a visible, actionable
|
||||
# error, never a silent failure.
|
||||
#
|
||||
# Quick Connect (app/connection/add.tsx handleQuickConnect) calls
|
||||
# testConnection() -> client.global.health() BEFORE saving the connection.
|
||||
# When that call fails (any thrown error, including a 401 API error), the
|
||||
# screen runs probeConnection() and shows a native Alert titled
|
||||
# "Connection Failed" with the underlying error text — this is the CURRENT,
|
||||
# already-correct behavior we are locking in with this test.
|
||||
#
|
||||
# Known gap (see report): Advanced-mode "Save Connection" (handleAdvancedSave)
|
||||
# does NOT call testConnection() at all — it saves the connection and
|
||||
# navigates back regardless of server reachability, so a 401 there is
|
||||
# currently silent. That gap is not covered by this flow (Quick Connect is the
|
||||
# only entry point that already does the right thing) — flagged in the E2E
|
||||
# report as follow-up work, not silently fixed here.
|
||||
#
|
||||
# The CI job starts `node tests/fixtures/mock-opencode-server.ts --port 4097
|
||||
# --fail-auth` on the runner host BEFORE this flow runs, so every request the
|
||||
# app makes to it (including /global/health) returns HTTP 401.
|
||||
|
||||
- launchApp:
|
||||
clearState: true
|
||||
- assertVisible:
|
||||
id: "telemetry-consent-card"
|
||||
- tapOn:
|
||||
id: "telemetry-decline-button"
|
||||
- takeScreenshot: negative-S1_consent_dismissed
|
||||
|
||||
- assertVisible:
|
||||
text: "No Connection"
|
||||
- tapOn:
|
||||
id: "add-connection-button"
|
||||
- takeScreenshot: negative-S2_add_connection_form
|
||||
|
||||
- tapOn:
|
||||
id: "connect-ip-input"
|
||||
- inputText: "10.0.2.2"
|
||||
- tapOn:
|
||||
id: "connect-port-input"
|
||||
- eraseText
|
||||
- inputText: "4097"
|
||||
- hideKeyboard
|
||||
- takeScreenshot: negative-S3_401_server_url_entered
|
||||
- tapOn:
|
||||
id: "connect-submit-button"
|
||||
|
||||
- extendedWaitUntil:
|
||||
visible:
|
||||
text: "Connection Failed"
|
||||
timeout: 20000
|
||||
- assertVisible:
|
||||
text: "Connection Failed"
|
||||
- assertVisible:
|
||||
text: "401"
|
||||
- takeScreenshot: negative-S4_visible_error_alert
|
||||
|
||||
# The connection must NOT have been silently saved: dismiss the alert and
|
||||
# confirm we are still on the empty/no-connection state, not a fake
|
||||
# "connected" screen.
|
||||
- tapOn: "OK"
|
||||
- assertVisible:
|
||||
text: "No Connection"
|
||||
- takeScreenshot: negative-S5_still_disconnected_after_dismiss
|
||||
74
.maestro/flows/activation-positive.yaml
Normal file
74
.maestro/flows/activation-positive.yaml
Normal file
@@ -0,0 +1,74 @@
|
||||
appId: cc.agentlabs.opencode
|
||||
name: Activation - positive path (consent -> connect -> send -> reply)
|
||||
---
|
||||
# Positive activation flow: first launch -> telemetry consent -> quick connect
|
||||
# to the mock opencode server (tests/fixtures/mock-opencode-server.ts, run
|
||||
# normally on the port below) -> connected indicator -> new session -> send a
|
||||
# message -> assert the streamed canned reply renders.
|
||||
#
|
||||
# The CI job starts `node tests/fixtures/mock-opencode-server.ts --port 4096`
|
||||
# on the runner host BEFORE this flow runs. The Android emulator reaches the
|
||||
# runner host via the standard emulator alias 10.0.2.2.
|
||||
|
||||
- launchApp:
|
||||
clearState: true
|
||||
- takeScreenshot: positive-S1_launch_consent_modal
|
||||
- assertVisible:
|
||||
id: "telemetry-consent-card"
|
||||
- tapOn:
|
||||
id: "telemetry-decline-button"
|
||||
|
||||
- assertVisible:
|
||||
text: "No Connection"
|
||||
- takeScreenshot: positive-S2_empty_state
|
||||
- tapOn:
|
||||
id: "add-connection-button"
|
||||
|
||||
- assertVisible:
|
||||
id: "connect-ip-input"
|
||||
- takeScreenshot: positive-S3_add_connection_form
|
||||
- tapOn:
|
||||
id: "connect-ip-input"
|
||||
- inputText: "10.0.2.2"
|
||||
- tapOn:
|
||||
id: "connect-port-input"
|
||||
- eraseText
|
||||
- inputText: "4096"
|
||||
- hideKeyboard
|
||||
- takeScreenshot: positive-S4_server_url_entered
|
||||
- tapOn:
|
||||
id: "connect-submit-button"
|
||||
|
||||
- extendedWaitUntil:
|
||||
visible:
|
||||
id: "connection-status-dot"
|
||||
timeout: 20000
|
||||
- assertVisible:
|
||||
id: "connection-status-dot"
|
||||
- takeScreenshot: positive-S5_connected
|
||||
|
||||
- tapOn:
|
||||
id: "new-session-fab"
|
||||
- extendedWaitUntil:
|
||||
visible:
|
||||
id: "chat-message-input"
|
||||
timeout: 15000
|
||||
- takeScreenshot: positive-S6_session_opened
|
||||
|
||||
- tapOn:
|
||||
id: "chat-message-input"
|
||||
- inputText: "Hello from the activation e2e test"
|
||||
- hideKeyboard
|
||||
- takeScreenshot: positive-S7_message_typed
|
||||
- tapOn:
|
||||
id: "chat-send-button"
|
||||
|
||||
- extendedWaitUntil:
|
||||
visible:
|
||||
text: "Hello from the mock opencode server"
|
||||
timeout: 20000
|
||||
- assertVisible:
|
||||
id: "chat-bubble-assistant"
|
||||
- assertVisible:
|
||||
text: "Hello from the mock opencode server"
|
||||
- takeScreenshot: positive-S8_reply_received
|
||||
Reference in New Issue
Block a user