docs: add HANDOFF.md with project status and memory notes

This commit is contained in:
Ubuntu
2026-05-19 09:06:32 +00:00
parent 37bb8bfe72
commit 5071c445ac
5 changed files with 410 additions and 38 deletions

67
HANDOFF.md Normal file
View File

@@ -0,0 +1,67 @@
# Handoff: OpenCode Mobile App
## Status: CUA E2E Tests Working, v0.2.0 Release Triggered
## What Was Done
### 1. CUA (Computer-Use Agent) E2E Test Infrastructure
- Built `scripts/android-cua-smoke.py` — a vision-LLM-powered Android E2E test that drives the app via ADB screenshots + AI actions
- Two scenarios pass reliably: `send_message` (7 steps) and `multi_turn` (13 steps)
- Added custom `{"type": "send"}` action that auto-locates the send button via uiautomator XML (solves coordinate accuracy issues with vision models)
- Uses **Azure AI Services gpt-5.4** for vision inference
### 2. CI/CD
- `.github/workflows/cua-smoke.yml` — GitHub Actions workflow for emulator + CUA test
- `.github/workflows/build.yml` — Builds APK, creates GitHub Release on tag push
- `.github/workflows/publish-play-store.yml` — Fastlane Play Store publish
- GitHub secrets set: `AZURE_OPENAI_API_KEY`, `AZURE_OPENAI_ENDPOINT`
- Tagged `v0.2.0` — CI build + release workflow triggered
### 3. App Features (prior sessions)
- SSE reconnect with exponential backoff
- Notification support
- Message deduplication
- Biometric auth gate (disabled on emulator)
- Server connection management UI
## Key Technical Details
### Azure AI Services (LLM API)
- **Correct endpoint**: `https://info-mjnxtt51-eastus2.cognitiveservices.azure.com`
- **DO NOT USE**: `vibe-dev-ai.cognitiveservices.azure.com` (has no deployments, only model catalog)
- **Env file**: `~/.env.d/azure-openai.env`
- **Available models**: gpt-5.4, gpt-5.2, gpt-5.1, gpt-4.1, grok-4, deepseek-r1, kimi-k2.5
- **API quirk**: Use `max_completion_tokens` (NOT `max_tokens`) for gpt-5.x models
### Android Emulator
- SDK at `/tmp/android-sdk/` (may need reinstall if /tmp is cleared)
- AVD name: `test`, API 34 x86_64, KVM required
- Start: `emulator -avd test -no-window -no-audio -no-boot-anim -gpu swiftshader_indirect -no-snapshot`
- Package: `ai.opencode.mobile`
### CUA Script Learnings
- Vision models (gpt-5.4) consistently mis-estimate Y coordinates by ~80px for bottom-of-screen elements
- Solution: `{"type": "send"}` action uses uiautomator XML to find the rightmost clickable element in the bottom bar
- Enter key inserts newline in this app (doesn't send) — model must dismiss keyboard + tap send button
- Screen resolution (1080x2400) included in prompt context helps coordinate accuracy
- Screenshot retry (3 attempts, 30s timeout) needed for emulator under load
### Server Connection
- OpenCode server: `100.108.64.76:4096` (Tailscale, hostname `openclaw-dev-1`)
- Must dismiss keyboard before tapping Connect button (keyboard obscures it)
- Model: `gpt-5.3-chat-latest` (via GitHub Copilot provider)
## What's Next
1. **Verify v0.2.0 release** — check CI at https://github.com/dzianisv/opencode-mobile/actions
2. **Run CUA in CI** — the `cua-smoke.yml` workflow needs a running opencode server to connect to (currently targets Tailscale IP which won't be reachable from GitHub runners). Options:
- Mock server in CI
- Use a public opencode server endpoint
- Run as self-hosted runner on the dev VM
3. **Add accessibility labels** — add `content-desc="Send"` to the send button in the app for better CUA reliability
4. **More scenarios** — settings toggle, reconnect after server restart, slash commands
## Repo & Auth
- Repo: `dzianisv/opencode-mobile`
- GitHub auth: `source ~/.env.d/github-dzianisv.env`
- Upstream issue: `anomalyco/opencode#10288`