Files
opencode-mobile/AGENTS.md
Ubuntu 37bb8bfe72 feat(cua): add 'send' action with UI auto-locate, fix coordinate accuracy, both scenarios pass
- Added 'send' action type that uses uiautomator XML to find the rightmost
  clickable element in the bottom input bar (the send button)
- Added screen resolution (1080x2400) to LLM context for better coordinate estimation
- Updated system prompt to instruct model to use 'send' action instead of manual tap
- Updated scenarios with clearer step-by-step instructions
- Both send_message and multi_turn scenarios pass reliably
2026-05-19 08:26:48 +00:00

4.2 KiB

Mobile App - AGENTS.md

Overview

React Native / Expo mobile client for opencode. Connects to an opencode server instance via HTTP + SSE for real-time updates.

Repo: dzianisv/opencode-mobile (standalone, not part of opencode monorepo) Package name: ai.opencode.mobile

Architecture

app/                    # Expo Router file-based routing
├── (tabs)/             # Tab navigation (sessions, connections, settings)
├── session/[id].tsx    # Chat screen
└── connection/         # Add/edit connection screens
src/
├── components/         # Reusable UI components
│   ├── markdown/       # Markdown renderer (wraps react-native-marked)
│   └── AuthGate.tsx    # Biometric auth gate
├── lib/
│   ├── sdk.ts          # HTTP + SSE client for opencode server API
│   └── types.ts        # Re-exported types
└── stores/             # Zustand state stores
    ├── sessions.ts     # Session list, messages, parts
    ├── connections.ts  # Server connections, client lifecycle
    ├── events.ts       # SSE event stream, status tracking, permissions, questions
    └── auth.ts         # Biometric auth
scripts/
└── android-cua-smoke.py  # LLM-powered CUA E2E test

Key Patterns

  • SSE for real-time: The events.ts store connects to /global/event and dispatches to other stores
  • Fire-and-forget sends: sendMessage posts to the API but doesn't await response; SSE events drive all UI updates
  • Session status: Derived from session.status events (idle/busy/retry) + last part type for status text
  • Markdown: react-native-marked wrapped in our own Markdown component with custom CodeBlock (copy button). Designed to be swappable/publishable later.

Style Guide

  • Prefer const over let
  • Avoid else statements, use early returns
  • Prefer single-word variable names
  • Avoid try/catch where possible
  • Avoid any type
  • Use Bun APIs where applicable (for scripts, not in RN runtime)

Running

npm install
npx expo start        # Expo dev server
npx expo run:android  # Android emulator

Connecting

Run opencode serve --hostname 0.0.0.0 --port 4096 on your machine, then add a connection in the app with your machine's local IP and port 4096.

Dev server: 100.108.64.76:4096 (Tailscale, hostname openclaw-dev-1)

Android Emulator (local dev)

export PATH="/tmp/android-sdk/platform-tools:/tmp/android-sdk/emulator:$PATH"
emulator -avd test -no-window -no-audio -no-boot-anim -gpu swiftshader_indirect -no-snapshot -port 5554
adb wait-for-device
# Wait for boot:
timeout 120 bash -c 'while [ "$(adb shell getprop sys.boot_completed 2>/dev/null)" != "1" ]; do sleep 2; done'

SDK location: /tmp/android-sdk/ (API 34, x86_64 system image).

CUA Smoke Test (E2E via Vision LLM)

The script scripts/android-cua-smoke.py drives the emulator via ADB using a vision model loop (screenshot → LLM → action → repeat).

Running locally

source ~/.env.d/azure-openai.env
python3 scripts/android-cua-smoke.py --model gpt-5.4 --include-xml

Azure OpenAI credentials

The correct Azure AI Services endpoint (with actual deployments) is:

  • Endpoint: https://info-mjnxtt51-eastus2.cognitiveservices.azure.com
  • Env file: ~/.env.d/azure-openai.env
  • Available vision models: gpt-5.4, gpt-5.2, gpt-5.1, gpt-4.1

Warning

: The vibe-dev-ai.cognitiveservices.azure.com resource has NO deployments (only a model catalog). Do NOT use it for inference. Always use info-mjnxtt51-eastus2.

API notes for gpt-5.x models

  • Use max_completion_tokens (NOT max_tokens) — the older param is rejected.
  • Use AzureOpenAI client from the openai Python SDK with api_version="2024-08-01-preview".
  • Vision works: pass image_url with data:image/png;base64,... in user message content array.

CI

GitHub Actions workflow: .github/workflows/cua-smoke.yml Secrets required: AZURE_OPENAI_API_KEY, AZURE_OPENAI_ENDPOINT (already set on dzianisv/opencode-mobile).

GitHub Auth

For pushes/gh CLI on this repo: source ~/.env.d/github-dzianisv.env

  • Upstream: anomalyco/opencode#10288
  • Branch on upstream fork: feat/android-backbone-10288 on dzianisv/opencode