retro: document 2026-06-23 sessions false-fixes + CF detour lessons

Key entries:
- Vacuous CUA test goal ('empty is fine') enabled 3x false 'fixed' reports
- Cloudflare 2h detour: wrong zone ID + 20+ repeated DOM failures
- task_complete gate: real-channel verification required
- Overhead sequencing: reproduce before document
- Regression guards must run before model-dependent phases

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This commit is contained in:
Dennis V
2026-06-23 14:10:59 +00:00
parent 6198e4e165
commit 73656c783d

View File

@@ -18,6 +18,61 @@ Lessons from past tasks. Read before starting related work to avoid repeating mi
--- ---
## 2026-06-23: Sessions loading false-fixes + Cloudflare detour (#32, v0.4.6)
**Problem**: "Recent sessions not loading" bug (#32) was claimed fixed 3× by AI agents. Each time the CUA test "passed" because the test literally accepted an empty sessions list as success.
### Root causes
**Mistake 1 — Vacuous test goal (most critical)**: The `session_list` CUA phase goal said *"The session list may be empty — that is fine. Report done when you can see the session list screen (even if empty)."* An empty list IS the bug. Test and bug were definitionally equivalent — impossible to fail. Every subsequent agent inherited this test as "trusted infrastructure" without reading the pass condition.
**Correct pattern**: Before calling any bug "fixed," answer: *"If the bug were still present, would this test fail?"* If no → rewrite the test. Always pre-create server state via API, then assert that specific named data appears in the UI. Forbidden phrases in CUA goals: "empty is fine," "even if empty," "screen is visible."
---
**Mistake 2 — 2+ hours on Cloudflare (wrong zone ID, repeated DOM failures)**: Spent 2h+ trying wrong account/zone IDs. The correct `CLOUDFLARE_API_TOKEN` + zone ID were in `~/.config/codebox/env.sh` — never checked. Also: 20+ identical attempts to set a React controlled input via raw DOM (`nativeInputValueSetter`) despite the same uid-collision error each time. AGENTS.md "stop at 3×" was ignored.
**Correct credential search order** (stop at first hit):
1. `echo $VAR_NAME` (current env)
2. `~/.env.d/*.env`
3. **`~/.config/codebox/env.sh`** ← check this explicitly for CF/GCP/hosting
4. `find ~/workspace -name '.env' -maxdepth 4`
5. Bitwarden (`~/.bitwarden_credentials`)
6. `~/.config/*/`
7. Ask user
**Correct browser automation fallback**:
1. Uid from snapshot → 2. Coordinates → 3. Keyboard nav → 4. Opus subagent → BLOCKED
Never repeat the same uid target more than 3 times.
---
**Mistake 3 — `task_complete` as escape hatch**: Called `task_complete` twice with unverified work. User had to re-prompt both times. Used it to escape stalled loops rather than emit a clean BLOCKED.
**Correct gate for `task_complete`**:
- Real-channel test ran this turn or last (not just unit tests)
- "Test would fail on broken code because [reason]" — explicitly stated
- CI green on release tag
- No open "verify later" items
---
**Mistake 4 — Overhead before understanding**: Created GitHub issue and task tracking files before understanding the root cause. Issue filing is not progress.
**Correct pattern**: Reproduce first (5 min). Document after confirmed.
---
**Mistake 5 — sessions_reload placed after typescript phase**: The new regression guard was initially blocked behind the model-dependent `typescript` phase. When model was unavailable in CI, `sessions_reload` never ran.
**Correct pattern**: Regression guards must run before any phase that can fail due to external dependencies (model availability, network, etc.).
---
**Time cost**: ~6h total. ~2h from vacuous test discovery/fix, ~3h from Cloudflare detour, ~1h from task_complete false closures. `grep -r CLOUDFLARE ~/.config` (30s) at the start would have saved the 2h detour. Reading the CUA test goal (2 min) at the start would have saved the 2h sessions bug investigation loop.
---
## Template for future entries ## Template for future entries
``` ```