90 lines
4.0 KiB
Markdown
90 lines
4.0 KiB
Markdown
# ZeroMesh E2E Test Progress — 2026-06-17
|
||
|
||
## Setup
|
||
- Environment: Docker Compose on Linux amd64
|
||
- Go 1.26.4, SQLite via CGO (`golang:1.26-alpine` with gcc)
|
||
- Gitea: `http://gitea.xieyao.vip/xieyao/zeromesh`
|
||
- Volume mounts broken → Dockerfiles with `COPY` used instead
|
||
- Docker Compose YAML: all shell `$` must be `$$` (interpolation)
|
||
|
||
## Architecture
|
||
|
||
```
|
||
server (172.30.0.10:10001 HTTP + :19993 UDP/controller)
|
||
├─ init — creates admin + testnet (10.7.0.0/24)
|
||
├─ client1 — zmclient, UDP 7001, TAP ztmesh0
|
||
├─ client2 — zmclient, UDP 7002, TAP ztmesh1
|
||
├─ client3 — zmclient, UDP 7003, TAP ztmesh2
|
||
└─ tester — curl API queries + verification
|
||
```
|
||
|
||
## What Works
|
||
|
||
### Server
|
||
- HTTP server on `:10001` with Gin + GORM + SQLite auto-migrate
|
||
- JWT auth (login, register, init admin)
|
||
- Network CRUD with auto IP range generation
|
||
- Node registration + authorization
|
||
- Online node list
|
||
- VL1 controller UDP transport on `:19993`
|
||
- Health endpoint at `/api/health`
|
||
|
||
### Init Container
|
||
- Creates admin user (first run) or logs in (subsequent)
|
||
- Creates `testnet` network with `10.7.0.0/24`
|
||
|
||
### Test Clients (`cmd/testclient/main.go`)
|
||
- Admin login → init admin if not exists (retry up to 30×2s)
|
||
- Identity generation via ed25519 key pair → JSON key file
|
||
- Node registration on controller
|
||
- Network auto-join: finds existing testnet, creates one if none found
|
||
- Self-authorization on network → gets virtual IP
|
||
- Agent starts: UDP bind + TAP device open
|
||
- `ControllerHello()` — sends public key to controller UDP :19993
|
||
- `SyncPeers()` — fetches online nodes, establishes encrypted P2P handshake
|
||
- `SetTAPIP()` — configures TAP with MAC + IP
|
||
- TCP echo server on virtual IP `:7777`
|
||
- Client ready log: `name=` + `ip=` + `tap=` + `peers=N` (all 3 see 2 peers)
|
||
|
||
### Tester Container
|
||
- Login as admin, query `/api/v1/network/list` → sees testnet
|
||
- Query `/api/v1/node/online` → sees 3 clients with docker IPs and names
|
||
|
||
### SDK Components (`sdk/`)
|
||
| Component | File | Status |
|
||
|-----------|------|--------|
|
||
| REST client | `api.go` | Working (login, register, init, networks, nodes, members, authorize, dashboard) |
|
||
| Identity | `identity.go` | Working (load/generate/save JSON key file) |
|
||
| Agent | `agent.go` | Handshake OK; data plane WIP |
|
||
| TAP (Linux) | `tap_linux.go` | Working (open, set MAC, set IP, read/write) |
|
||
| TAP (stub) | `tap_stub.go` | Placeholder for non-Linux |
|
||
| VL1 protocol | `agent.go` | Header (ver/type/netID/len) + Handshake/Data/Keepalive packets |
|
||
| Encryption | `agent.go` | ChaCha20-Poly1305 with PSK + key derivation |
|
||
|
||
## What's Broken (Next Session)
|
||
|
||
### P2P Data Plane — Decrypt Failure
|
||
- **Symptom**: ping between virtual IPs → 100% packet loss; ARP `FAILED`
|
||
- **Root cause**: `deriveKeys` used `(localPub, remotePub)` in fixed order → each side computed different send/recv keys
|
||
- **Fix committed** in `a8b411c`: sort public keys with `bytes.Compare` before hashing → both sides get same key pair
|
||
- **Status**: fix committed but **untested** — rebuild images + restart compose needed
|
||
|
||
### Debug Logging Still Active
|
||
- `tap miss`, `send data`, `handle data`, `decrypt failed` — Info-level logs in `agent.go`
|
||
- Remove after data plane is verified
|
||
|
||
## Key Learnings
|
||
1. Docker volume mounts (`-v /host/path:/container/path`) don't work in this environment → use `COPY` in Dockerfiles
|
||
2. Docker Compose v2 interprets `$VAR` in YAML strings → must use `$$VAR` for shell variables
|
||
3. TAP MAC must match agent's software-generated MAC → call `SetMAC()` during `SetTAPIP()`
|
||
4. ChaCha20-Poly1305 key derivation must sort public keys so both sides agree on send/recv keys
|
||
|
||
## Next Tasks
|
||
1. Rebuild Docker images → `docker compose up -d` → test ping + TCP echo
|
||
2. Remove debug logging from `agent.go`
|
||
3. Write Go unit tests (VL1 packet, key derivation, MAC table, peer manager, API client)
|
||
4. Implement IP collision detection + retry in network service
|
||
5. Build standalone agent binary (`cmd/zeromesh-agent-sdk/`)
|
||
6. Start Flutter plugin (gomobile/FFI binding)
|
||
7. WebSocket real-time updates for Dashboard
|