kassam/fleet
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Update 2026-07-16 14:14:47

Browse Source
This commit is contained in:
kassam 2026-07-16 14:14:48 +04:00
parent 90a3f07a87
commit 6a1ab4bbde
11 changed files with 736 additions and 436 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

169
PIPELINE.md
View File

@ -43,7 +43,7 @@ repo currently implements the **bold** ones; the rest are documented for later.
| endpoint | method | agent | status | | endpoint | method | agent | status |
|---|---|---|---| |---|---|---|---|
| **`/api/v1/fleet/ingest/telemetry`** | POST | G1, R1, Go2 | ✅ ~2 s; 27 fields incl. software/firmware/control + all status mirrors | | **`/api/v1/fleet/ingest/telemetry`** | POST | G1, R1, Go2 | ✅ ~2 s; 27 fields incl. software/firmware/control + all status mirrors |
| **`/api/v1/fleet/ingest/{sn}/map`** | POST | G1, R1, Go2 | ✅ once per content; rtabmap `.db` (≤~7 MB) + slam_toolbox `pgm/yaml`→PNG | | **`/api/v1/fleet/ingest/{sn}/map`** | POST | G1, R1, Go2 | ✅ once per content; rtabmap `.db` (≤~7 MB) + slam_toolbox/Nav2/Pudu `pgm+yaml`→PNG |
| **`/api/v1/fleet/ingest/{sn}/alert`** | POST | G1, R1, Go2 | ✅ each NEW fault, rising edge, string body | | **`/api/v1/fleet/ingest/{sn}/alert`** | POST | G1, R1, Go2 | ✅ each NEW fault, rising edge, string body |
| **`/api/v1/fleet/ingest/{sn}/logs`** | POST | G1, R1, Go2 | ✅ agent + `[sanadr1-logs]` project lines every `LOGS_INTERVAL` (60 s) | | **`/api/v1/fleet/ingest/{sn}/logs`** | POST | G1, R1, Go2 | ✅ agent + `[sanadr1-logs]` project lines every `LOGS_INTERVAL` (60 s) |
| **`/api/v1/fleet/ingest/{sn}/remote`** | POST | G1, R1, Go2 | ✅ registers `web` (Sanad dashboard URL) + `ssh` (`ssh unitree@<ip>`) every 60 s | | **`/api/v1/fleet/ingest/{sn}/remote`** | POST | G1, R1, Go2 | ✅ registers `web` (Sanad dashboard URL) + `ssh` (`ssh unitree@<ip>`) every 60 s |
@ -54,7 +54,7 @@ Auth header (all): `Authorization: Bearer <device_token>`.
--- ---
## 3. Telemetry pipeline (R1, Go2) ## 3. Telemetry pipeline (all agents)
**Goal:** a fresh status record every ~2 s; never go dark. **Goal:** a fresh status record every ~2 s; never go dark.
@ -72,13 +72,14 @@ Auth header (all): `Authorization: Bearer <device_token>`.
### 3.1 Sources per robot ### 3.1 Sources per robot
| field | R1 (`unitree_hg`) | Go2 (`unitree_go`) | | field | G1 / R1 (`unitree_hg`) | Go2 (`unitree_go`) |
|---|---|---| |---|---|---|
| `battery` (0–100) | `rt/lf/bmsstate` → `BmsState_.soc` | `rt/lowstate` → `LowState_.bms_state.soc` | | `battery` (0–100) | `rt/lf/bmsstate` → `BmsState_.soc` | `rt/lowstate` → `LowState_.bms_state.soc` |
| `charging` | `BmsState_.current` > +0.05 A | `bms_state.current` > +0.05 A | | `charging` | `BmsState_.current` > +0.05 A | `bms_state.current` > +0.05 A |
| `faults[]` | `rt/lowstate` motor temps + staleness | same | | `faults[]` | `rt/lowstate` motor temps + staleness | same |
| `status` | derived (see 3.3); optional loco FSM `GET 7001` | derived | | `status` | derived (see 3.3); optional loco FSM `GET 7001` | derived |
| `position` | optional `rosbridge /odom` | optional `rosbridge /odom` or `rt/lf/sportmodestate` | | `position` | G1: `rt/lf/odommodestate` · R1: optional `rosbridge /odom` | optional `rosbridge /odom` or `rt/lf/sportmodestate` |
| `control` | Sanad `GET /api/controller/status` — **read-only** loco mode (G1 `200/4/2/702`, R1 `0/1/4/811`) | same path, Go2 labels |
| `mac` | NIC (`/sys/class/net/<iface>/address`) | NIC | | `mac` | NIC (`/sys/class/net/<iface>/address`) | NIC |
DDS is initialized once (`ChannelFactoryInitialize(domain, interface)`); each DDS is initialized once (`ChannelFactoryInitialize(domain, interface)`); each
@ -155,14 +156,15 @@ else → "idle"
### 3.4 Faults ### 3.4 Faults
Ordinary faults ride inside telemetry `faults[]` (the spec reserves the separate Faults ride inside telemetry `faults[]` as **strings** (the ingest 500s on fault
`/alert` endpoint for critical, immediate events like e-stop): objects) — and each **new** fault also fires the `/alert` endpoint (rising edge):
| code | trigger | severity | | code | trigger | severity |
|---|---|---| |---|---|---|
| `LOW_BATTERY` | `soc ≤ LOW_SOC` (default 15) | warning | | `LOW_BATTERY` | `soc ≤ LOW_SOC` (default **50**) | warning |
| `MOTOR_OVERTEMP` | any motor temp ≥ `MOTOR_TEMP_MAX` (default 85 °C) | warning | | `MOTOR_OVERTEMP` | any motor temp ≥ `MOTOR_TEMP_MAX` (default 85 °C) | warning |
| `COMMS_STALE` | no `rt/lowstate` for > 3 s | critical | | `COMMS_STALE` | no `rt/lowstate` for > 3 s | critical |
| `GEMINI_BILLING` / `ROBOT_ERROR` | project-log scan (see §5) | critical |
### 3.5 Heartbeat & failure handling ### 3.5 Heartbeat & failure handling
@ -176,42 +178,73 @@ Ordinary faults ride inside telemetry `faults[]` (the spec reserves the separate
--- ---
## 4. Map pipeline (G1) ## 4. Map pipeline (all agents)
**Goal:** keep the server's copy of the nav map current, sending only on change. **Goal:** keep the server's copy of every saved nav map current, sending each map
**once per content change**.
``` ```
files on disk agent (30 s scan) server files on disk agent (30 s scan) server
───────────── ───────────────── ────── ───────────── ───────────────── ──────
maps/<robot>/*.db ─┐ discover ┌ fingerprint (size+mtime) ─┐ changed? ┌ POST MAPS_DIR/<robot>/*.db ─┐ discover ┌ fingerprint fmt:size:mtime ┐ changed? ┌ POST
maps_meta.json ├────────▶ │ if changed: sha256 ├──yes──────▶│ /ingest/ MAPS_DIR/**.yaml + .pgm ├─────────▶ │ .db → sha256+multipart├──yes────▶│ /ingest/
web/data/<robot>/ │ │ load places → points[] │ │ {sn}/map EXTRA_MAP_DIRS (Nav2/Pudu) ─┘ │ pgm+yaml → PNG image JSON │ │ {sn}/map
places/<map>.json ┘ └ build meta + db bytes ─────┘ └ (multipart) └────────────────────────────┘ └
│ unchanged │ unchanged
└────────▶ skip (state/uploaded.json) └────────▶ skip (state/uploaded.json)
``` ```
### 4.1 Sources ### 4.1 Formats & discovery
- **Map file:** RTAB-Map SQLite `.db` — `maps/<robot>/<name>.db` (web_nav3). Two map formats are understood:
- **Metadata:** `maps/<robot>/maps_meta.json` → `{ "<db>": {description, created_at} }`.
- **Places → points:** `web/data/<robot>/places/<map>.json` - **RTAB-Map `.db`** (VSLAM) — sent as-is (multipart), with `maps_meta.json`
(`{ "<name>": {x, y, z, qx, qy, qz, qw} }`), converted to description and places → `points[]`. **Skipped when larger than
`{name, type:"waypoint", x, y, yaw}` (yaw computed from the quaternion). `MAP_MAX_UPLOAD_MB`** (7 — the server 413s at ~8 MB) with a `map.error` note.
A 120 s **stability guard** skips a `.db` still being written (active mapping →
`map.state: "pending"`).
- **`slam_toolbox` / Nav2 / Pudu sets** — `<stem>.yaml` + `<stem>.pgm`
(+ optional `.posegraph`/`.data`) → uploaded as the spec's **image JSON**
(pure-stdlib PGM→PNG + `resolution` + `origin` + size). Small; always uploads.
**Search roots** (each existing dir is scanned):
`MAPS_DIR`, `MAPS_DIR/<robot>`, `MAPS_DIR/maps_slam`, plus every dir in
`EXTRA_MAP_DIRS` (colon-separated — the installer detects the robot's
`*_nav2_docker/maps` dir and mounts it at `/data/nav2_maps`).
**Pudu twin dedup:** the Pudu converter emits a plain map *and* a keepout-**baked**
twin; when both `<stem>.yaml` and `<stem>_keepout_baked.yaml` exist, only the baked
one uploads — the server gets **one** canonical map.
**Places → points:** `web/data/<robot>/places/<map>.json`
(`{ "<name>": {x, y, z, qx, qy, qz, qw} }`) → `{name, type:"waypoint", x, y, yaw}`
(yaw from the quaternion).
### 4.2 Change detection ### 4.2 Change detection
1. Cheap pre-check: `size + mtime` vs `STATE_DIR/uploaded.json`. 1. Cheap pre-check: `fmt : size : mtime` vs `STATE_DIR/uploaded.json`.
2. If different, compute `sha256` of the `.db` (content-true). 2. If different, compute `sha256` (of the `.db`, or of the rendered image JSON).
3. Upload; on success, record the new fingerprint. A restart re-reads state → no 3. Upload; on success, record the new fingerprint. A restart re-reads state → no
redundant re-upload. redundant re-upload.
`MAP_SELECT` chooses scope: `all` (default), `active` (via web_nav3 `MAP_SELECT` chooses scope: `all` (default), `active` (via web_nav3
`GET /api/status`), or `newest`. `GET /api/status`), or `newest`.
### 4.3 Payload (two wire formats) ### 4.3 Wire formats
`multipart/form-data` (default) — file part `db` + form field `meta`: **Image JSON** (slam_toolbox / Nav2 / Pudu sets — the spec's map body):
```json
POST /api/v1/fleet/ingest/E39N4000Q6D7E70F/map
{ "sn": "E39N4000Q6D7E70F", "name": "map_keepout_baked", "format": "image",
"resolution": 0.05, "origin": [-5.85, -11.8, 0.0],
"width": 178, "height": 304,
"image_base64": "<PNG bytes, base64>",
"points": [ {"name":"dock","type":"waypoint","x":1.2,"y":3.4,"yaw":0.0} ] }
```
**Multipart** (RTAB-Map `.db`, default `MAP_UPLOAD_MODE=multipart`) — file part
`db` + form field `meta`:
``` ```
POST /api/v1/fleet/ingest/g1_7892/map POST /api/v1/fleet/ingest/g1_7892/map
@ -228,25 +261,72 @@ Content-Type: multipart/form-data
`base64json` (set `MAP_UPLOAD_MODE=base64json`) — same fields as JSON with the `base64json` (set `MAP_UPLOAD_MODE=base64json`) — same fields as JSON with the
`.db` as `db_base64`. `.db` as `db_base64`.
> **Server note:** this uploads the raw RTAB-Map `.db` (not a rendered PNG), so the > **Server note:** raster (`pgm+yaml`) maps use the spec's image body and display
> server must accept a `format:"rtabmap_db"` artifact. The spec's image-based map > on the dashboard. The raw `.db` path additionally requires the server to accept a
> body (`image_base64`/`resolution`/`origin`) would instead require rendering the > `format:"rtabmap_db"` artifact — and is size-capped (~8 MB), so a big VSLAM `.db`
> live `/map` OccupancyGrid over rosbridge — a different path, not used here. > is skipped; export a raster with `tests/map_export_once.py` instead.
### 4.4 One-shot upload (`--map-only`)
`--map-only` runs one discovery + upload pass with **no DDS and no telemetry** —
push a new map without disturbing a live feed:
```bash
docker run --rm --network host --env-file .env \
-e EXTRA_MAP_DIRS=/data/nav2_maps -v <host-maps>:/data/nav2_maps:ro \
sanad-api-<type>:latest --map-only --force
```
--- ---
## 5. Timing ## 5. Logs, alerts & remote registration
```
sanad app json-log ──tail──▶ ProjectLogTail ──┐
agent's own lines ──ring───────────────────── ├─▶ every 60 s ─▶ POST /{sn}/logs
┘ (failed ships requeue, last ~400 lines)
project log ──scan 10 s──▶ LogAlertScanner ──new signature──▶ POST /{sn}/alert
faults[] ──rising edge──────────────────────────────────▶ POST /{sn}/alert
localhost ports 8001,8014,8011-8013,8000,8080 ──probe──▶ dashboard URL
└─▶ every 60 s ─▶ POST /{sn}/remote (kind "web" + kind "ssh")
```
- **Logs** — the agent tails the robot's Sanad container json-log via the read-only
`/host` mount (no docker socket): auto-discovers a running `sanad*` container,
unwraps the json-log, labels lines `[sanadr1-logs]`, filters uvicorn access-log
noise, backfills the last `PROJECT_LOG_BACKFILL` (100) relevant lines on start.
- **Alerts** — `ALERT_LOG_PATTERNS` (`CODE=regex` split by `;;`, **case-sensitive**;
`(?i)` inline where needed) scan the project log every 10 s; each **new**
signature (code + digit-stripped line) alerts once per `ALERT_LOG_COOLDOWN`
(300 s). On startup the last `ALERT_BACKFILL_BYTES` (8 MiB) are scanned so an
already-active error still alerts. Defaults catch **Gemini billing** and any
`ERROR`/`CRITICAL`/`Traceback`.
- **Remote** — probes `REMOTE_PORTS` on localhost for the Sanad dashboard page and
registers `{kind:"web", url:"http://<lan-ip>:<port>"}` plus
`{kind:"ssh", command:"ssh unitree@<ip>"}` — no change to the Sanad app.
⚠ LAN-only URL unless `REMOTE_URL` pins a public tunnel.
---
## 6. Timing
| stream | cadence | trigger | | stream | cadence | trigger |
|---|---|---| |---|---|---|
| telemetry (R1/Go2) | every `POLL_INTERVAL` (2 s) | timer | | telemetry | every `POLL_INTERVAL` (2 s) | timer |
| map (G1) | scan every `POLL_INTERVAL` (30 s) | uploads only on content change | | map | scan every `MAP_POLL_INTERVAL` (30 s) | uploads only on content change |
| alert scan | every `ALERT_SCAN_INTERVAL` (10 s) | new log signature / rising-edge fault |
| log ship | every `LOGS_INTERVAL` (60 s) | ring buffer + project tail |
| remote register | every `REMOTE_INTERVAL` (60 s) | re-asserts the dashboard/ssh entry |
| DDS reads | continuous (subscriber callbacks) | firmware publish rate | | DDS reads | continuous (subscriber callbacks) | firmware publish rate |
| heartbeat | same as telemetry cadence | when state is unreadable | | heartbeat | same as telemetry cadence | when state is unreadable |
--- ---
## 6. Where it runs ## 7. Where it runs
![Deploy — workstation to robot over SSH](deploy.svg)
``` ```
robot host robot host
@ -264,7 +344,7 @@ robot host
--- ---
## 7. End-to-end sequences ## 8. End-to-end sequences
### Telemetry tick ### Telemetry tick
@ -293,16 +373,25 @@ scan(30s) disk agent fleet server
--- ---
## 8. Verified behavior (real hardware) ## 9. Verified behavior (real hardware)
Confirmed against the live fleet during bring-up: Confirmed against the live fleet during bring-up:
- **R1** streamed real telemetry every 2 s; battery read live and drained - **R1 is LIVE on production** (`https://eco.yslootahrobotics.com`, sn
`97 → 80%` across the session; `mac`/`sn` correct; heartbeat kicked in when the `E39N4000Q6D7E70F`): telemetry every 2 s (HTTP 200), real battery draining live,
server was down. `control.mode` tracks the actual loco FSM (`lock`/`zero_torque` observed).
- **G1** uploaded a map (`multipart`, `format:rtabmap_db`, points with correct - **Maps:** R1's VSLAM raster export uploaded and displayed **on the dashboard**;
yaw) to the server. the converted **Pudu office map** (`map_keepout_baked`, 178×304 @ 0.05 m)
- The fleet server received a **G1 map and R1 telemetry in the same window** — the uploaded via `--map-only` → HTTP 200. Oversized `.db` correctly skipped (413 cap).
multi-robot pipeline works concurrently. - **Alerts verified live:** `LOW_BATTERY` fired at 50/49/48 %…, `GEMINI_BILLING`
and `ROBOT_ERROR` fired from the real sanadr1 log — no false positives after the
case-sensitivity fix.
- **Remote:** R1 registered `http://10.255.254.82:8001` (web) + `ssh unitree@…`
→ HTTP 200.
- **Logs:** shipped labeled `[sanadr1-logs]` lines; requeue held them through the
server-side `/logs` 500 until the handler was fixed (then `stored:105`).
- **G1** uploaded a map (`multipart`, `format:rtabmap_db`, points with correct yaw);
G1 map + R1 telemetry landed in the same window — multi-robot works concurrently.
⚠ G1 currently offline (battery) + deploy hold.
- **Go2**: pipeline code + image verified (builds, `unitree_go` imports, simulate - **Go2**: pipeline code + image verified (builds, `unitree_go` imports, simulate
payloads correct); **not yet run on a real Go2**. payloads correct); **not yet run on a real Go2**.

561
README.md
View File

@ -1,190 +1,246 @@
# Sanad Fleet Agents # Sanad Fleet Agents
On-robot agents that report each robot's state to the **YS Lootah fleet server**. On-robot agents that report each Unitree robot's state to the **YS Lootah fleet
Every robot type ships as its own self-contained **Docker** image with a server**. One agent per robot type (**G1 · R1 · Go2**), each a self-contained
user-level **systemd auto-start service**, deployed, managed, and removed over **Docker** image with a user-level **systemd auto-start service**, deployed and
SSH by a single script. No `docker-compose` on the robot, **no `sudo`** required. managed over SSH by a single script — **no `docker-compose`, no `sudo`** on the robot.
![Sanad Fleet — architecture and data pipeline](diagram.svg) ![Sanad Fleet — one agent, five endpoints](diagram.svg)
> For the end-to-end data flow (topics → agent → HTTP → server), see > Full topic-by-topic data flow and payload schema: **[PIPELINE.md](PIPELINE.md)**.
> [PIPELINE.md](PIPELINE.md).
---
## At a glance
- **One agent = one service = everything.** Each robot runs a single container that
streams telemetry **and** uploads maps, ships logs, raises alerts, and registers
its dashboard — across **five endpoints**.
- **Read-only toward the robot.** It reads DDS + files; it **never commands motion**.
- **Outbound HTTPS only.** No inbound port is opened on the robot; every request
carries `Authorization: Bearer <token>` and is keyed by the robot's serial (`sn`).
- **One source of truth.** `agents/g1/` is canonical; `r1` and `go2` are **generated**
from it (`tools/gen_agents.py`). You never hand-edit r1/go2, and never edit on the robot.
| endpoint | when | what |
|---|---|---|
| `POST …/ingest/telemetry` | every ~2 s | 27 fields — identity, software/firmware, battery(+detail), motor temps, storage, status, position, **control mode**, faults, + status of map/logs/alerts/remote |
| `POST …/ingest/{sn}/map` | on change | saved nav map → PNG + resolution + origin (also RTAB-Map `.db`) |
| `POST …/ingest/{sn}/alert` | on new fault | battery ≤ 50 %, Gemini-billing, any error (as strings) |
| `POST …/ingest/{sn}/logs` | every 60 s | the agent's own lines **+ the robot's Sanad app logs** |
| `POST …/ingest/{sn}/remote` | every 60 s | dashboard URL (`web`) + `ssh unitree@<ip>` (`ssh`) |
*(A `GET …/ingest/{sn}/commands` channel also exists; remote mode-**switching** is
deliberately **not** built — status only. See [§2](#2-the-agent).)*
--- ---
## Table of contents ## Table of contents
1. [What this is](#1-what-this-is) 1. [Directory layout](#1-directory-layout)
2. [Directory layout](#2-directory-layout) 2. [The agent](#2-the-agent)
3. [The three agents](#3-the-three-agents) 3. [What it reports (the 27 fields)](#3-what-it-reports-the-27-fields)
4. [Prerequisites](#4-prerequisites) 4. [Maps: discovery & upload](#4-maps-discovery--upload)
5. [Quick start](#5-quick-start) 5. [Prerequisites](#5-prerequisites)
6. [Installer reference](#6-installer-reference) 6. [Quick start](#6-quick-start)
7. [The auto-start service (systemd)](#7-the-auto-start-service-systemd) 7. [How a deploy works](#7-how-a-deploy-works)
8. [Configuration reference](#8-configuration-reference) 8. [Installer reference](#8-installer-reference)
9. [Per-robot specifics](#9-per-robot-specifics) 9. [The auto-start service (systemd)](#9-the-auto-start-service-systemd)
10. [The fleet test server](#10-the-fleet-test-server) 10. [Configuration reference](#10-configuration-reference)
11. [How a deploy works internally](#11-how-a-deploy-works-internally) 11. [Per-robot specifics](#11-per-robot-specifics)
12. [Fleet inventory](#12-fleet-inventory) 12. [The fleet test server](#12-the-fleet-test-server)
13. [Troubleshooting](#13-troubleshooting) 13. [Fleet inventory](#13-fleet-inventory)
14. [Security notes](#14-security-notes) 14. [Troubleshooting](#14-troubleshooting)
15. [Security notes](#15-security-notes)
--- ---
## 1. What this is ## 1. Directory layout
The fleet server (built by YS Lootah — "their side") needs each robot to push its
status outbound over HTTPS. This repo is **the robot side**: a small agent per
robot type that reads local state and POSTs it to documented endpoints, plus the
tooling to install/manage it across the fleet.
- **G1** → uploads the **navigation map**.
- **R1** / **Go2** → stream **telemetry** (battery, charging, status, position, faults).
Each is a drop-in Docker image; install it on any new robot of that type and it
just works.
---
## 2. Directory layout
``` ```
Project/fleet/ Project/fleet/
├── README.md ← this file ├── README.md ← this file
├── PIPELINE.md ← end-to-end data-flow reference ├── PIPELINE.md ← end-to-end data-flow + payload schema
├── diagram.svg deploy.svg ← the two diagrams in this doc
├── fleet_install.sh ← deploy / manage / remove over SSH (interactive + scriptable) ├── fleet_install.sh ← deploy / manage / remove over SSH (interactive + scriptable)
├── fleet_test_server.py ← workstation stand-in for the fleet server (for tests) ├── fleet_test_server.py ← workstation stand-in for the fleet server (for tests)
├── tools/
│ └── gen_agents.py ← regenerates r1 + go2 from the canonical g1
├── tests/ ← mock capture + one-shot map export helpers
└── agents/ └── agents/
├── g1/ ← MAP uploader ├── g1/ ← CANONICAL source (edit here)
│ ├── sanad_api_g1.py │ ├── sanad_api_g1.py
│ ├── Dockerfile (lean: python + requests) │ ├── Dockerfile (lean: python + requests)
│ ├── requirements.txt │ ├── requirements.txt · docker-compose.yml (local use only) · .env.example
│ ├── docker-compose.yml (manual local use only) ├── r1/ ← generated · unitree_hg · R1 FSM ids · eth10
│ └── .env.example │ ├── sanad_api_r1.py · Dockerfile (DDS: CycloneDDS + unitree_sdk2py) · vendor/ · .env.example
├── r1/ ← TELEMETRY (unitree_hg) └── go2/ ← generated · unitree_go · ⚠ unverified on hardware
│ ├── sanad_api_r1.py └── sanad_api_go2.py · Dockerfile · vendor/ · .env.example
│ ├── Dockerfile (DDS: CycloneDDS + unitree_sdk2py)
│ ├── vendor/ (unitree_sdk2py wheel + crc libs)
│ └── .env.example
└── go2/ ← TELEMETRY (unitree_go) [unverified on hardware]
├── sanad_api_go2.py
├── Dockerfile
├── vendor/
└── .env.example
``` ```
`agents/` is the **single source of truth**. The installer rsyncs the selected > **Golden rule:** edit `agents/g1/`, then run `python3 tools/gen_agents.py` to
agent to the robot; you never edit files on the robot. > regenerate `r1` and `go2`. The installer rsyncs the selected agent to the robot;
> files on the robot are never edited by hand.
--- ---
## 3. The three agents ## 2. The agent
| agent (type) | robot | DDS family | notes | | agent (type) | robot | DDS family | notes |
|---|---|---|---| |---|---|---|---|
| `sanad_api_g1` (`g1`) | Unitree G1 | `unitree_hg` | canonical source (see below) | | `sanad_api_g1` (`g1`) | Unitree G1 | `unitree_hg` | **canonical source** |
| `sanad_api_r1` (`r1`) | Unitree R1 EDU | `unitree_hg` | generated from g1; R1 FSM ids, `eth10` | | `sanad_api_r1` (`r1`) | Unitree R1 EDU | `unitree_hg` | generated from g1 · R1 FSM ids `0/1/4/811` · `eth10` |
| `sanad_api_go2` (`go2`) | Unitree Go2 | `unitree_go` | generated from g1; battery nested in `LowState.bms_state` — ⚠ unverified on hardware | | `sanad_api_go2` (`go2`) | Unitree Go2 | `unitree_go` | generated from g1 · battery nested in `LowState.bms_state` · ⚠ unverified on hardware |
**One agent = one full-feature service.** Each robot runs a single container that All three run the **same five loops** (telemetry 2 s · map 30 s · alert scan 10 s ·
does everything, across five endpoints: log ship 60 s · remote register 60 s). They differ only in the DDS layer and a few
robot-specific ids — which is exactly what the generator handles.
- **telemetry** (`…/telemetry`, ~2 s) — 27 fields: identity, `software`, `firmware`,
battery(+detail), motor temps, storage, status, position, `control` (loco mode),
faults, and status mirrors for map/logs/alerts/remote + full timing.
- **map** (`…/{sn}/map`, on change) — the Sanad/SLAM saved map, uploaded ONCE per
content. Formats: **RTAB-Map `.db`** (skipped over the ~8 MB server cap) and
**slam_toolbox** `pgm/yaml` → image JSON (PNG + resolution + origin). Status shown
in telemetry `map`.
- **alerts** (`…/{sn}/alert`) — each NEW fault, immediately (strings — the ingest
500s on fault objects).
- **logs** (`…/{sn}/logs`, 60 s) — the agent's own lines **+ the robot's Sanad app
logs** (`[sanadr1-logs]`, auto-discovered, backfilled, noise-filtered).
- **remote** (`…/{sn}/remote`, 60 s) — registers the Sanad **dashboard URL** (`web`)
and **`ssh unitree@<ip>`** (`ssh`) — no changes to the Sanad app.
**All three agents are generated from `g1`** by `tools/gen_agents.py` — run it after
any change to `agents/g1/` to keep r1/go2 in lockstep (no manual editing of r1/go2).
All requests carry `Authorization: Bearer <device_token>`. Full payload schema and
pipeline in [PIPELINE.md](PIPELINE.md).
**Design principles** **Design principles**
- **No ROS.** Maps read from files; state read from DDS via `unitree_sdk2py`. - **No ROS.** Maps are read from files; robot state from DDS via `unitree_sdk2py`.
- **Read-only toward the robot.** Never commands motion — the control panel is - **Read-only.** The control panel reports the loco mode but **cannot switch it** —
status-only; mode switching is deliberately not built (it can drop the robot). mode switching can drop the robot, so it is intentionally not built (`CONTROL_ENABLE=0`).
- **Never crash the loop.** Every tick is wrapped; a heartbeat keeps the robot - **Never crash the loop.** Every tick is wrapped; a heartbeat keeps the robot
"online" when state is unreadable. "online" when state is momentarily unreadable.
- **Change-detected + one-time uploads.** Maps re-send only on content change. - **Idempotent uploads.** Maps re-send only when their content changes.
- **Resilient shipping.** Failed log ships are requeued (last ~400 lines) until the - **Resilient shipping.** Failed log ships are requeued (last ~400 lines) until the
server accepts them. server accepts them.
--- ---
## 4. Prerequisites ## 3. What it reports (the 27 fields)
**Workstation (the machine you deploy from):** Every telemetry POST is one JSON object. Grouped for readability:
| group | fields |
|---|---|
| **identity** | `sn` · `name` · `mac` · `brand` · `type` · `model` |
| **system** | `software{ros,os,os_version,kernel,arch,python,agent}` · `firmware{board,l4t,kernel,robot,bms}` |
| **power** | `battery` · `charging` · `battery_detail{voltage_v,current_a,temp_c,soh,cycles}` |
| **health** | `motor_temp{max,avg,min}` *(null = not receiving)* · `storage{total_gb,free_gb,used_percent}` · `faults[]` *(strings)* |
| **state** | `status` *(idle/moving/charging/offline)* · `position{x,y}` · `control{fsm_id,mode,armed,walk_ready,teleop_active,…}` |
| **sub-status** | `map{…}` · `logs{…}` · `project_logs` · `remote{…}` · `alerts{…}` |
| **timing** | `time` · `started_at` · `last_start` · `uptime_s` · `ts` |
`control.mode` is a friendly label of the loco FSM (R1: `zero_torque`/`damp`/`lock`/
`running`; G1: `running`/`lock`/`squat`/…), read from the Sanad dashboard status API
— no DDS command, no motion. Full schema in **[PIPELINE.md](PIPELINE.md)**.
---
## 4. Maps: discovery & upload
The agent finds saved maps on the robot and uploads each **once per content change**.
It understands two formats:
- **`slam_toolbox` / Nav2 / Pudu sets** — `map.pgm` + `map.yaml` → rendered to an
image JSON (pure-stdlib PGM→PNG + resolution + origin). Small; always uploads.
- **RTAB-Map `.db`** — sent as-is, but **skipped if larger than the server's ~8 MB
cap** (`MAP_MAX_UPLOAD_MB`, default 7) with a note in `map.error`.
**Where it looks** (each existing dir is scanned for `*.pgm` + `*.yaml` and `*.db`):
- `MAPS_DIR` (`/data/maps`) — the robot's primary SLAM/nav maps dir (installer probes it).
- `EXTRA_MAP_DIRS` (`/data/nav2_maps`) — a **separate Nav2/Pudu deploy-maps dir**
(e.g. `~/r1_nav2_docker/maps`) mounted alongside. The installer auto-detects and
mounts it, so converted **Pudu office maps** are discovered and uploaded too.
Two conveniences:
- The Pudu converter emits a plain map **and** a keepout-**baked** twin; when both
exist the agent keeps only the baked one, so the server gets **one** canonical map.
- `--map-only` uploads discovered maps once with **no DDS and no telemetry** — handy
to push a new map without disturbing a live feed:
```bash
docker run --rm --network host --env-file .env \
-e EXTRA_MAP_DIRS=/data/nav2_maps -v <host-maps>:/data/nav2_maps:ro \
sanad-api-<type>:latest --map-only --force
```
Map status is mirrored into every telemetry post as
`map:{uploaded,state,maps_found,last_map,error,checked_ts}`.
---
## 5. Prerequisites
**Workstation (deploy host)**
- `bash`, `ssh`, `rsync`, `python3`. - `bash`, `ssh`, `rsync`, `python3`.
- SSH **key** access to each robot (`ssh unitree@<ip>` must work without a - SSH **key** access to each robot — `ssh unitree@<ip>` must work without a password
password — the installer uses `BatchMode=yes`). (the installer uses `BatchMode=yes`).
- On the same network as the robots (they POST back to the workstation during - On the same network as the robots (they POST back to the workstation during `test`).
`test`).
**Robot:** **Robot**
- Docker (Engine ≥ 20). The `unitree` user must be in the `docker` group. - Docker (Engine ≥ 20); the `unitree` user in the `docker` group.
- Internet at build time (pulls the base image; R1/Go2 also `apt`/`pip` the DDS - Internet at build time (base image; r1/go2 also build the DDS stack).
stack). - `systemd` **user** bus (standard on Ubuntu 20.04+). **No sudo needed.**
- `systemd` with a user bus (standard on Ubuntu 20.04+). **No sudo needed.**
- Architecture: arm64 (Jetson / backpack). Images build natively on the robot. - Architecture: arm64 (Jetson / backpack). Images build natively on the robot.
--- ---
## 5. Quick start ## 6. Quick start
```bash ```bash
cd Project/fleet cd Project/fleet
# Interactive — asks robot type, IP, and (if new) robot name + server: # Interactive — asks robot type, IP, and (if new) name + token + server:
./fleet_install.sh ./fleet_install.sh
# …or scripted: # …or scripted against the real fleet server:
./fleet_install.sh install r1 10.255.254.82 --sn E39N4000Q6D7E70F --name r1_82 \ ./fleet_install.sh install r1 10.255.254.82 \
--server-ip 10.255.254.83 --port 8799 --token <device-token> --sn E39N4000Q6D7E70F --name r1_82 \
--token <device-token> --server-url https://eco.yslootahrobotics.com
# See what it's sending, tail logs, check the service: # Inspect / manage:
./fleet_install.sh data r1 10.255.254.82 ./fleet_install.sh data r1 10.255.254.82 # what it's currently sending
./fleet_install.sh logs r1 10.255.254.82 ./fleet_install.sh logs r1 10.255.254.82 # live docker logs
./fleet_install.sh status r1 10.255.254.82 ./fleet_install.sh status r1 10.255.254.82 # systemd + container state
./fleet_install.sh test r1 10.255.254.82 # e2e against the workstation server
# End-to-end test against your workstation acting as the server: ./fleet_install.sh uninstall r1 10.255.254.82 # remove everything
./fleet_install.sh test r1 10.255.254.82
# Remove everything:
./fleet_install.sh uninstall r1 10.255.254.82
``` ```
--- ---
## 6. Installer reference ## 7. How a deploy works
### Interactive mode (no arguments) ![Deploy — workstation to robot over SSH](deploy.svg)
```
rsync agents/<type>/ → unitree@<ip>:~/sanad_api_<type>/ (--delete; keeps .env + state)
write ~/sanad_api_<type>/.env (SERVER_URL, TOKEN, SN, iface, …)
ssh: docker build -t sanad-api-<type>:latest . (native arm64)
ssh: docker create --name sanad-api-<type> <run-args> (no docker restart policy)
write ~/.config/systemd/user/sanad-api-<type>.service
ssh: loginctl enable-linger ; systemctl --user enable --now sanad-api-<type>
```
**run-args** (all agents): `--network host --env-file …/.env` plus read-only mounts:
`-v /:/host:ro` (real disk/firmware + Sanad json-log), `-v <maps>:/data/maps:ro`,
`-v <web_data>:/data/web_data:ro`, `-v <nav2 maps>:/data/nav2_maps:ro` *(when present)*,
and `-v …/state:/data/state` (rw, upload fingerprints).
`--network host` is required so the robot's DDS multicast is visible (it doesn't cross
a NAT bridge) and so the agent can reach the Sanad dashboard on `127.0.0.1`.
---
## 8. Installer reference
### Interactive (no arguments)
``` ```
./fleet_install.sh ./fleet_install.sh
``` ```
Flow:
1. **Which robot?** `1) g1 2) r1 3) go2` 1. **Which robot?** `1) g1 2) r1 3) go2`
2. **Robot IP** and **SSH user** (default `unitree`; it verifies SSH works). 2. **Robot IP** + **SSH user** (default `unitree`; verifies SSH works).
3. It **detects whether the agent is already installed** (systemd unit file *or* 3. **Detects whether it's already installed** (unit file *or* container) and branches:
container present) and branches: - **Installed** → menu: show data · status · logs · reinstall · **uninstall** · quit.
- **Installed** → menu: `1) show data 2) status 3) logs 4) reinstall - **Not installed** → prompts **SN (required)**, **server** (URL or auto-detected IP),
5) UNINSTALL 6) quit`. **token**, then installs.
- **Not installed** → prompts **Robot name (SN)**, **Fleet server IP**
(auto-detected default), **port**, **device token**, then installs.
### Scriptable commands ### Scriptable
``` ```
./fleet_install.sh <command> <g1|r1|go2> <ip> [options] ./fleet_install.sh <command> <g1|r1|go2> <ip> [options]
@ -192,49 +248,42 @@ Flow:
| command | action | | command | action |
|---|---| |---|---|
| `install` | rsync agent → robot, build image, create container, install + enable systemd service | | `install` | rsync → build → create → install + enable the systemd service |
| `uninstall` | disable/remove service, remove container, image, and `~/sanad_api_<type>` | | `uninstall` | disable/remove service, container, image, and `~/sanad_api_<type>` |
| `status` | systemd service state + container state | | `status` | systemd service state + container state |
| `data` | recent telemetry/map log lines (what it's currently sending) | | `data` | recent telemetry/map log lines (what it's sending) |
| `logs` | `docker logs -f` (live tail) | | `logs` | `docker logs -f` (live tail) |
| `test` | start the workstation server, push a real post from the robot, verify receipt (PASS/FAIL) | | `test` | start the workstation server, push a real post from the robot, verify (PASS/FAIL) |
### Options `install` requires the essentials to be explicit:
**`install` requires the essentials to be entered explicitly** — the robot's real
serial, the token, and the server:
```bash ```bash
./fleet_install.sh install <type> <ip> --sn <robot-serial> --token <device-token> \ ./fleet_install.sh install <type> <ip> --sn <serial> --token <token> \
--server-url https://eco.yslootahrobotics.com [--post <ingest-path>] [--name <display>] --server-url https://eco.yslootahrobotics.com [--post <path>] [--name <display>]
``` ```
| option | default | meaning | | option | default | meaning |
|---|---|---| |---|---|---|
| `--sn SERIAL` | **required for install** | robot's REAL serial (keys the robot on the server), e.g. `E39N4000Q6D7E70F` | | `--sn SERIAL` | **required** | robot's REAL serial — keys it on the server (e.g. `E39N4000Q6D7E70F`) |
| `--name NAME` | `<model>_<last-octet>` | friendly display name (e.g. `r1_82`, `g1_58`) | | `--name NAME` | `<model>_<last-octet>` | friendly display name (`r1_82`, `g1_58`) |
| `--token TOK` | `test-token` | device bearer token | | `--token TOK` | `test-token` | device bearer token |
| `--server-url URL` | — | full fleet server URL (`https://…`) → `VERIFY_TLS=1` | | `--server-url URL` | — | full HTTPS fleet server → `VERIFY_TLS=1` |
| `--post PATH` | agent default | ingest POST path (telemetry endpoint; map endpoint for `g1`) | | `--post PATH` | agent default | override the telemetry ingest path |
| `--server-ip IP` | auto (route toward robot) | alternative: local test server by IP (`VERIFY_TLS=0`) | | `--server-ip IP` | auto (route to robot) | use the local test server instead → `VERIFY_TLS=0` |
| `--port N` | `8799` | test-server port (with `--server-ip`) | | `--port N` | `8799` | test-server port (with `--server-ip`) |
| `--user USER` | `unitree` | SSH user on the robot | | `--user USER` | `unitree` | SSH user on the robot |
| `--keep-server` | off | (test) leave the workstation test server running | | `--keep-server` | off | (test) leave the workstation test server running |
The interactive flow prompts for the same set: robot type → IP → **SN (required)**
→ display name → SERVER_URL-or-IP → token → POST endpoint.
--- ---
## 7. The auto-start service (systemd) ## 9. The auto-start service (systemd)
Because the robots have **no passwordless sudo**, the agent runs as a **user-level Because the robots have **no passwordless sudo**, the agent runs as a **user-level**
systemd service** (no root needed): systemd service (no root):
- Unit file: `~/.config/systemd/user/sanad-api-<type>.service` - Unit: `~/.config/systemd/user/sanad-api-<type>.service`
- Boot auto-start: enabled via `loginctl enable-linger <user>` (allowed without - Boot auto-start via `loginctl enable-linger <user>` (allowed without sudo).
sudo) so the user manager starts at boot before login. - systemd owns the container lifecycle:
- The service owns the container lifecycle:
```ini ```ini
[Service] [Service]
Type=simple Type=simple
@ -243,171 +292,161 @@ systemd service** (no root needed):
ExecStart=/usr/bin/docker start -a sanad-api-<type> ExecStart=/usr/bin/docker start -a sanad-api-<type>
ExecStop=/usr/bin/docker stop -t 10 sanad-api-<type> ExecStop=/usr/bin/docker stop -t 10 sanad-api-<type>
``` ```
- The container is created with `docker create` (no docker restart policy) so - Created with `docker create` (no docker restart policy) so **systemd is the single
**systemd is the single owner** — `Restart=always` also covers the boot race owner**; `Restart=always` also covers the boot race where the user manager starts
where the user manager starts before `dockerd` is ready (it retries every 5 s). before `dockerd` is ready.
**Managing it on the robot:**
**On the robot:**
```bash ```bash
systemctl --user status sanad-api-r1 systemctl --user status sanad-api-r1
systemctl --user restart sanad-api-r1 systemctl --user restart sanad-api-r1
systemctl --user stop sanad-api-r1
journalctl --user -u sanad-api-r1 -f # or: docker logs -f sanad-api-r1 journalctl --user -u sanad-api-r1 -f # or: docker logs -f sanad-api-r1
``` ```
--- ---
## 8. Configuration reference ## 10. Configuration reference
Config is env-only. The installer writes `~/sanad_api_<type>/.env` on the robot; Config is env-only; the installer writes `~/sanad_api_<type>/.env`. Every key is
`.env.example` in each agent dir documents every key. documented in each agent's `.env.example`.
### Common (all agents) ### Core (all agents)
| var | required | default | meaning |
|---|---|---|---|
| `SERVER_URL` | ✅ | — | fleet server base URL |
| `DEVICE_TOKEN` | ✅ | — | bearer token (per robot) |
| `SN` | — | `<type>_0000` | robot fleet id |
| `VERIFY_TLS` | — | `1` | verify server TLS cert (`0` for self-signed dev) |
| `HTTP_TIMEOUT` | — | 10–30 | per-request timeout (s) |
| `POLL_INTERVAL` | — | 2 (tel) / 30 (map) | loop cadence (s) |
### G1 map uploader
| var | default | meaning | | var | default | meaning |
|---|---|---| |---|---|---|
| `ROBOT` | `sanad` | web_nav3 robot name → maps subdir + `X-Robot-Name` | | `SERVER_URL` ✅ | — | fleet server base URL |
| `MAPS_DIR` | `/data/maps` | mounted web_nav3 `maps/` (`<robot>/*.db`) | | `DEVICE_TOKEN` ✅ | — | bearer token (per robot) |
| `DATA_DIR` | — | mounted web_nav3 `web/data/` (per-map places) | | `SN` | `<type>_0000` | robot's real serial (server key) |
| `LEGACY_PLACES` | — | optional legacy `places.json` | | `ROBOT_NAME` | `<model>_<octet>` | friendly display name |
| `WEB_NAV3_URL` | — | optional `http://127.0.0.1:8765` (to learn the active map) | | `ROBOT_BRAND` / `ROBOT_TYPE` / `ROBOT_MODEL` | `unitree` / `humanoid`\|`dog` / `g1`\|`r1`\|`go2` | identity |
| `MAP_SELECT` | `all` | `all` · `active` · `newest` | | `POLL_INTERVAL` | `2` | telemetry cadence (s) |
| `MAP_UPLOAD_MODE` | `multipart` | `multipart` · `base64json` | | `VERIFY_TLS` | `1` | verify server TLS (`0` for the local test server) |
| `MAP_ENDPOINT` | `/api/v1/fleet/ingest/{sn}/map` | path template | | `HTTP_TIMEOUT` | `30` | per-request timeout (s) |
| `STATE_DIR` | `/data/state` | upload-fingerprint state | | `SOFTWARE_ROS` | `foxy` | reported in `software.ros` |
### R1 / Go2 telemetry ### DDS / telemetry
| var | default | meaning | | var | default | meaning |
|---|---|---| |---|---|---|
| `DDS_INTERFACE` | R1 `eth10` / Go2 `eth0` | NIC that sees robot DDS | | `DDS_INTERFACE` | r1 `eth10` · else `eth0` | NIC that sees robot DDS |
| `DDS_DOMAIN` | `0` | DDS domain id | | `DDS_DOMAIN` | `0` | DDS domain id |
| `MAC_INTERFACE` | = `DDS_INTERFACE` | NIC whose MAC is reported | | `MAC_INTERFACE` | = `DDS_INTERFACE` | NIC whose MAC is reported |
| `R1_READ_FSM` | `0` | (R1) read loco FSM for status — read-only GET RPC | | `G1_READ_FSM` | `0` | read loco FSM for status (read-only GET RPC) |
| `R1_POSITION_SOURCE` / `GO2_POSITION_SOURCE` | `none` | `none` · `rosbridge` (· `sportmode` Go2) | | `POSITION_SOURCE` | `odom`\|`none` | `odom` (DDS) · `rosbridge` · `none` |
| `ROSBRIDGE_URL` | `ws://127.0.0.1:9090` | position source when rosbridge | | `ROSBRIDGE_URL` | `ws://127.0.0.1:9090` | position source when `rosbridge` |
| `LOW_SOC` | `15` | %→ `LOW_BATTERY` fault | | `MOTOR_TEMP_MAX` | `85` | °C → `MOTOR_OVERTEMP` fault |
| `MOTOR_TEMP_MAX` | `85` | °C→ `MOTOR_OVERTEMP` fault |
| `TELEMETRY_ENDPOINT` | `/api/v1/fleet/ingest/telemetry` | path | ### Maps
| var | default | meaning |
|---|---|---|
| `MAPS_DIR` | `/data/maps` | primary SLAM/nav maps mount |
| `EXTRA_MAP_DIRS` | `/data/nav2_maps` | extra roots (colon-sep) for Nav2/Pudu `pgm+yaml` sets |
| `MAP_SELECT` | `all` | `all` · `active` · `newest` |
| `MAP_POLL_INTERVAL` | `30` | map check cadence (s) |
| `MAP_MAX_UPLOAD_MB` | `7` | skip `.db` larger than this (server cap) |
| `MAP_UPLOAD_MODE` | `multipart` | `multipart` · `base64json` |
| `DATA_DIR` | `/data/web_data` | per-map places (waypoints) |
| `STATE_DIR` | `/data/state` | upload-fingerprint + start-time state |
### Alerts
| var | default | meaning |
|---|---|---|
| `LOW_SOC` | `50` | battery % → `LOW_BATTERY` fault + alert |
| `ALERT_SCAN_INTERVAL` | `10` | project-log scan cadence (s) |
| `ALERT_LOG_COOLDOWN` | `300` | per-signature re-alert gap (s) |
| `ALERT_LOG_PATTERNS` | Gemini-billing + error/critical/traceback | `CODE=regex` entries split by `;;` (case-sensitive; use `(?i)` inline) |
| `ALERT_BACKFILL_BYTES` | `8 MiB` | startup scan window so active errors alert on boot |
### Logs · remote · control
| var | default | meaning |
|---|---|---|
| `LOGS_INTERVAL` | `60` | log-ship cadence (s) |
| `PROJECT_LOG_CONTAINER` | `auto` | robot's Sanad container to tail (`auto` discovers it) |
| `PROJECT_LOG_PATH` / `_LABEL` / `_BACKFILL` / `_EXCLUDE` | — / — / `100` / access-log filter | project-log overrides |
| `REMOTE_ENABLE` | `1` | register the dashboard URL + ssh |
| `REMOTE_PORTS` | `8001,8014,8011-8013,8000,8080` | dashboard ports to probe |
| `REMOTE_URL` | — | pin an explicit URL (e.g. a public tunnel) |
| `SSH_REGISTER` / `SSH_USER` | `1` / `unitree` | register `ssh <user>@<ip>` |
| `CONTROL_ENABLE` | `0` | remote mode-**switch** (motion) — **off by design** |
| `CONTROL_STATUS_URL` | — | Sanad `/api/controller/status` for read-only loco mode |
--- ---
## 9. Per-robot specifics ## 11. Per-robot specifics
### G1 (map) — real maps live inside the nav container ### R1 (live on production)
- DDS link is **`eth10`** (= `192.168.123.164`); WiFi is the fleet LAN.
- Battery from `rt/lf/bmsstate` (`BmsState_`); coexists with the running `sanadr1`
app (DDS allows many readers).
- Maps: VSLAM `~/r1_vslam_docker/data` (RTAB-Map `.db`) **and** the converted
**Pudu office map** in `~/r1_nav2_docker/maps` (mounted at `/data/nav2_maps`).
The G1's saved maps are stored **inside** the `p4_Foxy_sanad` (Package_4 nav) ### G1 (⚠ STANDING HOLD — stage only, do not deploy)
container at `/home/unitree/marcus_nav2_test/maps`, which Package_4 does **not** - Do **not** deploy to the new G1 until the hold is lifted; stage maps only.
bind-mount to the host. So the uploader (which mounts a host dir) sees **0 maps** - Saved maps live **inside** the `p4_Foxy_sanad` nav container at
until you do one of: `~/marcus_nav2_test/maps` (not host-mounted by default) — add a host bind-mount
(and to `web/data/`) so the agent can read them, or use a shared volume.
1. Add a host bind-mount for `maps/` (and `web/data/`) to the Package_4 `nav` ### Go2 (⚠ unverified on hardware)
service, then set the uploader's `MAPS_HOST_DIR`/`DATA_HOST_DIR` to those paths. - Uses `unitree_go`; battery is nested in **`rt/lowstate.bms_state`** (no separate
2. Put both containers on a shared named volume for the maps dir. BMS topic). Image builds and imports, simulate is correct — but confirm the
`bms_state` current sign (charge polarity) and `sportmodestate` fields on a real Go2.
The `test` command seeds a throwaway fixture map so the upload path is verified
regardless.
### R1 (telemetry)
- DDS link is **`eth10`** (= `192.168.123.164`); `wlan0` is the fleet LAN.
- Battery: `rt/lf/bmsstate` (`BmsState_`), `soc` 0–100, charging from `current`
sign. Coexists fine with the running `sanadr1` app (DDS allows many readers).
### Go2 (telemetry) — ⚠️ unverified on hardware
- Uses `unitree_go` DDS. Battery is nested in **`rt/lowstate.bms_state`** (Go2 has
no separate BMS topic).
- Written from the SDK layout; image builds and `unitree_go` imports, simulate is
correct, but **not yet run on a real Go2** — confirm the `bms_state` current sign
(charging polarity) and `sportmodestate` fields on the robot.
--- ---
## 10. The fleet test server ## 12. The fleet test server
`fleet_test_server.py` stands in for the real fleet server so you can verify a `fleet_test_server.py` stands in for the real fleet server so you can verify a deploy
deploy end-to-end from your workstation. end-to-end from the workstation.
```bash ```bash
PORT=8799 REQLOG=/tmp/fleet.jsonl python3 fleet_test_server.py PORT=8799 REQLOG=/tmp/fleet.jsonl python3 fleet_test_server.py
``` ```
- Binds `0.0.0.0:8799`. - Binds `0.0.0.0:8799`; `GET /ping` → `200` (the reachability check the installer uses).
- `GET /ping` → `200` (reachability check the installer uses). - Accepts every ingest POST, logs each to `REQLOG`, prints a live summary.
- Accepts `POST …/map` and `POST …/telemetry`; logs each request to `REQLOG` and
prints a live summary.
`./fleet_install.sh test …` starts it automatically, triggers a post from the `./fleet_install.sh test …` starts it, triggers a post from the robot, asserts the
robot, asserts the payload arrived, and prints `PASS`/`FAIL`. payload arrived, and prints **PASS**/**FAIL**.
--- ---
## 11. How a deploy works internally ## 13. Fleet inventory
``` | robot | IP | SSH | arch | DDS iface | SN (name) | status |
install:
rsync agents/<type>/ → unitree@<ip>:~/sanad_api_<type>/ (--delete, minus .env/state)
write ~/sanad_api_<type>/.env (SERVER_URL, TOKEN, SN, iface…)
ssh: docker build -t sanad-api-<type>:latest . (native arm64)
ssh: docker create --name sanad-api-<type> <run-args> … (no docker restart policy)
write ~/.config/systemd/user/sanad-api-<type>.service
ssh: loginctl enable-linger ; systemctl --user enable --now …
run-args: --network host --env-file …/.env
(g1 also: -v maps:ro -v web_data:ro -v state)
```
`--network host` is required: G1 to reach `127.0.0.1:8765` (web_nav3) if used, and
R1/Go2 so the robot's DDS traffic is visible (DDS multicast doesn't cross a NAT
bridge).
---
## 12. Fleet inventory
| robot | agent(s) | IP | SSH | arch | DDS iface | SN |
|---|---|---|---|---|---|---| |---|---|---|---|---|---|---|
| G1 | telemetry + map (`g1`) | `10.255.254.58` | `unitree` (key) | arm64 | `eth0` | `E21D6000PB89GF88` (name `g1_58`) | | R1 | `10.255.254.82` (wired `192.168.123.164`) | `unitree` (key) | arm64 | `eth10` | `E39N4000Q6D7E70F` (`r1_82`) | **live on prod** · office map uploaded |
| R1 | telemetry (`r1`) | `10.255.254.82` | `unitree` (key) | arm64 | `eth10` | `E39N4000Q6D7E70F` (name `r1_82`) | | G1 | `10.255.254.58` | `unitree` (key) | arm64 | `eth0` | `E21D6000PB89GF88` (`g1_58`) | ⚠ hold + offline (battery) |
| Go2 | telemetry (`go2`) | *(TBD)* | `unitree` | arm64 | `eth0` | — | | Go2 | *(TBD)* | `unitree` | arm64 | `eth0` | — | unverified |
Workstation (deploy host + test fleet server): **`10.255.254.83`** (`wlp4s0`). Workstation (deploy host + test fleet server): **`10.255.254.83`** (`wlp4s0`).
Production fleet server: **`https://eco.yslootahrobotics.com`**.
--- ---
## 13. Troubleshooting ## 14. Troubleshooting
| symptom | cause / fix | | symptom | cause / fix |
|---|---| |---|---|
| `robot cannot reach the workstation server (http 000)` during `test` | robot→workstation blocked. Check both are on the same subnet; open the port if a firewall is on (`sudo ufw allow 8799`). | | `test`: robot can't reach the workstation (`http 000`) | same-subnet? open the port (`sudo ufw allow 8799`). |
| G1 uploads nothing / "0 maps" | no saved map, or maps aren't mounted — see [§9 G1](#g1-map--real-maps-live-inside-the-nav-container). | | map: "0 maps" / nothing uploaded | no saved map, or maps aren't mounted — see [§4](#4-maps-discovery--upload) and [§11 G1](#g1--standing-hold--stage-only-do-not-deploy). |
| R1/Go2 `telemetry POST failed … Connection refused` | the target server (e.g. the test server) is down. Point at the real server: `install … --server-ip <fleet> --token <real>`. | | `telemetry POST failed … Connection refused` | target server down / wrong URL. Re-point: `install … --server-url <real>`. |
| R1/Go2 `battery=null, status=offline` forever | DDS not seen. Wrong `DDS_INTERFACE` (R1 = `eth10`) or robot firmware down. Confirm with `ip -o addr` on the robot. | | `battery=null, status=offline` forever | DDS not seen. Wrong `DDS_INTERFACE` (R1 = `eth10`) or robot firmware down — check `ip -o addr` on the robot. |
| build is very slow (R1/Go2) | first build compiles CycloneDDS (minutes). Run detached and poll: `ssh … 'setsid bash -c "cd ~/sanad_api_r1 && docker build -t sanad-api-r1:latest . > build.log 2>&1" </dev/null &'` then watch `build.log`. Docker layer cache survives reboots. | | build very slow (r1/go2) | first build compiles CycloneDDS (minutes). Run detached: `setsid bash -c 'cd ~/sanad_api_r1 && docker build -t sanad-api-r1:latest . >build.log 2>&1' </dev/null &` then watch `build.log`. Layer cache survives reboots. |
| service didn't come up after reboot | linger must be on: `loginctl show-user <user> | grep Linger` → `Linger=yes`. The installer sets it. | | service didn't start after reboot | linger must be on: `loginctl show-user <user> \| grep Linger` → `Linger=yes` (the installer sets it). |
| `unbound variable` / weird prompt behavior | ensure you're on the current `fleet_install.sh` (uses `ssh -n` for command calls so SSH doesn't eat stdin). | | r1/go2 out of sync with g1 | you edited g1 without regenerating — run `python3 tools/gen_agents.py`. |
--- ---
## 14. Security notes ## 15. Security notes
- **Outbound only.** Agents open no inbound ports on the robot; they POST out over - **Outbound only.** No inbound port on the robot; POSTs out over HTTPS with a
HTTPS with a per-robot bearer token. per-robot bearer token.
- **Least privilege.** Runs as the `unitree` user (docker group), user-level - **Least privilege.** Runs as `unitree` (docker group), user-level systemd, no root.
systemd, no root. Read-only w.r.t. the robot (no motion commands). Read-only toward the robot — **no motion commands**.
- **Token handling.** `.env` holds the device token and is never rsynced back or - **Token handling.** `.env` holds the device token, is never rsynced back, and is
committed (`.gitignore`d). The installer writes it directly to the robot. `.gitignore`d. The installer writes it straight to the robot; it is never printed.
- **TLS.** `VERIFY_TLS=1` in production; `0` only for the local test server. - **TLS.** `VERIFY_TLS=1` in production; `0` only for the local test server.

82
agents/g1/.env.example
View File

@ -1,48 +1,74 @@
# sanad_api_g1 — copy to .env and fill in. docker compose reads it automatically. # sanad_api_g1 — copy to .env and fill in. docker compose reads it automatically.
# ONE agent = telemetry + map + logs + alerts + remote (see fleet/README.md).
# ── fleet server (REQUIRED — YS Lootah gives you these two) ────────────────── # ── fleet server (REQUIRED — YS Lootah gives you these two) ──────────────────
SERVER_URL=https://fleet.example.com SERVER_URL=https://fleet.example.com
DEVICE_TOKEN=REPLACE_WITH_DEVICE_TOKEN DEVICE_TOKEN=REPLACE_WITH_DEVICE_TOKEN
# ── identity ───────────────────────────────────────────────────────────────── # ── identity ─────────────────────────────────────────────────────────────────
# This robot's fleet id — used as {sn} in POST /api/v1/fleet/ingest/{sn}/map. # The robot's REAL Unitree serial — keys the robot on the server ({sn} routes).
SN=g1_7892 SN=E21D6000PB89GF88
# Friendly display name shown on the dashboard (default <model>_<ip-octet>).
ROBOT_NAME=g1_58
ROBOT_BRAND=unitree
ROBOT_TYPE=humanoid
ROBOT_MODEL=g1
# web_nav3 robot name: the maps subdir (<MAPS_DIR>/<ROBOT>/*.db) + X-Robot-Name. # web_nav3 robot name: the maps subdir (<MAPS_DIR>/<ROBOT>/*.db) + X-Robot-Name.
# Must match web_nav3's robot_config.yaml robot_name (default: sanad).
ROBOT=sanad ROBOT=sanad
# Optional: Sanad data dir whose size is reported in storage (as /host/<path>
# when the installer's read-only /:/host mount is used). Empty = omit.
STORAGE_DATA_PATH=
# ── where the map files live on the HOST (bind-mounted read-only) ──────────── # ── DDS (reading robot state) ────────────────────────────────────────────────
# Point these at the robot's web_nav3 install. Defaults assume a Package_4 # Use the interface that sees the robot's DDS traffic (G1 backpack: eth0).
# robot (nav container -> /home/unitree/marcus_nav2_test). For the workstation DDS_INTERFACE=eth0
# dev copy use .../Project/G1/Nav2_Projects/web_nav3/{maps,web/data}. DDS_DOMAIN=0
MAPS_HOST_DIR=/home/unitree/marcus_nav2_test/maps # Which NIC's MAC to report as the robot identity (defaults to DDS_INTERFACE).
DATA_HOST_DIR=/home/unitree/marcus_nav2_test/web/data # MAC_INTERFACE=eth0
STATE_HOST_DIR=./data/state
# Optional legacy per-robot places file (older sanad setups). Leave blank if # ── status / position options ────────────────────────────────────────────────
# each map already has its own places under DATA_DIR/<robot>/places/. # Read the loco FSM for a richer status (READ-ONLY GET RPC, never commands
# LEGACY_PLACES=/data/legacy/places.json # motion). 0 = derive status from battery + joint motion only (safe default).
G1_READ_FSM=0
# Position source: odom (rt/lf/odommodestate over DDS) | rosbridge | none.
G1_POSITION_SOURCE=odom
ROSBRIDGE_URL=ws://127.0.0.1:9090
# ── behaviour ──────────────────────────────────────────────────────────────── # ── fault thresholds ─────────────────────────────────────────────────────────
# Which maps to upload: all | active | newest. active/newest need WEB_NAV3_URL. LOW_SOC=50
MOTOR_TEMP_MAX=85
# ── cadence / transport ──────────────────────────────────────────────────────
POLL_INTERVAL=2
VERIFY_TLS=1
HTTP_TIMEOUT=10
# ── map sync (uploaded ONCE per content; status shown in telemetry "map") ────
# The installer mounts the robot's real maps dir at /data/maps (probed on the
# robot: rtabmap*.db / slam_toolbox yaml+pgm sets). Fallback = agent's own dir.
MAPS_DIR=/data/maps
# Extra map roots (colon-separated container paths) — Nav2/Pudu deploy maps
# (map.pgm + map.yaml). The installer mounts the robot's *_nav2_docker/maps
# here; safe to leave set even when nothing is mounted (missing dirs skipped).
EXTRA_MAP_DIRS=/data/nav2_maps
DATA_DIR=/data/web_data
STATE_DIR=/data/state
# Which maps to upload: all | active | newest. active needs WEB_NAV3_URL.
MAP_SELECT=all MAP_SELECT=all
# Optional — only used to learn which map is ACTIVE (MAP_SELECT=active). # Optional — only used to learn which map is ACTIVE (MAP_SELECT=active).
WEB_NAV3_URL=http://127.0.0.1:8765 WEB_NAV3_URL=http://127.0.0.1:8765
# Optional legacy per-robot places file (older sanad setups). Leave blank if
# Wire format the server accepts on the map endpoint: # each map already has its own places under DATA_DIR/<robot>/places/.
# multipart -> multipart/form-data: file field `db` + form field `meta` (JSON) # LEGACY_PLACES=/data/legacy/places.json
# base64json -> JSON body with the .db as `db_base64` + the same meta fields
MAP_UPLOAD_MODE=multipart MAP_UPLOAD_MODE=multipart
# Endpoint path template ({sn} substituted). Change only if the server differs. MAP_POLL_INTERVAL=30
MAP_ENDPOINT=/api/v1/fleet/ingest/{sn}/map # Skip .db maps larger than this (the prod server 413s at ~8 MB). Raster
# (pgm+yaml) maps are tiny and never skipped.
# Scan cadence for the loop (seconds). Maps rarely change, so 30s is plenty. MAP_MAX_UPLOAD_MB=7
POLL_INTERVAL=30
# Verify the fleet server's TLS cert (1 recommended; 0 only for self-signed dev).
VERIFY_TLS=1
HTTP_TIMEOUT=30
# ── logs + alerts ───────────────────────────────────────────────────────────── # ── logs + alerts ─────────────────────────────────────────────────────────────
# agent log lines shipped to /{sn}/logs every LOGS_INTERVAL; new faults POSTed
# to /{sn}/alert immediately (rising edge).
LOGS_INTERVAL=60 LOGS_INTERVAL=60
# ── project logs (shared alongside agent logs) ─────────────────────────────── # ── project logs (shared alongside agent logs) ───────────────────────────────
@ -66,7 +92,7 @@ REMOTE_INTERVAL=60
# ── control panel (READ-ONLY loco mode; remote SWITCH is off by design) ─────── # ── control panel (READ-ONLY loco mode; remote SWITCH is off by design) ───────
# The agent reads the Sanad dashboard's /api/controller/status (no DDS, no # The agent reads the Sanad dashboard's /api/controller/status (no DDS, no
# motion) and reports the current mode (zero_torque/damp/lock/running) + the # motion) and reports the current mode (G1: running/lock/squat/…) + the
# switchable set in telemetry.control. CONTROL_ENABLE stays 0: remote mode- # switchable set in telemetry.control. CONTROL_ENABLE stays 0: remote mode-
# SWITCHING is deliberately NOT built (it commands motion and can drop the robot; # SWITCHING is deliberately NOT built (it commands motion and can drop the robot;
# switching is done in person on the Sanad dashboard). Leave URL blank to use the # switching is done in person on the Sanad dashboard). Leave URL blank to use the

20
agents/g1/sanad_api_g1.py
View File

@ -131,6 +131,7 @@ class Config:
# map sync # map sync
robot: str robot: str
maps_dir: Path maps_dir: Path
extra_map_dirs: List[Path] # extra roots to scan for pgm+yaml sets (e.g. Nav2/Pudu maps)
web_data_dir: Optional[Path] web_data_dir: Optional[Path]
legacy_places: Optional[Path] legacy_places: Optional[Path]
web_nav3_url: str web_nav3_url: str
@ -210,6 +211,9 @@ class Config:
telemetry_endpoint=_env("TELEMETRY_ENDPOINT", "/api/v1/fleet/ingest/telemetry"), telemetry_endpoint=_env("TELEMETRY_ENDPOINT", "/api/v1/fleet/ingest/telemetry"),
robot=_env("ROBOT", "sanad"), robot=_env("ROBOT", "sanad"),
maps_dir=Path(_env("MAPS_DIR", "/data/maps")), maps_dir=Path(_env("MAPS_DIR", "/data/maps")),
# colon-separated extra roots (mounted Nav2/Pudu map dirs). Any *.yaml+*.pgm
# set found here is rendered to PNG and uploaded like a slam_toolbox map.
extra_map_dirs=[Path(p) for p in _env("EXTRA_MAP_DIRS", "").split(":") if p.strip()],
web_data_dir=Path(data_dir) if data_dir else None, web_data_dir=Path(data_dir) if data_dir else None,
legacy_places=Path(legacy) if legacy else None, legacy_places=Path(legacy) if legacy else None,
web_nav3_url=_env("WEB_NAV3_URL", "").rstrip("/"), web_nav3_url=_env("WEB_NAV3_URL", "").rstrip("/"),
@ -797,6 +801,7 @@ def _discover_slam_sets(cfg: Config) -> List[MapArtifact]:
"""Find slam_toolbox / map_server map sets: <stem>.yaml + <stem>.pgm """Find slam_toolbox / map_server map sets: <stem>.yaml + <stem>.pgm
(+ optional .posegraph/.data) under the maps roots and maps_slam/.""" (+ optional .posegraph/.data) under the maps roots and maps_slam/."""
roots = [cfg.maps_dir, cfg.maps_dir / cfg.robot, cfg.maps_dir / "maps_slam"] roots = [cfg.maps_dir, cfg.maps_dir / cfg.robot, cfg.maps_dir / "maps_slam"]
roots += list(cfg.extra_map_dirs) # Nav2/Pudu map dirs mounted via EXTRA_MAP_DIRS
seen: set = set() seen: set = set()
out: List[MapArtifact] = [] out: List[MapArtifact] = []
for root in roots: for root in roots:
@ -824,6 +829,11 @@ def _discover_slam_sets(cfg: Config) -> List[MapArtifact]:
out.append(MapArtifact(path=y, name=y.name, stem=y.stem, out.append(MapArtifact(path=y, name=y.name, stem=y.stem,
size=size, mtime=mtime, size=size, mtime=mtime,
fmt="slam_toolbox", files=files)) fmt="slam_toolbox", files=files))
# Pudu/Nav2 converter emits a plain map + a keepout-BAKED twin (obstacles baked
# in for Foxy, which has no KeepoutFilter). The baked one is the deploy map — drop
# the redundant plain twin so the fleet server gets one canonical map, not two.
baked = {m.stem[: -len("_keepout_baked")] for m in out if m.stem.endswith("_keepout_baked")}
out = [m for m in out if m.stem not in baked]
return out return out
@ -1772,6 +1782,7 @@ def main(argv: Optional[List[str]] = None) -> int:
ap = argparse.ArgumentParser(description="G1 fleet agent: telemetry + map sync") ap = argparse.ArgumentParser(description="G1 fleet agent: telemetry + map sync")
ap.add_argument("--simulate", action="store_true", help="synthetic DDS state (map scan stays real)") ap.add_argument("--simulate", action="store_true", help="synthetic DDS state (map scan stays real)")
ap.add_argument("--once", action="store_true", help="one map pass + one telemetry post, then exit") ap.add_argument("--once", action="store_true", help="one map pass + one telemetry post, then exit")
ap.add_argument("--map-only", action="store_true", help="upload discovered maps once (no DDS, no telemetry), then exit")
ap.add_argument("--dry-run", action="store_true", help="build payloads, never POST") ap.add_argument("--dry-run", action="store_true", help="build payloads, never POST")
ap.add_argument("--force", action="store_true", help="re-upload maps even if unchanged") ap.add_argument("--force", action="store_true", help="re-upload maps even if unchanged")
ap.add_argument("--list", action="store_true", help="list discovered maps and exit") ap.add_argument("--list", action="store_true", help="list discovered maps and exit")
@ -1804,6 +1815,14 @@ def main(argv: Optional[List[str]] = None) -> int:
cfg.position_source, cfg.maps_dir, cfg.map_poll_interval, cfg.position_source, cfg.maps_dir, cfg.map_poll_interval,
" [SIMULATE]" if args.simulate else "") " [SIMULATE]" if args.simulate else "")
session = requests.Session()
if args.map_only:
# map upload only — search all map roots (incl. EXTRA_MAP_DIRS) and ship,
# without opening DDS or posting telemetry (won't disturb a live feed).
map_sync_once(cfg, session, force=args.force, dry_run=args.dry_run)
return 0
reader = None reader = None
pos = None pos = None
if not args.simulate: if not args.simulate:
@ -1812,7 +1831,6 @@ def main(argv: Optional[List[str]] = None) -> int:
pos = RosbridgePosition(cfg) pos = RosbridgePosition(cfg)
time.sleep(1.0) time.sleep(1.0)
session = requests.Session()
tick = 0 tick = 0
def one_telemetry() -> None: def one_telemetry() -> None:

7
agents/go2/.env.example
View File

@ -40,11 +40,18 @@ STORAGE_DATA_PATH=
# The installer mounts the robot's real maps dir at /data/maps (probed on the # The installer mounts the robot's real maps dir at /data/maps (probed on the
# robot: rtabmap*.db / slam_toolbox yaml+pgm sets). Fallback = agent's own dir. # robot: rtabmap*.db / slam_toolbox yaml+pgm sets). Fallback = agent's own dir.
MAPS_DIR=/data/maps MAPS_DIR=/data/maps
# Extra map roots (colon-separated container paths) — Nav2/Pudu deploy maps
# (map.pgm + map.yaml). The installer mounts the robot's *_nav2_docker/maps
# here; safe to leave set even when nothing is mounted (missing dirs skipped).
EXTRA_MAP_DIRS=/data/nav2_maps
DATA_DIR=/data/web_data DATA_DIR=/data/web_data
STATE_DIR=/data/state STATE_DIR=/data/state
MAP_SELECT=all MAP_SELECT=all
MAP_UPLOAD_MODE=multipart MAP_UPLOAD_MODE=multipart
MAP_POLL_INTERVAL=30 MAP_POLL_INTERVAL=30
# Skip .db maps larger than this (the prod server 413s at ~8 MB). Raster
# (pgm+yaml) maps are tiny and never skipped.
MAP_MAX_UPLOAD_MB=7
# ── logs + alerts ───────────────────────────────────────────────────────────── # ── logs + alerts ─────────────────────────────────────────────────────────────
# agent log lines shipped to /{sn}/logs every LOGS_INTERVAL; new faults POSTed # agent log lines shipped to /{sn}/logs every LOGS_INTERVAL; new faults POSTed

20
agents/go2/sanad_api_go2.py
View File

@ -131,6 +131,7 @@ class Config:
# map sync # map sync
robot: str robot: str
maps_dir: Path maps_dir: Path
extra_map_dirs: List[Path] # extra roots to scan for pgm+yaml sets (e.g. Nav2/Pudu maps)
web_data_dir: Optional[Path] web_data_dir: Optional[Path]
legacy_places: Optional[Path] legacy_places: Optional[Path]
web_nav3_url: str web_nav3_url: str
@ -210,6 +211,9 @@ class Config:
telemetry_endpoint=_env("TELEMETRY_ENDPOINT", "/api/v1/fleet/ingest/telemetry"), telemetry_endpoint=_env("TELEMETRY_ENDPOINT", "/api/v1/fleet/ingest/telemetry"),
robot=_env("ROBOT", "sanad"), robot=_env("ROBOT", "sanad"),
maps_dir=Path(_env("MAPS_DIR", "/data/maps")), maps_dir=Path(_env("MAPS_DIR", "/data/maps")),
# colon-separated extra roots (mounted Nav2/Pudu map dirs). Any *.yaml+*.pgm
# set found here is rendered to PNG and uploaded like a slam_toolbox map.
extra_map_dirs=[Path(p) for p in _env("EXTRA_MAP_DIRS", "").split(":") if p.strip()],
web_data_dir=Path(data_dir) if data_dir else None, web_data_dir=Path(data_dir) if data_dir else None,
legacy_places=Path(legacy) if legacy else None, legacy_places=Path(legacy) if legacy else None,
web_nav3_url=_env("WEB_NAV3_URL", "").rstrip("/"), web_nav3_url=_env("WEB_NAV3_URL", "").rstrip("/"),
@ -751,6 +755,7 @@ def _discover_slam_sets(cfg: Config) -> List[MapArtifact]:
"""Find slam_toolbox / map_server map sets: <stem>.yaml + <stem>.pgm """Find slam_toolbox / map_server map sets: <stem>.yaml + <stem>.pgm
(+ optional .posegraph/.data) under the maps roots and maps_slam/.""" (+ optional .posegraph/.data) under the maps roots and maps_slam/."""
roots = [cfg.maps_dir, cfg.maps_dir / cfg.robot, cfg.maps_dir / "maps_slam"] roots = [cfg.maps_dir, cfg.maps_dir / cfg.robot, cfg.maps_dir / "maps_slam"]
roots += list(cfg.extra_map_dirs) # Nav2/Pudu map dirs mounted via EXTRA_MAP_DIRS
seen: set = set() seen: set = set()
out: List[MapArtifact] = [] out: List[MapArtifact] = []
for root in roots: for root in roots:
@ -778,6 +783,11 @@ def _discover_slam_sets(cfg: Config) -> List[MapArtifact]:
out.append(MapArtifact(path=y, name=y.name, stem=y.stem, out.append(MapArtifact(path=y, name=y.name, stem=y.stem,
size=size, mtime=mtime, size=size, mtime=mtime,
fmt="slam_toolbox", files=files)) fmt="slam_toolbox", files=files))
# Pudu/Nav2 converter emits a plain map + a keepout-BAKED twin (obstacles baked
# in for Foxy, which has no KeepoutFilter). The baked one is the deploy map — drop
# the redundant plain twin so the fleet server gets one canonical map, not two.
baked = {m.stem[: -len("_keepout_baked")] for m in out if m.stem.endswith("_keepout_baked")}
out = [m for m in out if m.stem not in baked]
return out return out
@ -1726,6 +1736,7 @@ def main(argv: Optional[List[str]] = None) -> int:
ap = argparse.ArgumentParser(description="Go2 fleet agent: telemetry + map sync") ap = argparse.ArgumentParser(description="Go2 fleet agent: telemetry + map sync")
ap.add_argument("--simulate", action="store_true", help="synthetic DDS state (map scan stays real)") ap.add_argument("--simulate", action="store_true", help="synthetic DDS state (map scan stays real)")
ap.add_argument("--once", action="store_true", help="one map pass + one telemetry post, then exit") ap.add_argument("--once", action="store_true", help="one map pass + one telemetry post, then exit")
ap.add_argument("--map-only", action="store_true", help="upload discovered maps once (no DDS, no telemetry), then exit")
ap.add_argument("--dry-run", action="store_true", help="build payloads, never POST") ap.add_argument("--dry-run", action="store_true", help="build payloads, never POST")
ap.add_argument("--force", action="store_true", help="re-upload maps even if unchanged") ap.add_argument("--force", action="store_true", help="re-upload maps even if unchanged")
ap.add_argument("--list", action="store_true", help="list discovered maps and exit") ap.add_argument("--list", action="store_true", help="list discovered maps and exit")
@ -1758,6 +1769,14 @@ def main(argv: Optional[List[str]] = None) -> int:
cfg.position_source, cfg.maps_dir, cfg.map_poll_interval, cfg.position_source, cfg.maps_dir, cfg.map_poll_interval,
" [SIMULATE]" if args.simulate else "") " [SIMULATE]" if args.simulate else "")
session = requests.Session()
if args.map_only:
# map upload only — search all map roots (incl. EXTRA_MAP_DIRS) and ship,
# without opening DDS or posting telemetry (won't disturb a live feed).
map_sync_once(cfg, session, force=args.force, dry_run=args.dry_run)
return 0
reader = None reader = None
pos = None pos = None
if not args.simulate: if not args.simulate:
@ -1766,7 +1785,6 @@ def main(argv: Optional[List[str]] = None) -> int:
pos = RosbridgePosition(cfg) pos = RosbridgePosition(cfg)
time.sleep(1.0) time.sleep(1.0)
session = requests.Session()
tick = 0 tick = 0
def one_telemetry() -> None: def one_telemetry() -> None:

7
agents/r1/.env.example
View File

@ -46,11 +46,18 @@ STORAGE_DATA_PATH=
# The installer mounts the robot's real maps dir at /data/maps (probed on the # The installer mounts the robot's real maps dir at /data/maps (probed on the
# robot: rtabmap*.db / slam_toolbox yaml+pgm sets). Fallback = agent's own dir. # robot: rtabmap*.db / slam_toolbox yaml+pgm sets). Fallback = agent's own dir.
MAPS_DIR=/data/maps MAPS_DIR=/data/maps
# Extra map roots (colon-separated container paths) — Nav2/Pudu deploy maps
# (map.pgm + map.yaml). The installer mounts the robot's *_nav2_docker/maps
# here; safe to leave set even when nothing is mounted (missing dirs skipped).
EXTRA_MAP_DIRS=/data/nav2_maps
DATA_DIR=/data/web_data DATA_DIR=/data/web_data
STATE_DIR=/data/state STATE_DIR=/data/state
MAP_SELECT=all MAP_SELECT=all
MAP_UPLOAD_MODE=multipart MAP_UPLOAD_MODE=multipart
MAP_POLL_INTERVAL=30 MAP_POLL_INTERVAL=30
# Skip .db maps larger than this (the prod server 413s at ~8 MB). Raster
# (pgm+yaml) maps are tiny and never skipped.
MAP_MAX_UPLOAD_MB=7
# ── logs + alerts ───────────────────────────────────────────────────────────── # ── logs + alerts ─────────────────────────────────────────────────────────────
# agent log lines shipped to /{sn}/logs every LOGS_INTERVAL; new faults POSTed # agent log lines shipped to /{sn}/logs every LOGS_INTERVAL; new faults POSTed

20
agents/r1/sanad_api_r1.py
View File

@ -131,6 +131,7 @@ class Config:
# map sync # map sync
robot: str robot: str
maps_dir: Path maps_dir: Path
extra_map_dirs: List[Path] # extra roots to scan for pgm+yaml sets (e.g. Nav2/Pudu maps)
web_data_dir: Optional[Path] web_data_dir: Optional[Path]
legacy_places: Optional[Path] legacy_places: Optional[Path]
web_nav3_url: str web_nav3_url: str
@ -210,6 +211,9 @@ class Config:
telemetry_endpoint=_env("TELEMETRY_ENDPOINT", "/api/v1/fleet/ingest/telemetry"), telemetry_endpoint=_env("TELEMETRY_ENDPOINT", "/api/v1/fleet/ingest/telemetry"),
robot=_env("ROBOT", "sanad"), robot=_env("ROBOT", "sanad"),
maps_dir=Path(_env("MAPS_DIR", "/data/maps")), maps_dir=Path(_env("MAPS_DIR", "/data/maps")),
# colon-separated extra roots (mounted Nav2/Pudu map dirs). Any *.yaml+*.pgm
# set found here is rendered to PNG and uploaded like a slam_toolbox map.
extra_map_dirs=[Path(p) for p in _env("EXTRA_MAP_DIRS", "").split(":") if p.strip()],
web_data_dir=Path(data_dir) if data_dir else None, web_data_dir=Path(data_dir) if data_dir else None,
legacy_places=Path(legacy) if legacy else None, legacy_places=Path(legacy) if legacy else None,
web_nav3_url=_env("WEB_NAV3_URL", "").rstrip("/"), web_nav3_url=_env("WEB_NAV3_URL", "").rstrip("/"),
@ -797,6 +801,7 @@ def _discover_slam_sets(cfg: Config) -> List[MapArtifact]:
"""Find slam_toolbox / map_server map sets: <stem>.yaml + <stem>.pgm """Find slam_toolbox / map_server map sets: <stem>.yaml + <stem>.pgm
(+ optional .posegraph/.data) under the maps roots and maps_slam/.""" (+ optional .posegraph/.data) under the maps roots and maps_slam/."""
roots = [cfg.maps_dir, cfg.maps_dir / cfg.robot, cfg.maps_dir / "maps_slam"] roots = [cfg.maps_dir, cfg.maps_dir / cfg.robot, cfg.maps_dir / "maps_slam"]
roots += list(cfg.extra_map_dirs) # Nav2/Pudu map dirs mounted via EXTRA_MAP_DIRS
seen: set = set() seen: set = set()
out: List[MapArtifact] = [] out: List[MapArtifact] = []
for root in roots: for root in roots:
@ -824,6 +829,11 @@ def _discover_slam_sets(cfg: Config) -> List[MapArtifact]:
out.append(MapArtifact(path=y, name=y.name, stem=y.stem, out.append(MapArtifact(path=y, name=y.name, stem=y.stem,
size=size, mtime=mtime, size=size, mtime=mtime,
fmt="slam_toolbox", files=files)) fmt="slam_toolbox", files=files))
# Pudu/Nav2 converter emits a plain map + a keepout-BAKED twin (obstacles baked
# in for Foxy, which has no KeepoutFilter). The baked one is the deploy map — drop
# the redundant plain twin so the fleet server gets one canonical map, not two.
baked = {m.stem[: -len("_keepout_baked")] for m in out if m.stem.endswith("_keepout_baked")}
out = [m for m in out if m.stem not in baked]
return out return out
@ -1772,6 +1782,7 @@ def main(argv: Optional[List[str]] = None) -> int:
ap = argparse.ArgumentParser(description="R1 fleet agent: telemetry + map sync") ap = argparse.ArgumentParser(description="R1 fleet agent: telemetry + map sync")
ap.add_argument("--simulate", action="store_true", help="synthetic DDS state (map scan stays real)") ap.add_argument("--simulate", action="store_true", help="synthetic DDS state (map scan stays real)")
ap.add_argument("--once", action="store_true", help="one map pass + one telemetry post, then exit") ap.add_argument("--once", action="store_true", help="one map pass + one telemetry post, then exit")
ap.add_argument("--map-only", action="store_true", help="upload discovered maps once (no DDS, no telemetry), then exit")
ap.add_argument("--dry-run", action="store_true", help="build payloads, never POST") ap.add_argument("--dry-run", action="store_true", help="build payloads, never POST")
ap.add_argument("--force", action="store_true", help="re-upload maps even if unchanged") ap.add_argument("--force", action="store_true", help="re-upload maps even if unchanged")
ap.add_argument("--list", action="store_true", help="list discovered maps and exit") ap.add_argument("--list", action="store_true", help="list discovered maps and exit")
@ -1804,6 +1815,14 @@ def main(argv: Optional[List[str]] = None) -> int:
cfg.position_source, cfg.maps_dir, cfg.map_poll_interval, cfg.position_source, cfg.maps_dir, cfg.map_poll_interval,
" [SIMULATE]" if args.simulate else "") " [SIMULATE]" if args.simulate else "")
session = requests.Session()
if args.map_only:
# map upload only — search all map roots (incl. EXTRA_MAP_DIRS) and ship,
# without opening DDS or posting telemetry (won't disturb a live feed).
map_sync_once(cfg, session, force=args.force, dry_run=args.dry_run)
return 0
reader = None reader = None
pos = None pos = None
if not args.simulate: if not args.simulate:
@ -1812,7 +1831,6 @@ def main(argv: Optional[List[str]] = None) -> int:
pos = RosbridgePosition(cfg) pos = RosbridgePosition(cfg)
time.sleep(1.0) time.sleep(1.0)
session = requests.Session()
tick = 0 tick = 0
def one_telemetry() -> None: def one_telemetry() -> None:

69
deploy.svg Normal file
View File

@ -0,0 +1,69 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 1060 230" font-family="'Segoe UI',Roboto,Helvetica,Arial,'DejaVu Sans',sans-serif">
<defs>
<marker id="d" markerWidth="9" markerHeight="9" refX="7" refY="3" orient="auto">
<path d="M0,0 L7,3 L0,6 Z" fill="#5f6368"/>
</marker>
</defs>
<rect width="1060" height="230" fill="#f6f8fa"/>
<text x="30" y="34" font-size="19" font-weight="700" fill="#202124">Deploy &#8212; workstation -&gt; robot, over SSH (no sudo)</text>
<text x="30" y="55" font-size="12.5" fill="#5f6368">One command: <tspan font-family="'Roboto Mono',Menlo,Consolas,monospace" fill="#3c4043">./fleet_install.sh install &lt;type&gt; &lt;ip&gt; --sn ... --token ... --server-url ...</tspan></text>
<!-- cards -->
<g>
<!-- 1 workstation -->
<rect x="45" y="78" width="145" height="92" rx="9" fill="#f1f6ff" stroke="#4285f4" stroke-width="1.5"/>
<text x="117" y="102" text-anchor="middle" font-size="12.5" font-weight="700" fill="#1a73e8">WORKSTATION</text>
<text x="117" y="127" text-anchor="middle" font-size="11" fill="#3c4043" font-family="'Roboto Mono',Menlo,Consolas,monospace">agents/&lt;type&gt;/</text>
<text x="117" y="147" text-anchor="middle" font-size="10.5" fill="#5f6368">single source</text>
<text x="117" y="161" text-anchor="middle" font-size="10.5" fill="#5f6368">of truth</text>
<!-- 2 rsync -->
<rect x="210" y="78" width="145" height="92" rx="9" fill="#f0faf3" stroke="#34a853"/>
<text x="282" y="102" text-anchor="middle" font-size="12.5" font-weight="700" fill="#188038">rsync -&gt; robot</text>
<text x="282" y="127" text-anchor="middle" font-size="10.5" fill="#3c4043" font-family="'Roboto Mono',Menlo,Consolas,monospace">~/sanad_api_&lt;type&gt;</text>
<text x="282" y="148" text-anchor="middle" font-size="10.5" fill="#5f6368">--delete</text>
<text x="282" y="162" text-anchor="middle" font-size="10.5" fill="#5f6368">keeps .env, state</text>
<!-- 3 build -->
<rect x="375" y="78" width="145" height="92" rx="9" fill="#f0faf3" stroke="#34a853"/>
<text x="447" y="102" text-anchor="middle" font-size="12.5" font-weight="700" fill="#188038">docker build</text>
<text x="447" y="127" text-anchor="middle" font-size="11" fill="#3c4043">native arm64</text>
<text x="447" y="148" text-anchor="middle" font-size="10.5" fill="#5f6368">on the robot</text>
<text x="447" y="162" text-anchor="middle" font-size="10.5" fill="#5f6368">(DDS stack for r1/go2)</text>
<!-- 4 create -->
<rect x="540" y="78" width="145" height="92" rx="9" fill="#f0faf3" stroke="#34a853"/>
<text x="612" y="102" text-anchor="middle" font-size="12.5" font-weight="700" fill="#188038">docker create</text>
<text x="612" y="127" text-anchor="middle" font-size="11" fill="#3c4043">container</text>
<text x="612" y="148" text-anchor="middle" font-size="10.5" fill="#5f6368">--network host</text>
<text x="612" y="162" text-anchor="middle" font-size="10.5" fill="#5f6368">-v /:/host:ro &#183; maps</text>
<!-- 5 systemd -->
<rect x="705" y="78" width="145" height="92" rx="9" fill="#f0faf3" stroke="#34a853"/>
<text x="777" y="102" text-anchor="middle" font-size="12.5" font-weight="700" fill="#188038">systemd --user</text>
<text x="777" y="127" text-anchor="middle" font-size="11" fill="#3c4043">+ enable-linger</text>
<text x="777" y="148" text-anchor="middle" font-size="10.5" fill="#5f6368">auto-start on boot</text>
<text x="777" y="162" text-anchor="middle" font-size="10.5" fill="#5f6368">Restart=always</text>
<!-- 6 running -->
<rect x="870" y="78" width="145" height="92" rx="9" fill="#202124"/>
<text x="942" y="102" text-anchor="middle" font-size="12.5" font-weight="700" fill="#ffffff">RUNNING</text>
<text x="942" y="127" text-anchor="middle" font-size="11" fill="#8ab4f8">-&gt; HTTPS -&gt;</text>
<text x="942" y="148" text-anchor="middle" font-size="10.5" fill="#bdc1c6">fleet server</text>
<text x="942" y="162" text-anchor="middle" font-size="10.5" fill="#bdc1c6">Bearer token</text>
</g>
<!-- arrows -->
<g stroke="#5f6368" stroke-width="2" marker-end="url(#d)">
<line x1="190" y1="124" x2="208" y2="124"/>
<line x1="355" y1="124" x2="373" y2="124"/>
<line x1="520" y1="124" x2="538" y2="124"/>
<line x1="685" y1="124" x2="703" y2="124"/>
<line x1="850" y1="124" x2="868" y2="124"/>
</g>
<!-- robot bracket -->
<path d="M210,188 L210,196 L1015,196 L1015,188" fill="none" stroke="#dadce0" stroke-width="1.5"/>
<text x="612" y="215" text-anchor="middle" font-size="11" fill="#80868b">on the robot &#160;&#183;&#160; ssh unitree@&lt;ip&gt; &#160;&#183;&#160; no root, no docker-compose</text>
</svg>
Side by Side

After

Width:  |  Height:  |  Size: 4.6 KiB

191
diagram.svg
View File

@ -1,114 +1,109 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 1000 600" font-family="system-ui,-apple-system,Segoe UI,Roboto,Helvetica,Arial,sans-serif"> <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 1060 500" font-family="'Segoe UI',Roboto,Helvetica,Arial,'DejaVu Sans',sans-serif">
<defs> <defs>
<marker id="arw" markerWidth="10" markerHeight="10" refX="8" refY="3" orient="auto"> <marker id="arw" markerWidth="9" markerHeight="9" refX="7" refY="3" orient="auto">
<path d="M0,0 L8,3 L0,6 Z" fill="#475569"/> <path d="M0,0 L7,3 L0,6 Z" fill="#5f6368"/>
</marker>
<marker id="arwL" markerWidth="12" markerHeight="12" refX="9" refY="4" orient="auto">
<path d="M0,0 L10,4 L0,8 Z" fill="#0e7490"/>
</marker> </marker>
</defs> </defs>
<!-- card --> <rect width="1060" height="500" fill="#f6f8fa"/>
<rect x="8" y="8" width="984" height="584" rx="16" fill="#f8fafc" stroke="#e2e8f0"/>
<!-- header --> <!-- title -->
<text x="32" y="46" font-size="22" font-weight="700" fill="#0f172a">Sanad Fleet — Architecture &amp; Data Pipeline</text> <text x="30" y="34" font-size="22" font-weight="700" fill="#202124">Sanad Fleet — one agent, five endpoints</text>
<text x="33" y="70" font-size="13" fill="#64748b">Robot state -&gt; containerized agent -&gt; fleet server · outbound HTTPS only</text> <text x="30" y="57" font-size="13.5" fill="#5f6368">Robot state -&gt; containerized agent -&gt; YS Lootah fleet server &#183; outbound HTTPS only, read-only toward the robot</text>
<!-- zone labels --> <!-- =============== ROBOT =============== -->
<text x="44" y="106" font-size="11" font-weight="700" letter-spacing="1.5" fill="#94a3b8">ON THE ROBOT (edge)</text> <rect x="24" y="84" width="300" height="372" rx="12" fill="#ffffff" stroke="#4285f4" stroke-width="1.5"/>
<text x="452" y="106" font-size="11" font-weight="700" letter-spacing="1.5" fill="#94a3b8">AGENT</text> <text x="40" y="110" font-size="15" font-weight="700" fill="#1a73e8">ROBOT &#183; G1 &#183; R1 &#183; Go2</text>
<text x="704" y="106" font-size="11" font-weight="700" letter-spacing="1.5" fill="#94a3b8">YS LOOTAH FLEET</text> <text x="40" y="128" font-size="11.5" fill="#5f6368">unitree_hg (G1/R1) &#183; unitree_go (Go2)</text>
<!-- ============ robot source cards ============ --> <rect x="40" y="144" width="268" height="150" rx="8" fill="#f1f6ff" stroke="#c6dafc"/>
<!-- G1 --> <text x="54" y="166" font-size="12.5" font-weight="700" fill="#202124">DDS topics</text>
<g> <text x="54" y="190" font-size="11" fill="#3c4043" font-family="'Roboto Mono',Menlo,Consolas,monospace">rt/lf/bmsstate</text><text x="182" y="190" font-size="11" fill="#5f6368">-&gt; battery, SoH</text>
<rect x="40" y="130" width="280" height="112" rx="10" fill="#ffffff" stroke="#cbd5e1"/> <text x="54" y="212" font-size="11" fill="#3c4043" font-family="'Roboto Mono',Menlo,Consolas,monospace">rt/lowstate</text><text x="182" y="212" font-size="11" fill="#5f6368">-&gt; temps, faults</text>
<rect x="40" y="130" width="6" height="112" rx="3" fill="#b45309"/> <text x="54" y="234" font-size="11" fill="#3c4043" font-family="'Roboto Mono',Menlo,Consolas,monospace">rt/lf/odommodestate</text>
<text x="62" y="160" font-size="15" font-weight="700" fill="#0f172a">G1 — map + telemetry</text> <text x="54" y="250" font-size="11" fill="#5f6368">-&gt; position</text>
<text x="62" y="184" font-size="11.5" fill="#475569">map: web_nav3 .db + places (files)</text> <text x="54" y="274" font-size="11" fill="#3c4043" font-family="'Roboto Mono',Menlo,Consolas,monospace">sport RPC 7001</text><text x="182" y="274" font-size="11" fill="#5f6368">-&gt; loco mode (ro)</text>
<text x="62" y="205" font-size="11" fill="#64748b" font-family="ui-monospace,Menlo,Consolas,monospace">telem: rt/lf/bmsstate · rt/lowstate</text>
<text x="62" y="224" font-size="11" fill="#64748b" font-family="ui-monospace,Menlo,Consolas,monospace">position: rt/lf/odommodestate</text> <rect x="40" y="304" width="268" height="66" rx="8" fill="#f1f6ff" stroke="#c6dafc"/>
<text x="54" y="326" font-size="12.5" font-weight="700" fill="#202124">Files &#183; via /host:ro mount</text>
<text x="54" y="348" font-size="11" fill="#5f6368">SLAM <tspan font-family="'Roboto Mono',Menlo,Consolas,monospace" fill="#3c4043">.db</tspan> &#183; Nav2/Pudu <tspan font-family="'Roboto Mono',Menlo,Consolas,monospace" fill="#3c4043">map.pgm+yaml</tspan></text>
<text x="54" y="363" font-size="10.5" fill="#80868b">rendered to PNG for upload</text>
<rect x="40" y="380" width="268" height="64" rx="8" fill="#f1f6ff" stroke="#c6dafc"/>
<text x="54" y="402" font-size="12.5" font-weight="700" fill="#202124">Sanad app</text>
<text x="54" y="422" font-size="11" fill="#5f6368">docker json-log -&gt; project logs</text>
<text x="54" y="438" font-size="11" fill="#5f6368">dashboard :8001 -&gt; remote URL</text>
<!-- robot -> agent -->
<line x1="324" y1="250" x2="380" y2="250" stroke="#5f6368" stroke-width="2" marker-end="url(#arw)"/>
<text x="352" y="242" text-anchor="middle" font-size="10.5" fill="#5f6368">reads</text>
<!-- =============== AGENT =============== -->
<rect x="380" y="84" width="300" height="372" rx="12" fill="#ffffff" stroke="#34a853" stroke-width="2"/>
<text x="530" y="110" text-anchor="middle" font-size="16" font-weight="700" fill="#188038">sanad_api_&lt;type&gt;</text>
<text x="530" y="130" text-anchor="middle" font-size="11.5" fill="#5f6368">Docker &#183; --network host &#183; user systemd &#183; no sudo</text>
<text x="530" y="148" text-anchor="middle" font-size="11.5" fill="#5f6368">read-only &#183; never commands motion</text>
<!-- loop rows -->
<g font-family="'Segoe UI',Roboto,Helvetica,Arial,'DejaVu Sans',sans-serif">
<!-- telemetry -->
<rect x="398" y="167" width="264" height="42" rx="7" fill="#f1f6ff" stroke="#c6dafc"/><rect x="398" y="167" width="5" height="42" rx="2.5" fill="#1a73e8"/>
<text x="414" y="185" font-size="12.5" font-weight="700" fill="#202124">Telemetry &#183; 2s</text>
<text x="414" y="201" font-size="10.5" fill="#5f6368">battery &#183; status &#183; position &#183; faults &#183; control mode</text>
<!-- map -->
<rect x="398" y="219" width="264" height="42" rx="7" fill="#f0faf3" stroke="#b7e1c4"/><rect x="398" y="219" width="5" height="42" rx="2.5" fill="#34a853"/>
<text x="414" y="237" font-size="12.5" font-weight="700" fill="#202124">Map sync &#183; 30s</text>
<text x="414" y="253" font-size="10.5" fill="#5f6368">discover pgm+yaml / .db &#183; upload once per change</text>
<!-- alert -->
<rect x="398" y="271" width="264" height="42" rx="7" fill="#fdecea" stroke="#f5c0ba"/><rect x="398" y="271" width="5" height="42" rx="2.5" fill="#ea4335"/>
<text x="414" y="289" font-size="12.5" font-weight="700" fill="#202124">Alert scan &#183; 10s</text>
<text x="414" y="305" font-size="10.5" fill="#5f6368">battery&lt;=50 &#183; Gemini billing &#183; any error</text>
<!-- logs -->
<rect x="398" y="323" width="264" height="42" rx="7" fill="#f7f0fe" stroke="#d9c2f5"/><rect x="398" y="323" width="5" height="42" rx="2.5" fill="#a142f4"/>
<text x="414" y="341" font-size="12.5" font-weight="700" fill="#202124">Log ship &#183; 60s</text>
<text x="414" y="357" font-size="10.5" fill="#5f6368">agent lines + <tspan font-family="'Roboto Mono',Menlo,Consolas,monospace">[sanadr1-logs]</tspan></text>
<!-- remote -->
<rect x="398" y="375" width="264" height="42" rx="7" fill="#fef4e5" stroke="#fadfa8"/><rect x="398" y="375" width="5" height="42" rx="2.5" fill="#f29900"/>
<text x="414" y="393" font-size="12.5" font-weight="700" fill="#202124">Remote register &#183; 60s</text>
<text x="414" y="409" font-size="10.5" fill="#5f6368">dashboard URL + <tspan font-family="'Roboto Mono',Menlo,Consolas,monospace">ssh unitree@ip</tspan></text>
</g> </g>
<!-- R1 --> <text x="530" y="437" text-anchor="middle" font-size="10.5" fill="#80868b">all three generated from agents/g1 &#183; gen_agents.py</text>
<g>
<rect x="40" y="258" width="280" height="112" rx="10" fill="#ffffff" stroke="#cbd5e1"/> <!-- agent -> server arrows -->
<rect x="40" y="258" width="6" height="112" rx="3" fill="#0e7c86"/> <g stroke-width="2" marker-end="url(#arw)">
<text x="62" y="288" font-size="15" font-weight="700" fill="#0f172a">R1 — telemetry</text> <line x1="680" y1="188" x2="736" y2="188" stroke="#1a73e8"/>
<text x="62" y="312" font-size="11.5" fill="#475569">source: DDS (unitree_hg)</text> <line x1="680" y1="240" x2="736" y2="240" stroke="#34a853"/>
<text x="62" y="333" font-size="11" fill="#64748b" font-family="ui-monospace,Menlo,Consolas,monospace">rt/lf/bmsstate · rt/lowstate</text> <line x1="680" y1="292" x2="736" y2="292" stroke="#ea4335"/>
<text x="62" y="352" font-size="11" fill="#64748b" font-family="ui-monospace,Menlo,Consolas,monospace">iface eth10</text> <line x1="680" y1="344" x2="736" y2="344" stroke="#a142f4"/>
</g> <line x1="680" y1="396" x2="736" y2="396" stroke="#f29900"/>
<!-- Go2 -->
<g>
<rect x="40" y="386" width="280" height="112" rx="10" fill="#ffffff" stroke="#cbd5e1"/>
<rect x="40" y="386" width="6" height="112" rx="3" fill="#4f46e5"/>
<text x="62" y="416" font-size="15" font-weight="700" fill="#0f172a">Go2 — telemetry</text>
<text x="62" y="440" font-size="11.5" fill="#475569">source: DDS (unitree_go)</text>
<text x="62" y="461" font-size="11" fill="#64748b" font-family="ui-monospace,Menlo,Consolas,monospace">rt/lowstate.bms_state</text>
<text x="62" y="480" font-size="11" fill="#94a3b8">unverified on hardware</text>
</g> </g>
<!-- source -> agent arrows --> <!-- =============== SERVER =============== -->
<line x1="322" y1="186" x2="366" y2="230" stroke="#475569" stroke-width="2" marker-end="url(#arw)"/> <rect x="736" y="84" width="300" height="372" rx="12" fill="#ffffff" stroke="#5f6368" stroke-width="1.5"/>
<line x1="322" y1="314" x2="366" y2="314" stroke="#475569" stroke-width="2" marker-end="url(#arw)"/> <text x="886" y="110" text-anchor="middle" font-size="15" font-weight="700" fill="#202124">Fleet server</text>
<line x1="322" y1="442" x2="366" y2="398" stroke="#475569" stroke-width="2" marker-end="url(#arw)"/> <text x="886" y="129" text-anchor="middle" font-size="11.5" fill="#5f6368" font-family="'Roboto Mono',Menlo,Consolas,monospace">eco.yslootahrobotics.com</text>
<text x="886" y="148" text-anchor="middle" font-size="10.5" fill="#80868b">Authorization: Bearer &lt;token&gt; &#183; keyed by sn</text>
<!-- ============ agent ============ -->
<g>
<rect x="370" y="130" width="250" height="368" rx="12" fill="#ecfeff" stroke="#06b6d4"/>
<text x="495" y="164" text-anchor="middle" font-size="16" font-weight="700" fill="#0e7490">sanad_api_&lt;type&gt;</text>
<text x="495" y="187" text-anchor="middle" font-size="11.5" fill="#0f766e">Docker · --network host</text>
<text x="495" y="205" text-anchor="middle" font-size="11.5" fill="#0f766e">read-only · never moves</text>
<!-- read/build -->
<text x="495" y="238" text-anchor="middle" font-size="11.5" font-weight="600" fill="#334155">read -&gt; build payload -&gt; POST</text>
<!-- systemd box -->
<rect x="392" y="256" width="206" height="66" rx="8" fill="#ffffff" stroke="#94a3b8"/>
<text x="495" y="282" text-anchor="middle" font-size="12.5" font-weight="700" fill="#0f172a">systemd --user service</text>
<text x="495" y="303" text-anchor="middle" font-size="11" fill="#475569">auto-start on boot · no sudo</text>
<!-- heartbeat box -->
<rect x="392" y="336" width="206" height="62" rx="8" fill="#fff7ed" stroke="#fdba74"/>
<text x="495" y="360" text-anchor="middle" font-size="12" font-weight="600" fill="#9a3412">heartbeat if source silent</text>
<text x="495" y="380" text-anchor="middle" font-size="11" fill="#b45309">-&gt; robot stays "online"</text>
<text x="495" y="428" text-anchor="middle" font-size="11" fill="#0f766e">loop: 2 s telemetry · 30 s map (on change)</text>
<text x="495" y="452" text-anchor="middle" font-size="10.5" fill="#64748b">deployed via fleet_install.sh</text>
<text x="495" y="470" text-anchor="middle" font-size="10.5" fill="#64748b">(rsync + docker build on robot)</text>
</g>
<!-- agent -> server -->
<line x1="620" y1="235" x2="700" y2="200" stroke="#0e7490" stroke-width="2.5" marker-end="url(#arwL)"/>
<text x="660" y="180" text-anchor="middle" font-size="11" font-weight="700" fill="#0e7490">HTTPS POST</text>
<text x="660" y="222" text-anchor="middle" font-size="10" fill="#0e7490">Bearer token</text>
<!-- ============ fleet server ============ -->
<g>
<rect x="700" y="130" width="260" height="168" rx="12" fill="#0f172a"/>
<text x="718" y="160" font-size="15" font-weight="700" fill="#e2e8f0">Fleet Server — ingest API</text>
<text x="718" y="190" font-size="11" fill="#7dd3fc" font-family="ui-monospace,Menlo,Consolas,monospace">POST …/ingest/telemetry</text>
<text x="928" y="190" text-anchor="end" font-size="10" fill="#94a3b8">~2 s</text>
<text x="718" y="214" font-size="11" fill="#fcd34d" font-family="ui-monospace,Menlo,Consolas,monospace">POST …/ingest/{sn}/map</text>
<text x="928" y="214" text-anchor="end" font-size="10" fill="#94a3b8">on change</text>
<line x1="718" y1="234" x2="942" y2="234" stroke="#334155"/>
<text x="718" y="258" font-size="10.5" fill="#94a3b8" font-family="ui-monospace,Menlo,Consolas,monospace">Authorization: Bearer &lt;token&gt;</text>
<text x="718" y="280" font-size="10.5" fill="#64748b">keyed by sn · mac in telemetry</text>
</g>
<!-- server -> dashboard -->
<line x1="830" y1="298" x2="830" y2="332" stroke="#475569" stroke-width="2" marker-end="url(#arw)"/>
<g> <g>
<rect x="700" y="334" width="260" height="96" rx="12" fill="#ffffff" stroke="#cbd5e1"/> <rect x="754" y="167" width="264" height="42" rx="7" fill="#f1f6ff" stroke="#c6dafc"/>
<text x="830" y="368" text-anchor="middle" font-size="14" font-weight="700" fill="#0f172a">Dashboard · Storage · Alerts</text> <text x="768" y="185" font-size="11.5" font-weight="700" fill="#1a73e8" font-family="'Roboto Mono',Menlo,Consolas,monospace">POST /ingest/telemetry</text>
<text x="830" y="392" text-anchor="middle" font-size="11" fill="#64748b">live status · map view · fault alerts</text> <text x="768" y="201" font-size="10.5" fill="#5f6368">every ~2s &#183; 27 fields</text>
<text x="830" y="411" text-anchor="middle" font-size="10.5" fill="#94a3b8">(YS Lootah side — already built)</text> <rect x="754" y="219" width="264" height="42" rx="7" fill="#f0faf3" stroke="#b7e1c4"/>
<text x="768" y="237" font-size="11.5" font-weight="700" fill="#188038" font-family="'Roboto Mono',Menlo,Consolas,monospace">POST /ingest/{sn}/map</text>
<text x="768" y="253" font-size="10.5" fill="#5f6368">on change &#183; PNG + resolution + origin</text>
<rect x="754" y="271" width="264" height="42" rx="7" fill="#fdecea" stroke="#f5c0ba"/>
<text x="768" y="289" font-size="11.5" font-weight="700" fill="#c5221f" font-family="'Roboto Mono',Menlo,Consolas,monospace">POST /ingest/{sn}/alert</text>
<text x="768" y="305" font-size="10.5" fill="#5f6368">on new fault (strings)</text>
<rect x="754" y="323" width="264" height="42" rx="7" fill="#f7f0fe" stroke="#d9c2f5"/>
<text x="768" y="341" font-size="11.5" font-weight="700" fill="#8430ce" font-family="'Roboto Mono',Menlo,Consolas,monospace">POST /ingest/{sn}/logs</text>
<text x="768" y="357" font-size="10.5" fill="#5f6368">every 60s &#183; ring-buffered lines</text>
<rect x="754" y="375" width="264" height="42" rx="7" fill="#fef4e5" stroke="#fadfa8"/>
<text x="768" y="393" font-size="11.5" font-weight="700" fill="#b06000" font-family="'Roboto Mono',Menlo,Consolas,monospace">POST /ingest/{sn}/remote</text>
<text x="768" y="409" font-size="10.5" fill="#5f6368">kind: web + ssh</text>
</g> </g>
<text x="886" y="437" text-anchor="middle" font-size="10.5" fill="#80868b">also GET /ingest/{sn}/commands (+ ack)</text>
<!-- footer --> <!-- footer -->
<line x1="32" y1="524" x2="968" y2="524" stroke="#e2e8f0"/> <line x1="30" y1="474" x2="1030" y2="474" stroke="#e0e0e0"/>
<text x="32" y="548" font-size="11.5" fill="#64748b">Outbound HTTPS only — no inbound port on the robot. Each robot type = its own Docker image, installed &amp; managed over SSH by</text> <text x="30" y="492" font-size="11" fill="#80868b">No inbound port on the robot &#183; each robot type is its own Docker image &#183; installed &amp; managed over SSH by <tspan font-family="'Roboto Mono',Menlo,Consolas,monospace" fill="#5f6368">fleet_install.sh</tspan> &#183; user-level systemd auto-start, no sudo.</text>
<text x="32" y="566" font-size="11.5" font-weight="600" fill="#475569" font-family="ui-monospace,Menlo,Consolas,monospace">fleet_install.sh<tspan font-family="system-ui" font-weight="400" fill="#64748b"> · user-level systemd auto-start, no sudo.</tspan></text>
</svg> </svg>

Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 7.6 KiB

After

Width:  |  Height:  |  Size: 9.2 KiB

18
fleet_install.sh
View File

@ -43,11 +43,12 @@ detect_server_ip(){ [ -n "$SERVER_IP" ] && { echo "$SERVER_IP"; return; }
# Every agent now syncs maps: find where THIS robot's Sanad/SLAM stack keeps its # Every agent now syncs maps: find where THIS robot's Sanad/SLAM stack keeps its
# maps on the host. Falls back to the agent's own dirs (agent then reports # maps on the host. Falls back to the agent's own dirs (agent then reports
# map: no_map until a map exists / is exposed). # map: no_map until a map exists / is exposed).
MAPS_HOST=""; DATA_HOST="" MAPS_HOST=""; DATA_HOST=""; EXTRA_MAPS_HOST=""
probe_maps_dirs(){ probe_maps_dirs(){
local t="$1"; local rdir; rdir="$(rdir_of "$t")" local t="$1"; local rdir; rdir="$(rdir_of "$t")"
MAPS_HOST="/home/$USER_/$rdir/maps" MAPS_HOST="/home/$USER_/$rdir/maps"
DATA_HOST="/home/$USER_/$rdir/web_data" DATA_HOST="/home/$USER_/$rdir/web_data"
EXTRA_MAPS_HOST=""
if [ "$t" = g1 ]; then if [ "$t" = g1 ]; then
for c in "/home/$USER_/marcus_nav2_test/maps" \ for c in "/home/$USER_/marcus_nav2_test/maps" \
"/home/$USER_/sanad_deploy/Sanad_Package_4/nav/data/ros"; do "/home/$USER_/sanad_deploy/Sanad_Package_4/nav/data/ros"; do
@ -63,15 +64,27 @@ probe_maps_dirs(){
found=$(rmt "find /home/$USER_ -maxdepth 4 \( -name 'rtabmap*.db' -o -name '*.posegraph' \) 2>/dev/null | head -1" 2>/dev/null) found=$(rmt "find /home/$USER_ -maxdepth 4 \( -name 'rtabmap*.db' -o -name '*.posegraph' \) 2>/dev/null | head -1" 2>/dev/null)
[ -n "$found" ] && MAPS_HOST="$(dirname "$found")" [ -n "$found" ] && MAPS_HOST="$(dirname "$found")"
fi fi
# Nav2/Pudu deploy maps (map.pgm + map.yaml, incl. keepout-baked) usually live in
# a *_nav2_docker/maps dir SEPARATE from the SLAM db above. Find it and mount it
# as an extra map root so the agent discovers+uploads it too. Skip if it's the
# same as the primary maps mount (g1 already maps marcus_nav2_test/maps).
local e
e=$(rmt "for d in /home/$USER_/${t}_nav2_docker/maps /home/$USER_/marcus_nav2_test/maps \
/home/$USER_/go2_nav2_docker/maps /home/$USER_/r1_nav2_docker/maps; do \
ls \$d/*.yaml >/dev/null 2>&1 && { echo \$d; break; }; done" 2>/dev/null | head -1)
if [ -n "$e" ] && [ "$e" != "$MAPS_HOST" ]; then EXTRA_MAPS_HOST="$e"; fi
} }
run_args(){ run_args(){
# /host:ro → real disk stats; maps/web_data ro + state rw → map sync # /host:ro → real disk stats; maps/web_data ro + state rw → map sync
local rdir; rdir="$(rdir_of "$1")" local rdir; rdir="$(rdir_of "$1")"
# extra Nav2/Pudu map dir (mounted at /data/nav2_maps, matches EXTRA_MAP_DIRS)
local extra=""
[ -n "$EXTRA_MAPS_HOST" ] && extra=" -v ${EXTRA_MAPS_HOST}:/data/nav2_maps:ro"
echo "--network host --env-file /home/$USER_/$rdir/.env -v /:/host:ro \ echo "--network host --env-file /home/$USER_/$rdir/.env -v /:/host:ro \
-v ${MAPS_HOST:-/home/$USER_/$rdir/maps}:/data/maps:ro \ -v ${MAPS_HOST:-/home/$USER_/$rdir/maps}:/data/maps:ro \
-v ${DATA_HOST:-/home/$USER_/$rdir/web_data}:/data/web_data:ro \ -v ${DATA_HOST:-/home/$USER_/$rdir/web_data}:/data/web_data:ro \
-v /home/$USER_/$rdir/state:/data/state" -v /home/$USER_/$rdir/state:/data/state${extra}"
} }
# Resolve the SERVER_URL + VERIFY_TLS for a deploy: a full --server-url (real # Resolve the SERVER_URL + VERIFY_TLS for a deploy: a full --server-url (real
@ -114,6 +127,7 @@ POLL_INTERVAL=2
LOW_SOC=50 LOW_SOC=50
ROBOT=sanad ROBOT=sanad
MAPS_DIR=/data/maps MAPS_DIR=/data/maps
EXTRA_MAP_DIRS=/data/nav2_maps
DATA_DIR=/data/web_data DATA_DIR=/data/web_data
STATE_DIR=/data/state STATE_DIR=/data/state
MAP_SELECT=all MAP_SELECT=all