diff --git a/README.md b/README.md index 2f806a7..8b20310 100644 --- a/README.md +++ b/README.md @@ -139,68 +139,9 @@ a failure that looks exactly like healthy telemetry. at a plausible-looking wrong field. - **`map`** — `no_map`; there are no saved maps on this robot yet. -### What this robot reports that the X2 could not - -`motor_temp` (the X2 publishes no per-motor temperature at all — permanently -null there), `control.mode` by name plus the real transition set (the X2 had no -documented FSM id scheme and reported `"unknown"`), and three hardware fault -channels: `POWER_FAULT`/`POWER_DISABLED` from `PowerInfo.error_code`/`enable`, -and `MOTOR_FAULT`/`MOTOR_OFFLINE` from `MotorDebug.error_code[]`/`offline[]`. - --- -## 5. Two fleet servers, two tokens - -The X2 agent posts to one server. This one keeps a **list** of endpoints and -sends the identical payload to each: - -```ini -SERVER_URL=https://eco-dev.yslootahrobotics.com # server 1 -DEVICE_TOKEN= -SERVER_ENABLE=1 - -SERVER_2_URL=https://eco.yslootahrobotics.com # server 2 -SERVER_2_TOKEN= # ← paste eco's token -SERVER_2_ENABLE=0 # ← then flip to 1 -``` - -**The supplied token works on `eco-dev` and is rejected by `eco`.** Tested from -the robot: `eco-dev` returns `200 {"ok":true,"robot_id":162}`; `eco` returns -`401` for that token, for a garbage token and for no token at all — i.e. it is -the application's auth layer, not a firewall (the host is reachable; `/` answers -`302`). The two deployments maintain independent token stores. - -So server 2 ships **configured but disabled**. To turn it on: - -```bash -ssh ubuntu@10.210.136.150 -sudo sed -i 's#^SERVER_2_TOKEN=.*#SERVER_2_TOKEN=#; \ - s#^SERVER_2_ENABLE=.*#SERVER_2_ENABLE=1#' /opt/sanad_api_eng/.env -sudo systemctl restart sanad-api-eng -``` - -Every log line then carries both results: - -``` -telemetry ok: battery=21 … -> eco-dev=200 eco=200 -``` - -A failing server never blocks the other — each POST is independent, and repeated -failures are reported once per minute with a suppressed-count rather than once -every 2 s: - -``` -ERROR telemetry -> eco FAILED: HTTP 401 Unauthorized <- token rejected by eco; - each fleet server issues its OWN token … [+3 more since last report] -INFO telemetry ok: battery=21 … -> eco-dev=200 eco=401 -``` - -Maps track upload state **per server**, so enabling `eco` later uploads every -existing map to it rather than finding them already marked "uploaded". - ---- - -## 6. Operating it +## 5. Operating it ```bash # live log @@ -257,83 +198,4 @@ sudo -E python3 /opt/sanad_api_eng/sanad_api_eng.py --once # one real post, (both need the ROS overlay: `set +u; . /app/applications/install/bringup/ros_env.sh`) ---- -## 7. Why it runs as a root system service - -The X2 agent runs as a `--user` unit and has one open item: `linger` is off, so -it does **not** come back after a power cycle. That is fixed here by running a -**system** unit, which also solves a second problem: - -- **Reboot survival** — `systemctl enable`d, no `loginctl enable-linger` - needed. Verified `enabled` + `active`. -- **Project logs** — the Sanad app runs in the `sanad-t8` container and its log - lives at `/var/lib/docker/containers//-json.log`, root-owned mode 600. - As `ubuntu` that read is `EACCES` and `project_logs` would stay null forever. - -The unit sources the robot's **own** environment file so the agent follows -EngineAI's settings instead of restating them: - -```ini -ExecStart=/bin/bash -c 'set +u; \ - . /opt/ros/humble/setup.bash >/dev/null 2>&1 || true; \ - . /app/applications/install/bringup/ros_env.sh >/dev/null 2>&1 || true; \ - exec /usr/bin/python3 -u /opt/sanad_api_eng/sanad_api_eng.py' -``` - -`set +u` is required — ROS's `setup.bash` reads unbound variables and would -abort the unit under `set -u`. `ros_env.sh` is what sets `ROS_DOMAIN_ID=69`, -`RMW_IMPLEMENTATION=rmw_cyclonedds_cpp` and the CycloneDDS interface pinning; -without it `ros2 topic list` shows 2 topics instead of 44 and every field -silently reports null. - -The robot's own ROS apps run under **supervisord**, not systemd. The agent -deliberately does not join that group: a restart of the agent must never be able -to disturb the robot's motion stack. - -### Verified behaviour - -| | | -|---|---| -| `kill -9` | auto-restarts in ~5 s (`Restart=always`), `NRestarts: 1` | -| `systemctl restart` | `received SIGINT — stopping` → `Deactivated successfully` — clean, no abort | -| `systemctl is-enabled` | `enabled` — starts at boot | - -The clean stop matters: rclpy's CycloneDDS threads abort the process at -interpreter teardown (`terminate called without an active exception`), which -systemd records as a failed exit and which would mask a real crash. The agent -shuts ROS down explicitly on `SIGINT`/`SIGTERM`. - ---- - -## 8. Security - -- **Outbound only.** No inbound port is opened on the robot; every call is an - HTTPS POST carrying `Authorization: Bearer `, keyed by `sn`. -- **Read-only toward the robot.** It subscribes to four topics and reads two - local HTTP status pages. It never publishes, never calls a service, and never - commands motion. `/motion/motion_state`'s transition list is *reported*, never - *requested*; `CONTROL_ENABLE=0`. -- **Token handling.** Tokens live only in `/opt/sanad_api_eng/.env` on the - robot, mode `600` root-owned, and are `.gitignore`d here. Only `.env.example` - (placeholders) is in this repo. -- **TLS verified** (`VERIFY_TLS=1`). -- **Rate-limited.** The 500 Hz and 100 Hz streams are gated to 20 Hz / 5 Hz and - subscribed `raw`, so a dropped message is never deserialized. Measured cost: - **~24% of one core out of 12** (~2% of the machine), on the application - processor — the realtime motion controller is a separate board. See - `agent/AGENT_README.md` for the per-subscription breakdown and why it is not - optimised further. - ---- - -## 9. Relationship to the other fleet agents - -`sanad_api_eng.py` is standalone. It shares no files with `agi_fleet/` -(the X2 agent) or with the Unitree `fleet/` agents (G1 / R1 / Go2), and contains -zero AgiBot or Unitree code. The three can be changed independently. - -It does deliberately keep the X2 agent's **telemetry schema, endpoint paths, -env-var discipline and map/logs/alerts/remote loops** unchanged, so the same -fleet server ingests all of them identically — the only structural difference is -the multi-server endpoint list described in §5.