AGIBOT X2 Dashboard

A web control room for the AGIBOT X2 humanoid. Open http://<robot-ip>:8770 from a laptop, phone or tablet on the same network and you get live telemetry, motion control, camera and LiDAR feeds, a posed 3D digital twin, and speech/expression control — with no app to install and no ROS on the client.

The browser never speaks ROS 2. A small agent on the robot bridges ROS 2 DDS to plain TCP, and the dashboard serves an ordinary web page over HTTP.

---

Contents

• Architecture
• Quick start
• The interface — 13 tabs
• Safety model
• Backend modules
• The on-robot agent
• HTTP API
• Configuration
• Extensions
• Deployment
• Repository layout
• Design notes

---

Architecture

Two processes, one socket between them:

   Browser  ──HTTP + WebSocket──▶  Dashboard server  ──JSON lines over TCP──▶  Agent  ──▶  ROS 2 DDS
  (any device)      :8770          (FastAPI, Python)          :8781         (rclpy, on robot)

The dashboard server never imports rclpy. That single decision drives most of the design:

• It runs anywhere — including Windows — because it needs no ROS installation.
• The web UI stays up when the robot is off, so it can show an offline gate that says "power the robot on" instead of dying with a connection error.
• Reconnection is automatic and continuous. Switch the robot off and the UI shows the gate; switch it on and the dashboard reattaches by itself.

Every bridge implements one interface (bridge_base.py), so the server and the entire frontend are identical whether a real robot is attached or not.

Bridge mode	Behaviour
auto	Talk to the robot agent if a host is configured, otherwise simulate
agent	Require the real robot — never silently fall back to simulation
mock	Always simulate, for UI work away from the robot

The mock is deliberately more than a random-number generator: battery drains at a rate that depends on what the robot is doing, odometry integrates the velocity you command, joints ease toward their targets, and mode transitions enforce the same preconditions the real robot does. UI logic exercised against the mock behaves the same way once a real bridge is attached.

---

Quick start

On the robot (needs ROS 2 Humble and the AimDK workspace):

cd ~/x2_dashboard_agent
./run_agent.sh                 # sources ROS 2 + AimDK, starts the agent on :8781

The dashboard (on the robot, or on any machine that can reach it):

cd x2_dashboard
python3 -m backend             # binds 0.0.0.0:8770

It prints every address it is reachable on, so you can open it from a phone without knowing the host's IP in advance.

To run both together with a crash-restart watchdog, use run_dashboard.sh — it is idempotent and safe to call every minute (see Deployment).

DDS profile required. Camera and LiDAR topics will connect but deliver nothing without the vendor Fast DDS profile. run_agent.sh exports FASTRTPS_DEFAULT_PROFILES_FILE for this reason — see Design notes.

---

The interface — 13 tabs

Overview

The at-a-glance page: one hero figure, supporting stat tiles, and trend sparklines. Battery, mode, uptime, attitude and compass.

Control

Mode switching, input-source arbitration, driving, and preset motions.

• 18 motion modes in 6 groups — safety/basic, joint control, standing, locomotion, posture, external. Each carries a description and a danger flag.
• Driving with W/A/S/D or arrow keys; Space is an emergency stop that zeroes velocity immediately.
• 34 preset motions in 5 groups: greeting (wave, handshake, salute, bow), expressive (heart, thumbs up, hug, cheer, sad), gesture (clap, fist, cross arms), performance (two bass dances, photo poses) and head (point, shake).
• Velocity is only accepted in driveable modes; joint commands only in JOINT_DEFAULT / JOINT_FREEZE.

Motion

Joint-level control and end effectors. Four joint groups — head, waist, arms, legs — with per-joint sliders, live position/effort bars, and 4 hand presets.

Sensors

IMU (chest and torso), the 8-zone head touch sensor, LiDAR status, and a topic-liveness table so you can see which topics are actually publishing.

Vision

Nine selectable feeds — six live cameras plus three perception debug views — each switched on by hand:

Feed	Notes
Head front (imx900c)	2688×1944 JPEG, ~10 Hz
Head rear	2064×1552 JPEG, ~10 Hz, mounted upright
Orbbec Gemini 335 colour	Module is mounted upside down — rotated 180° by default
Orbbec depth	16-bit millimetre map, ~14 Hz, tops out ~4.2 m
Stereo pair (left / right)	Both eyes, JPEG ~10 Hz
Perception debug ×3	Input image, segmentation mask, line colour map

Nothing is subscribed on the robot until you switch a feed on, and switching it off destroys the subscription on the robot rather than merely hiding the <img>. This is not a nicety: a frame is 170–430 KB and the RGB feeds together publish at ~60 Hz, which pushed roughly 15 MB/s through DDS for pictures nobody was looking at — over the same Wi-Fi the robot uses to walk.

Frames are polled as single images rather than streamed, so a quiet topic degrades into "nothing is publishing" instead of a hung connection.

LiDAR

Live 3D point cloud from the chest sensor. A scan is ~25 500 points (816 KB) at 2 Hz, decimated on the robot to 4 000 points before crossing the network. Off until switched on, like the cameras.

Model — the digital twin

The real X2 URDF, posed and shaded by live telemetry. The robot reports all 31 revolute joint positions and efforts at 100 Hz; this tab draws them across a rigid-body tree of 41 links. An arm folded in the picture but straight in the room means the encoder, the model, or your idea of which robot you are connected to is wrong.

Shaded by effort, not temperature — this robot publishes no joint temperature at all. aimdk_msgs/JointState carries name, position, velocity, effort and error code and nothing else, and across all 53 aimdk_msgs types the only temperature fields belong to the PMU and the battery. Effort is honest and nearly as useful: motor heating is I²R and current tracks torque, so the joint pulling hardest is the joint getting hottest.

Load is shown as a percentage of each joint's rated torque, taken from the URDF's <limit effort="..."> and baked into model.json. That matters — rated torque here runs from 0.6 Nm to 120 Nm, so an absolute Nm ramp would paint every wrist permanently cold and every leg permanently hot regardless of what the robot was doing.

Geometry is baked offline: 112 MB of vendor STL down to 1.4 MB. Nothing parses a URDF at runtime.

Interaction

Speech (text-to-speech with priority and interrupt), volume and mute, screen expressions/emoji, and the LED strip. Also hosts the voice-session switch that starts and stops the conversation loop.

Power

Battery detail — percentage, voltage, current, temperature, cycle count, capacity in mAh, output power — plus 8 power rails (48 V bus, 48 V output, 12 V output, head power, Orin NX, RK3588, fan rail, 48 V PMOS), fan RPM and thermals, with history charts.

Navigation

Odometry, heading and travelled path. No mapping controls are offered on purpose: this firmware does not expose the documented SLAM interface (/integrated_command, /relocalization_pose, GetStoredMapByName are all absent from the live graph), and a button that silently does nothing is worse than no button.

Extensions

Renders whatever plugins the backend found. Nothing in this tab knows about any specific plugin — see Extensions.

Console

Event log, raw ROS access (publish to any topic, call any service) and the discovered ROS graph.

Settings

Robot discovery, network, transport and safety limits. This is where the "works on any IP" promise lives: the dashboard never stores a fixed robot address — it discovers one on whatever network you are currently attached to, and reports every address it is itself reachable on.

Includes a QR code of the numeric dashboard URL. Typing an IP into a phone is exactly the friction that makes people want a hostname; a QR sidesteps the naming question entirely, since it needs no mDNS, no multicast and no DNS — it works even on Wi-Fi that isolates clients from each other.

---

Safety model

Guard	Detail
Deadman	Velocity commands expire after locomotion_deadman_s (default 0.5 s). The UI republishes at 20 Hz while a key is held; release it and the robot stops.
Zero-torque confirm	Entering PASSIVE_DEFAULT requires an explicit confirmation dialog (require_confirm_zero_torque). These modes collapse a free-standing robot.
Danger flags	Passive, zero-torque, soft e-stop, run and both stair gaits are flagged in the spec, and the UI renders them as destructive actions.
Mode gating	Velocity is rejected outside driveable modes; joint commands are rejected outside joint-control modes.
Velocity clamps	Forward −0.6…0.8 m/s, lateral ±0.7 m/s, angular ±0.8 rad/s — configurable downward in Settings.
Space bar	Global emergency stop, zeroes velocity immediately.

---

Backend modules

Module	Lines	Purpose
server.py	1062	FastAPI app — all HTTP routes and the WebSocket
x2_spec.py	773	The interface specification (see below)
bridge_agent.py	610	TCP bridge to the on-robot agent, with auto-reconnect
nic.py	592	Picks the LAN address this machine is actually reachable on
bridge_mock.py	520	Physics-lite simulated X2
announce.py	348	mDNS — advertises the dashboard as <name>.local
netinfo.py	346	Address enumeration, subnet derivation, robot discovery sweep
voice_session.py	306	On/off switch for the conversation loop
bridge_base.py	241	The contract every bridge implements
plugin_api.py	229	The extension API
settings.py	227	Runtime config, persisted to config.json
registry.py	203	Plugin discovery, loading and hot reload
recovery.py	118	Find the robot and start its agent over SSH
hub.py	107	Fan-out from one bridge to N browsers
__main__.py	98	Entry point — python -m backend

x2_spec.py — the ground truth

Every constant in this file was read back from the robot itself by introspecting the installed aimdk_msgs package and the live ROS 2 graph — not copied from the published documentation. Where the two disagree, the robot wins. Differences found:

• McAction has 18 modes, not 5. PASSIVE_DEFAULT (1) and ZERO_TORQUE_DEFAULT (4) are distinct; the docs conflated them.
• McControlArea is a bitmask (LEFT_HAND=1, RIGHT_HAND=2, HEAD=4, WAIST=8) — which is why the docs' "area 3" (both arms) and "area 11" (whole body) work out: 3 = 1|2, 11 = 1|2|8.
• Several documented preset motion IDs (1007, 1010, 1011, 3017, 3024, 3025, 3031) do not exist in the firmware enum.
• PmuState spells its fan field fan_pecentage (sic), and battery_remaining_capacity is mAh, not percent — the percentage lives in battery_remaining_capacity_percentage.

hub.py — why each browser gets its own queue

One bridge produces state; N browsers consume it. Each client has a bounded queue, so a slow tab (a backgrounded phone, say) can never stall the telemetry loop for everyone else — it just drops frames.

nic.py — why address selection is not a connectivity test

The dashboard publishes http://<ip>:8770, so picking the wrong address hands someone a link that cannot work. This is genuinely hard: a disconnected Ethernet adapter still holding an address will answer a local HTTP fetch successfully, because Windows routes traffic to any of its own addresses through loopback, skipping the adapter entirely — and it answers faster than the real one. No connect, bind or fetch test can distinguish a reachable address from a dead one; the decision has to come from adapter metadata. The rule used here was chosen by running candidates against 23 synthetic adapter tables (VPNs of three shapes, Hyper-V bridges, mobile hotspot, docked Ethernet, duplicate-IP detection, campus public addressing, renamed adapters).

recovery.py — why recovery lives on the dashboard host

The X2 cannot reliably bring its own agent up after a power cycle:

• systemd --user units only run while the user has a login session, and the agi account cannot enable lingering (loginctl enable-linger is denied, and sudo forbids running as root).
• The cron fallback does not fire either — the robot's clock jumps backwards by several hours shortly after boot (RTC vs NTP), and cron stalls on a backward jump.

So the dashboard finds the robot wherever DHCP put it and starts the agent over SSH.

---

The on-robot agent

x2_dashboard_agent/x2_agent.py — ~1 480 lines, runs on PC2 (Jetson Orin). Its only dependency beyond rclpy is the standard library.

Protocol: newline-delimited JSON.

-> {"type":"hello","data":{...}}                       (agent, on connect)
-> {"type":"state","data":{...}}                       (agent, ~10 Hz)
<- {"type":"cmd","id":7,"name":"set_mode","args":{}}   (client)
-> {"type":"result","id":7,"ok":true,"message":"..."}  (agent)

Commands: set_mode, get_mode, set_velocity, stop, preset, register_source, set_joints, set_hand, speak, set_volume, set_mute, emoji, led, camera_frame, graph, publish_raw, ping, stream_set, stream_list, lidar_points.

Flags: --host (default 0.0.0.0), --port (default 8781), --hz (state broadcast rate, default 10).

Cameras and the LiDAR are on-demand: no subscription exists until stream_set turns one on.

---

HTTP API

All under /api. The WebSocket at /ws carries the live telemetry stream.

Area	Endpoints
State	GET /bootstrap · /state · /series · /series/keys · /events · /topics · /health
Settings	GET/POST /settings
Network	GET /network · POST /network/probe · /network/scan · /bridge/restart
Motion	GET/POST /mode · POST /velocity · /stop · /preset · /joints · /hand
Interaction	POST /speak · /volume · /mute · /emoji · /led · GET/POST /voice/session
Arbitration	POST /input-source
Raw ROS	POST /raw/publish · /raw/service
Streams	GET /streams · POST /streams/{key} · GET /lidar/points · /camera/{key}/frame
Recovery	POST /robot/find · /robot/wake · GET /robot/status · GET /qr
Plugins	GET /plugins · POST /plugins/reload · POST /plugins/{id}/{control}

Interactive API docs are served at /api/docs.

---

Configuration

config.json sits beside the project root. Every field is editable from the Settings tab while the server is running.

Key	Default	Meaning
robot_host	127.0.0.1	Agent address
agent_port	8781	Agent TCP port
robot_label	AGIBOT X2	Display name
dashboard_name	agibot	mDNS name → agibot.local
advertise_name	true	Publish that name over mDNS
auto_discover	true	Sweep the subnet when the saved address stops answering
auto_start_agent	false	SSH in and start the agent if it is not running
robot_ssh_user / _password / _port	agi / (empty) / 22	SSH credentials for recovery
agent_start_command	ensure_agent.sh	What to run over SSH
ros_domain_id	0	Shown for reference; used by the agent
rmw_implementation	rmw_fastrtps_cpp	Shown for reference
bridge_mode	agent	auto · agent · mock
host / port	0.0.0.0 / 8770	HTTP bind
telemetry_hz	10	State broadcast rate
history_seconds	120	Chart history window
require_confirm_zero_torque	true	Confirm before passive/zero-torque
locomotion_deadman_s	0.5	Velocity command expiry
max_forward_velocity	0.8	m/s
max_lateral_velocity	0.7	m/s
max_angular_velocity	0.8	rad/s
theme / accent	dark / blue	UI

host is the only key the browser may not change.

Note: robot_ssh_password ships empty. Fill it in through the Settings tab at runtime rather than committing a value.

---

Extensions

Drop a .py file into backend/plugins/ that builds a Plugin, and the dashboard grows a new panel for it — no frontend work, no server edits. Each control you declare is rendered by the browser from the manifest, and clicking it calls your handler.

from backend.plugin_api import Plugin

plugin = Plugin(id="hello", name="Hello", icon="👋")

@plugin.action("wave", label="Wave hello")
async def wave(ctx):
    await ctx.bridge.play_preset(motion=1002, area=2)
    return "Waved"

Press Reload in the Extensions tab to pick up edits without restarting the server. A plugin that fails to import does not take the dashboard down — the error is recorded and shown in the UI next to the plugin that caused it.

Two worked examples ship in backend/plugins/:

• example_greeter.py — sequencing several subsystems: one button drives mode, motion, speech, screen and lights together.
• example_battery_guard.py — a background monitor: watches the PMU battery level, warns once per threshold crossing, and can drop the robot into a safe mode before it browns out mid-stride. Demonstrates on_tick, ctx.push readouts and charted series.

_template.py is a blank starting point.

---

Deployment

Three systemd user units, in systemd_user/:

Unit	Role
x2-dashboard.service	Type=oneshot launcher — runs run_dashboard.sh, then returns
x2-dashboard.timer	Watchdog — fires 30 s after boot, then every wall-clock minute
x2-dashboard-agent.service	The ROS bridge alone

cp systemd_user/*.service systemd_user/*.timer ~/.config/systemd/user/
systemctl --user daemon-reload
systemctl --user enable --now x2-dashboard.timer

Two non-obvious details are load-bearing:

• Type=oneshot without RemainAfterExit. The unit must return to inactive after each run, or the timer never re-triggers.
• OnCalendar=*:*:00, not OnUnitActiveSec. The launcher setsids its children into their own sessions so they outlive the one-shot; an activity-relative timer would compute NextElapse=infinity. KillMode=none stops systemd reaping those children when the unit deactivates.

run_dashboard.sh is idempotent by design — it starts only what is not already running, so it is safe from the timer, from @reboot, and by hand. It guards on /proc rather than trusting pgrep -f alone, because pgrep -f also matches any shell whose command line merely mentions the pattern; that bug launched a second backend on top of the first during the ~10 s before the port binds. It also trims its own logs at 2 MB.

---

Repository layout

X2_dashboard/
├── x2_dashboard/                 the dashboard server + web UI
│   ├── backend/
│   │   ├── server.py             FastAPI app, all routes, WebSocket
│   │   ├── x2_spec.py            interface spec, read off the real robot
│   │   ├── bridge_base.py        the bridge contract
│   │   ├── bridge_agent.py       TCP bridge to the on-robot agent
│   │   ├── bridge_mock.py        simulated X2
│   │   ├── hub.py                fan-out to N browsers
│   │   ├── registry.py           plugin discovery + hot reload
│   │   ├── plugin_api.py         the extension API
│   │   ├── settings.py           config load/save
│   │   ├── announce.py           mDNS
│   │   ├── netinfo.py            discovery + addressing
│   │   ├── nic.py                reachable-address selection
│   │   ├── recovery.py           find + wake the robot over SSH
│   │   ├── voice_session.py      conversation-loop control
│   │   └── plugins/              drop-in extensions
│   ├── web/
│   │   ├── index.html
│   │   ├── css/app.css           design tokens, layout, components
│   │   ├── js/
│   │   │   ├── core.js           store, transport, notifications
│   │   │   ├── main.js           shell, rail, routing, shortcuts
│   │   │   ├── ui.js             DOM builders
│   │   │   ├── charts.js         SVG charts
│   │   │   ├── model3d.js        WebGL URDF renderer
│   │   │   └── tabs/             the 13 pages
│   │   └── model/                quantised URDF geometry (1.4 MB)
│   └── config.json
├── x2_dashboard_agent/           runs ON the robot
│   ├── x2_agent.py               ROS 2 ⇄ TCP bridge
│   ├── run_agent.sh              sources ROS 2 + AimDK + DDS profile
│   ├── run_dashboard.sh          idempotent launcher for both
│   ├── ensure_agent.sh           called over SSH by recovery
│   └── x2-dashboard-agent.service
└── systemd_user/                 unit files

---

Design notes

No framework, no CDN, no build step. The frontend is plain ES modules and hand-written DOM. The robot has no internet access, and the dashboard is often loaded over the robot's own Wi-Fi — the same link it uses to walk. A framework would cost more than it saves at this size.

That constraint also explains two hand-rolled renderers:

• charts.js — SVG charts. 2 px lines with round joins, area washes at 10 % opacity, ≥8 px end markers with a 2 px surface ring, hairline gridlines, endpoint-only direct labels, a legend from two series up, and a crosshair tooltip on hover. Text always wears text tokens, never the series colour.
• model3d.js — one shader, flat shading, a rigid-body tree of 41 links. Geometry arrives as uint16 positions quantised inside each mesh's bounding box and the shader expands them back to metres, so the 4× saving over float32 costs two extra instructions per vertex. Normals are computed per-face in the fragment shader from screen-space derivatives, so the vertex buffer carries positions and nothing else.

Nothing assumes a fixed IP. Every URL in the frontend is derived from window.location, so opening the page from a laptop, a phone, a tablet or a hostname all work with no configuration. The server binds 0.0.0.0 and enumerates its own addresses.

The Fast DDS profile is not optional. A camera frame is 170–430 KB and a LiDAR scan is 816 KB. Fast DDS defaults to a 512 KB shared-memory segment and small socket buffers, and it drops samples that large in complete silence — discovery succeeds, the reader matches the writer, ros2 topic info reports a publisher, and not one message is ever delivered. The vendor profile raises the segment to 32 MB and UDP buffers to 10 MB. An earlier revision of x2_spec.py concluded this unit had one camera and no chest LiDAR; it has six camera feeds and a LiDAR, and the agent simply could not receive them.

mDNS is implemented on the stdlib, not zeroconf. The advertised name is additive and reversible — the host keeps its own identity, and the extra name exists only while the dashboard is running, so it needs neither administrator rights nor a reboot.

---

What is not in this copy

Two things were deliberately excluded when this snapshot was taken from the robot:

• The Sanad API service (sanad_api_x2) — it carries live credentials in a .env, so it is not published here.
• Teach mode / Manual Recorder — the record-an-arm-motion-by-hand feature and its /api/recorder/* routes were removed from this copy. The dashboard runs without it; the feature still exists on the robot.

No API keys or credentials are present in this repository. The Gemini and LinkSoul credentials the voice session depends on are read at runtime from ~/.sanad_agibot_env on the robot and are never embedded in this code.