diff --git a/FEATURES.md b/FEATURES.md new file mode 100644 index 0000000..f13300a --- /dev/null +++ b/FEATURES.md @@ -0,0 +1,136 @@ +# SanadV3 — Feature Catalog + +Sanad is a bilingual (Arabic/English) humanoid receptionist/assistant on a +Unitree G1 (Jetson Orin NX, ROS 2 Foxy, Livox MID-360). This catalogs +**what's built today** (Part A) and **what's on the roadmap** (Part B). + +--- + +# Part A — Current features (built & running) + +Verified from the live subsystem registry (19 subsystems), dashboard tabs +(12), and API routers (22). + +## 1. Voice & Conversation +- **Gemini live voice** — real-time bilingual AR/EN spoken conversation (native-audio model) +- **Offline brain** — local pipeline via `ollama` (`SANAD_VOICE_BRAIN=local`), no cloud +- **Wake phrases** — configurable wake-word manager +- **Typed replay** — type text, robot speaks it (with speaker-monitor capture) +- **Local TTS** — on-device text-to-speech engine +- **Prompt management** — edit the system prompt from the dashboard +- **Lip-sync** — mask mouth driven by TTS `MOUTH` markers +- **Barge-in** — interrupt speech (volume-scaled threshold) + +## 2. Motion & Arm +- **35 arm actions** — 16 SDK built-ins + 19 custom JSONL motions +- **Macro record / playback** — capture and replay motion sequences +- **Teaching mode** — kinesthetic teach-and-repeat +- **Skills** — composed higher-level behaviors (skills.json) +- **Movement dispatch** — voice → motion (53 fixed + 10 parametric phrases, cooldown-gated) +- **Arm motion-block** — auto-inhibits arm moves while locomotion is active (safety interlock) + +## 3. Locomotion +- **LocoClient + MotionSwitcher** — walk / pose control via Unitree SDK (eth0) +- **E-STOP** — dashboard kill button +- **Single Ctrl+C teardown** — one signal cleanly stops every subsystem (~2s) + +## 4. LED Face Mask +- **Animated expressions** — neutral, smile, blink, look L/R, talk1–3, surprised, sad +- **Gestural-speaking events** — face reacts while speaking +- **Lip-sync** — mouth animates to speech + +## 5. Vision & Recognition +- **Face recognition** — identify people via camera +- **Face gallery** — enroll/manage known faces +- **Zone gallery / zones** — visual zone recognition +- **Camera feed** — attached to the live voice subprocess (vision-in-the-loop) + +## 6. Navigation (web_nav3 integration) +- **Live Map tab** — full embedded web_nav3 dashboard (set-pose, goals, bringup) +- **Navigation tab** — native canvas viewer (saved/live map, places, missions) +- **map_relay** — re-publishes the latched `/map` @1Hz so the map renders even when stationary +- **Saved maps** — load & view a pre-built `.db` (localize mode) +- **Places** — save named poses, one-click "Go" +- **Missions** — multi-waypoint routes (defined in web_nav3) +- **Cancel goal** — stop an active goal without tearing down bringup +- **SLAM** — RTABMap LiDAR-ICP, drift-corrected mapping/localization + +## 7. Audio +- **Device manager** — sink/source selection, live refresh +- **Audio profiles** — builtin / anker / hollyland_builtin (auto-switch on plug/unplug) + +## 8. Operations, System & Diagnostics +- **System control** — start/stop subsystems, status +- **Temperature monitor** — motor temps (live websocket stream) +- **Controller** — gamepad/teleop input +- **Web terminal** — shell in the browser (websocket) +- **Logs** — live log stream +- **Recordings & replay** — record/playback sessions +- **Scripts** — run saved scripts + +## Dashboard infrastructure +- 12 tabs, **fault-isolated routers** (one broken module never breaks the dashboard) +- WebSocket streams: log_stream, motor_temps, terminal +- No-store HTML (no stale-cache 404s after deploy) +- Lazy subsystem imports (missing dep → that subsystem unavailable, rest runs) + +--- + +# Part B — Roadmap (to add) + +Tiers = priority. 🏗️ = load-bearing · ⚠️ = Foxy constraint. + +## Tier 1 — Autonomous behaviors (the product) +1. **Voice-driven navigation** — "Sanad, go to the lobby" → nav goal +2. **Greeter mission** — recognized face → navigate → greet → express +3. **Named-person greeting** — identity → personalized line +4. **Patrol / guided tours** — ordered places, speech at each stop +5. **Return-to-base / dock-on-idle** — auto-home on idle/low battery + +## Tier 2 — Navigation & map (harden + edit) +6. 🏗️ **Map republish relay** — ✅ DONE (map_relay) +7. **Click-to-goal on Nav tab canvas** +8. **Live nav telemetry** — distance/ETA/waypoint, "arrived" toast +9. **Battery + nav-state status bar** +10. **Geofence zones on the map** +11. **Cancel-goal button** — ✅ DONE +### Map editing & annotation (all build on #6) +12. **Erase tool** — paint cells free; wipe ghost obstacles + the SLAM "spokes" +13. **Obstacle paint ("black points" / virtual walls)** — ⚠️ Foxy-safe KeepoutFilter substitute +14. **Shape tools + brush size** — line/rectangle/polygon +15. **Non-destructive overlay + undo/redo** +16. **Persist & auto-reload edits per map** +17. **Crop / trim map bounds** + +## Tier 3 — Voice & interaction +18. **Barge-in from dashboard** +19. **Quick-phrase soundboard** +20. **Conversation memory / visitor log** +21. **Per-speaker AR/EN auto-detect** +22. **Scheduled announcements** +23. **Bake edited map → PGM/YAML** (static map_server deploy) + +## Tier 4 — Face & presence +24. **Gaze / head-track recognized face** +25. **Emotion-from-context** (sentiment → expression) +26. **Idle breathing / look-around** +27. **Lip-sync to TTS amplitude** (enhance existing markers) + +## Tier 5 — Operator, fleet & reliability +28. 🏗️ **Global E-STOP button** — ✅ exists; surface consistently +29. **Health watchdog** — auto-restart dead subsystem + alert +30. **Per-subsystem enable/disable toggles** +31. **Behavior recorder → replay** (nav+voice timelines) +32. **Mission editor UI** (visual sequence builder) +33. **Remote access / tunnel** +34. **Reverse-proxy web_nav3 through :8001** — one origin, no iframe cross-port issues + +## Tier 6 — Future / blocked +35. **Speed / caution zones** — needs Galactic SpeedFilter or custom layer +36. **Multi-robot fleet** (SanadV3 ↔ BotBrain) — needs LocoClient arbitration + coordinator + +--- + +### Recommended next build order +**#1 voice→nav** → **#2 greeter mission** (the product), then **#12/#13 map editing** +(clean the spokes + virtual walls). #6 republish relay and #11 cancel are already done. diff --git a/G1_Controller/loco_controller.py b/G1_Controller/loco_controller.py index f2746db..22ba59d 100644 --- a/G1_Controller/loco_controller.py +++ b/G1_Controller/loco_controller.py @@ -255,12 +255,19 @@ class LocoController: period = max(0.02, min(0.1, self._wd_timeout / 2.0)) while not self._wd_stop.is_set(): fire = False + park = False # Read-and-decide under the lock (atomic check-then-act); the actual # StopMove runs after release so the critical section stays tiny. with self._lock: if self._teleop_active and (time.monotonic() - self._last_move_ts) > self._wd_timeout: self._teleop_active = False fire = True + # Self-park once there's nothing left to guard. The Gemini + # dispatch path uses step() directly and never calls + # disarm_movement(), so without this the watchdog would spin for + # the rest of the process lifetime after the first voice step. + if not self._armed and not self._teleop_active and not self._discrete_busy: + park = True if fire: log.warning("watchdog: teleop setpoint stale (>%.2fs) — StopMove", self._wd_timeout) @@ -268,6 +275,12 @@ class LocoController: self._raw_stop() except Exception: log.exception("watchdog StopMove failed") + if park: + # Nothing left to guard — stop the thread (a later move/step + # re-arms it via _start_watchdog()). Done AFTER any stale-stop + # above so we never skip a pending StopMove. + self._wd_stop.set() + break self._wd_stop.wait(period) def _raw_stop(self) -> bool: @@ -560,8 +573,21 @@ class LocoController: # ── shutdown helper ────────────────────────────────────────────────────── def shutdown(self): - """Best-effort StopMove + disarm for process shutdown.""" + """Best-effort StopMove + disarm for process shutdown. + + Uses _raw_stop() (NOT estop()) so teardown never builds a brand-new + LocoClient: estop() → _ensure_client() would lazily construct a client + and run bot.Init() (a DDS RPC) during interpreter teardown when we were + armed-but-never-built (Enable movement clicked, never moved, then + Ctrl+C). _raw_stop() no-ops when no client was ever created. Bump the + stop generation so any in-flight motion bails immediately.""" + with self._lock: + self._stop_gen += 1 + self._teleop_active = False + self._cur_v = (0.0, 0.0, 0.0) try: - self.estop() + self._raw_stop() # no-op when _bot is None — never re-inits + except Exception: + log.exception("StopMove on shutdown failed") finally: self.disarm_movement() diff --git a/config.py b/config.py index a25c673..47375df 100644 --- a/config.py +++ b/config.py @@ -271,7 +271,9 @@ def _resolve_dashboard_host() -> str: DASHBOARD_HOST = _resolve_dashboard_host() -DASHBOARD_PORT = 8000 +# Canonical SanadV3 port (matches shell_scripts/start_all.sh + docs). The +# legacy Sanad ran on :8000; SanadV3 is :8001 to never collide with it. +DASHBOARD_PORT = 8001 # -- Local TTS -- LOCAL_TTS_MODEL = "MBZUAI/speecht5_tts_clartts_ar" @@ -365,6 +367,11 @@ LIVE_TUNE: dict[str, str] = { CAMERA_SERVICE_PORT = 8091 DIRECT_CAMERA_URL = f"http://127.0.0.1:{CAMERA_SERVICE_PORT}" +# -- Navigation (web_nav3 / rosbridge) -- +WEB_NAV3_URL = os.environ.get("WEB_NAV3_URL", "http://127.0.0.1:8765") +ROSBRIDGE_URL = os.environ.get("ROSBRIDGE_URL", "ws://127.0.0.1:9090") +NAV_ROBOT_NAME = os.environ.get("NAV_ROBOT_NAME", "sanad") + # -- DDS / hardware -- # Jetson G1 default is eth0 (the robot's internal network). # Override with SANAD_DDS_INTERFACE=lo for desktop/sim development. diff --git a/config/mask_config.json b/config/mask_config.json new file mode 100644 index 0000000..fb3c488 --- /dev/null +++ b/config/mask_config.json @@ -0,0 +1,27 @@ +{ + "_comment": "Shining LED face mask (BLE). Driven by the FaceController subsystem (face/mask_face.py) which imports the standalone Mask project. Needs an env with bleak + Pillow (g1_env). Free the mask from the phone app before connecting.", + "mask_dir": "", + "_mask_dir": "Path to the Mask project (flat shiningmask lib). Empty -> auto: /Mask. Env override: SANAD_MASK_DIR.", + "name_prefix": "MASK", + "_name_prefix": "BLE scan prefix; the mask advertises e.g. 'MASK-02A711'. Env: SANAD_MASK_NAME_PREFIX.", + "address": "", + "_address": "Specific BLE MAC to connect to. Empty -> scan by name_prefix. Env: SANAD_MASK_ADDRESS.", + "adapter": "", + "_adapter": "BlueZ adapter (e.g. 'hci0'). Empty -> default. Env: SANAD_MASK_ADAPTER.", + "brightness": 95, + "_brightness": "0-128. Keep <=100 to avoid LED flicker (battery-limited).", + "fps": 8.0, + "_fps": "FaceAnimator (fallback driver) frame rate (PLAY commands/sec).", + "lifelike": true, + "_lifelike": "Use the LifelikeFace driver (face/face_motion.py): eye saccades, varied blinks, listening/thinking/speaking states, reactions, smooth lip-sync. false -> basic FaceAnimator.", + "autostart": true, + "_autostart": "Auto-connect + Start face on boot (best-effort, background — never blocks startup). After the one-time frame upload, later boots just connect + animate. false -> connect/start manually from the dashboard.", + "connect_timeout": 15.0, + "connect_attempts": 5, + "eye_color": [0, 230, 255], + "_eye_color": "Face eye/iris RGB (baked into the uploaded frames). Default cyan. Set via the dashboard 'Apply colors' (persisted here).", + "mouth_color": [255, 50, 50], + "_mouth_color": "Face mouth RGB. Default red.", + "sclera_color": [255, 255, 255], + "_sclera_color": "White-of-the-eye RGB. Default white." +} diff --git a/core/persona.py b/core/persona.py new file mode 100644 index 0000000..ef0810c --- /dev/null +++ b/core/persona.py @@ -0,0 +1,71 @@ +"""Active-persona selection — which script file Gemini loads as its system +prompt. + +The operator can keep several persona variants in scripts/ (e.g. +``sanad_script.txt``, ``sanad_script_v1.txt``, ``sanad_script_v2.txt``) and pick +which one is live. The selection is a single basename stored in +``data/active_persona.txt``; the DEFAULT (and reset target) is always the +configured persona (``sanad_script.txt``). The Gemini child resolves this at +session start, so a new selection takes effect on the next voice (re)connect. + +A missing/blank/stale pointer transparently falls back to the default, so this +can never break the voice — worst case it loads ``sanad_script.txt``. +""" +from __future__ import annotations + +from pathlib import Path + +from Project.Sanad.config import DATA_DIR, SCRIPTS_DIR + +ACTIVE_PERSONA_FILE = DATA_DIR / "active_persona.txt" + + +def default_persona_name() -> str: + """The configured default persona filename (core.script_files.persona).""" + try: + from Project.Sanad.core.config_loader import section as _section + name = (_section("core", "script_files") or {}).get("persona") + return (name or "sanad_script.txt").strip() or "sanad_script.txt" + except Exception: + return "sanad_script.txt" + + +def active_persona_name() -> str: + """Selected persona basename — the chosen variant if set AND still exists, + otherwise the default. Never raises.""" + default = default_persona_name() + try: + sel = ACTIVE_PERSONA_FILE.read_text(encoding="utf-8").strip() + except Exception: + sel = "" + if sel: + cand = SCRIPTS_DIR / Path(sel).name # basename only — no traversal + if cand.is_file(): + return cand.name + return default + + +def active_persona_path() -> Path: + """Full path to the persona script Gemini should load right now.""" + return SCRIPTS_DIR / active_persona_name() + + +def set_active_persona(name: str | None) -> str: + """Persist the selected persona basename. Passing None/"" or the default + name clears the pointer (revert to default). Returns the effective active + name. Raises FileNotFoundError if a non-default name doesn't exist.""" + nm = (Path(str(name)).name if name else "").strip() + default = default_persona_name() + if not nm or nm == default: + try: + ACTIVE_PERSONA_FILE.unlink() + except FileNotFoundError: + pass + except Exception: + pass + return default + if not (SCRIPTS_DIR / nm).is_file(): + raise FileNotFoundError(nm) + DATA_DIR.mkdir(parents=True, exist_ok=True) + ACTIVE_PERSONA_FILE.write_text(nm, encoding="utf-8") + return nm diff --git a/dashboard/app.py b/dashboard/app.py index 8261aab..03ab771 100644 --- a/dashboard/app.py +++ b/dashboard/app.py @@ -54,6 +54,9 @@ _REST_ROUTES: list[tuple[str, str, str]] = [ ("zones", "/api/zones", "zones"), ("temp_monitor", "/api/temp", "temperature"), ("controller", "/api/controller", "controller"), + ("mask", "/api/mask", "mask"), + ("mask_social", "/api/mask", "mask-social"), + ("navigation", "/api/nav", "navigation"), ] _WS_ROUTES: list[str] = ["log_stream", "motor_temps", "terminal"] @@ -113,7 +116,13 @@ async def root(): if index.exists(): from fastapi.responses import HTMLResponse try: - return HTMLResponse(index.read_text(encoding="utf-8")) + # no-store so the browser always re-fetches the dashboard HTML/JS + # after a deploy — otherwise stale cached JS keeps calling old + # endpoints (e.g. /nav/* instead of /api/nav/*) and 404s. + return HTMLResponse( + index.read_text(encoding="utf-8"), + headers={"Cache-Control": "no-store, must-revalidate"}, + ) except OSError as exc: return {"error": f"Could not read index.html: {exc}"} return { diff --git a/dashboard/routes/_arbiter.py b/dashboard/routes/_arbiter.py new file mode 100644 index 0000000..8d263da --- /dev/null +++ b/dashboard/routes/_arbiter.py @@ -0,0 +1,66 @@ +"""In-process arbitration between Nav2 (web_nav3) and the manual LocoController. + +Both stacks can drive the G1's legs via different command paths: + - Nav2 (web_nav3) publishes cmd_vel from a navigation goal/mission. + - LocoController issues LocoClient.Move()/step() from the Controller tab and + Gemini movement dispatch. + +The documented hazard is "two stacks must never both drive the legs at once". +This module is a tiny thread-safe gate that lets ONE commander own the legs at a +time. controller.py sets loco_active for arm/move/step and refuses when nav is +active; navigation.py sets nav_active for goto/missions/run and refuses when loco +is active. The E-STOP / cancel paths clear the relevant flag. + +Pure in-process state (no DDS, no HTTP) — both routers share this single module +instance, so the flags are coherent across the dashboard process. +""" + +from __future__ import annotations + +import threading + +_lock = threading.Lock() +_loco_active = False +_nav_active = False + + +def loco_active() -> bool: + with _lock: + return _loco_active + + +def nav_active() -> bool: + with _lock: + return _nav_active + + +def acquire_loco() -> bool: + """Claim the legs for manual loco. Returns False if Nav2 holds them.""" + global _loco_active + with _lock: + if _nav_active: + return False + _loco_active = True + return True + + +def release_loco() -> None: + global _loco_active + with _lock: + _loco_active = False + + +def acquire_nav() -> bool: + """Claim the legs for Nav2. Returns False if manual loco holds them.""" + global _nav_active + with _lock: + if _loco_active: + return False + _nav_active = True + return True + + +def release_nav() -> None: + global _nav_active + with _lock: + _nav_active = False diff --git a/dashboard/routes/audio_control.py b/dashboard/routes/audio_control.py index 6ed28f1..cd6f02e 100644 --- a/dashboard/routes/audio_control.py +++ b/dashboard/routes/audio_control.py @@ -160,7 +160,13 @@ async def audio_status(): "g1_speaker_muted": g1_muted, "g1_current_volume": _g1_current_volume, "g1_user_volume": _g1_user_volume, - "g1_available": _g1_audio_client is not None or (_g1_init_error == ""), + # Only report available once an AudioClient has actually been + # built — reporting True before any init attempt made the UI + # advertise G1 speaker controls that then 503 on first use. + # `g1_init_error` surfaces *why* it's unavailable (or "" if + # init was never attempted yet). + "g1_available": _g1_audio_client is not None, + "g1_init_error": _g1_init_error, "sink": sink, "source": source, "current": cur, @@ -312,7 +318,9 @@ async def get_g1_volume(): """ def _do(): return { - "available": _g1_audio_client is not None or (_g1_init_error == ""), + # True only after an AudioClient was actually constructed — + # `init_error` (below) explains an unavailable/never-tried state. + "available": _g1_audio_client is not None, "current_volume": _g1_current_volume, "user_volume": _g1_user_volume, "muted": _g1_current_volume == 0, @@ -343,35 +351,48 @@ async def set_g1_volume(payload: G1VolumePayload): if not 0 <= level <= 100: raise HTTPException(400, "level must be 0..100") + # 1) G1 chest speaker (DDS) — best-effort so it works even when an + # external sink (JBL) is the active output. + code = None client = _get_g1_audio_client() - if client is None: - raise HTTPException( - 503, - f"G1 AudioClient unavailable: {_g1_init_error or 'unknown'}", - ) - try: - with _g1_audio_lock: - code = client.SetVolume(level) - _g1_current_volume = level - if level > 0: - # Only update the "preferred unmuted" level when the - # user is setting a non-zero volume. Setting 0 is a - # mute, which shouldn't overwrite their preference. - _g1_user_volume = level - except Exception as exc: - raise HTTPException(500, f"SetVolume failed: {exc}") + if client is not None: + try: + with _g1_audio_lock: + code = client.SetVolume(level) + _g1_current_volume = level + except Exception as exc: + log.warning("G1 SetVolume failed: %s", exc) + if level > 0: + _g1_user_volume = level + + # 2) The ACTIVE profile's PulseAudio sink (JBL / Anker / …). Target the + # RESOLVED sink from the saved selection, NOT @DEFAULT_SINK@ — the PA + # default can be a different sink (e.g. the chest platform-sound) even + # when the JBL is the selected output, so @DEFAULT_SINK@ would move the + # wrong sink and the slider would appear to do nothing on the JBL. + pa_applied = False + try: + sink = (ad.load_state() or {}).get("sink") or "@DEFAULT_SINK@" + _pactl(["set-sink-volume", sink, "%d%%" % level]) + if level > 0: + _pactl(["set-sink-mute", sink, "0"]) + pa_applied = True + except Exception as exc: + log.warning("PA set-sink-volume failed: %s", exc) + + if client is None and not pa_applied: + raise HTTPException(503, "No speaker available (G1 + PulseAudio both failed)") - # Persist the user's preferred level (not the current) so a - # subsequent mute-then-restart restores to the preferred level _save_persisted_g1_volume(_g1_user_volume) - log.info("G1 volume → %d (user_pref=%d, rc=%s)", - level, _g1_user_volume, code) + log.info("volume → %d (g1_rc=%s, pa=%s, user_pref=%d)", + level, code, pa_applied, _g1_user_volume) return { "ok": True, "current_volume": level, "user_volume": _g1_user_volume, "muted": level == 0, "return_code": code, + "pa_applied": pa_applied, "persisted": True, } return await asyncio.to_thread(_do) @@ -471,6 +492,28 @@ async def apply_audio(): audio_mgr.refresh_devices() except Exception: pass + # Hot-swap the live Gemini voice to the selected profile too, so picking + # a device (e.g. the JBL) moves BOTH record playback AND the live voice + # to it — without dropping the session. Best-effort; no-op if not running. + try: + from Project.Sanad.main import live_sub + pid = (ad.load_state() or {}).get("profile_id") + if (pid and live_sub is not None and hasattr(live_sub, "send_profile") + and hasattr(live_sub, "is_running") and live_sub.is_running()): + live_sub.send_profile(pid, reason="dashboard audio Apply") + except Exception: + pass + # Restore the user's SAVED volume to the selected sink (USB/BT speakers + # like the JBL otherwise come back at a low PulseAudio default). Use the + # saved level, NOT a forced 100%, so the slider/sink keep the user's + # choice across selects + restarts. Target the resolved sink. + try: + sink = (ad.load_state() or {}).get("sink") or "@DEFAULT_SINK@" + _pactl(["set-sink-volume", sink, "%d%%" % _g1_user_volume]) + if _g1_user_volume > 0: + _pactl(["set-sink-mute", sink, "0"]) + except Exception: + pass return result return await asyncio.to_thread(_do) diff --git a/dashboard/routes/controller.py b/dashboard/routes/controller.py index f0b998d..1c55e13 100644 --- a/dashboard/routes/controller.py +++ b/dashboard/routes/controller.py @@ -22,6 +22,8 @@ from Project.Sanad.config import BASE_DIR from Project.Sanad.core.logger import get_logger from Project.Sanad.vision import recognition_state +from Project.Sanad.dashboard.routes import _arbiter + log = get_logger("controller_routes") router = APIRouter() @@ -75,6 +77,31 @@ def _require_armed(lc): raise HTTPException(409, "Movement is disarmed. Enable movement first.") +def _claim_loco(): + """Arbitration gate: refuse a leg command while a Nav2 goal owns the legs.""" + if not _arbiter.acquire_loco(): + raise HTTPException( + 409, "Navigation (Nav2) is active. Cancel the nav goal before manual movement." + ) + + +def _cancel_nav(): + """Cancel any in-flight Nav2 goal and clear the nav arbitration flag. + + Used by E-STOP so the global stop halts the legs no matter which stack is + driving them. Calls the nav client in-process (no HTTP self-call); never + raises into the caller. + """ + try: + from Project.Sanad.dashboard.routes.navigation import _CLIENT as _nav_client + if _nav_client is not None: + _nav_client.cancel() + except Exception: + log.exception("estop nav cancel failed") + finally: + _arbiter.release_nav() + + # ── reads ─────────────────────────────────────────────────── @router.get("/status") @@ -100,7 +127,17 @@ async def get_msc(): @router.post("/arm") async def set_arm(on: bool = Query(...)): lc = _require_loco() - res = await asyncio.to_thread(lc.arm_movement if on else lc.disarm_movement) + if on: + # Refuse to arm manual loco while Nav2 owns the legs. + _claim_loco() + try: + res = await asyncio.to_thread(lc.arm_movement) + except Exception: + _arbiter.release_loco() + raise + return res + res = await asyncio.to_thread(lc.disarm_movement) + _arbiter.release_loco() return res @@ -144,6 +181,10 @@ async def estop(): md.emergency_stop() except Exception: log.exception("estop dispatcher latch failed") + # Cancel any in-flight Nav2 goal too: the legs have exactly one commander, + # and an E-STOP must halt the legs whether loco or Nav2 is driving them. + await asyncio.to_thread(_cancel_nav) + _arbiter.release_loco() return {"ok": True, **res} @@ -168,6 +209,7 @@ class MoveBody(BaseModel): async def move(body: MoveBody): lc = _require_loco() _require_armed(lc) + _claim_loco() return await asyncio.to_thread(lc.move, body.vx, body.vy, body.vyaw, body.run) @@ -175,6 +217,7 @@ async def move(body: MoveBody): async def step(dir: str = Query(...)): lc = _require_loco() _require_armed(lc) + _claim_loco() res = await asyncio.to_thread(lc.step, dir) if not res.get("ok"): raise HTTPException(400, res.get("reason", "step failed")) diff --git a/dashboard/routes/live_subprocess.py b/dashboard/routes/live_subprocess.py index 831789c..43e1610 100644 --- a/dashboard/routes/live_subprocess.py +++ b/dashboard/routes/live_subprocess.py @@ -4,10 +4,15 @@ from __future__ import annotations import asyncio -from fastapi import APIRouter, HTTPException +from fastapi import APIRouter, HTTPException, Query + +from Project.Sanad.config import BASE_DIR +from Project.Sanad.vision import recognition_state router = APIRouter() +_STATE_PATH = BASE_DIR / "data" / ".recognition_state.json" + def _sub_or_503(): from Project.Sanad.main import live_sub @@ -19,9 +24,21 @@ def _sub_or_503(): @router.get("/status") async def subprocess_status(): from Project.Sanad.main import live_sub + # record_enabled is a live flag (recognition_state) the panel toggle drives; + # surface it so the UI shows the current state even before a session starts. + rec = bool(recognition_state.read(_STATE_PATH).record_enabled) if live_sub is None: - return {"available": False, "state": "unavailable"} - return live_sub.status() + return {"available": False, "state": "unavailable", "record_enabled": rec} + return {**live_sub.status(), "record_enabled": rec} + + +@router.post("/record") +async def set_record(on: bool = Query(...)): + """Toggle auto-recording of conversation turns to data/recordings/. Takes + effect live (the voice child syncs its recorder) — no session restart.""" + st = await asyncio.to_thread( + recognition_state.mutate, _STATE_PATH, record_enabled=bool(on)) + return {"ok": True, "record_enabled": st.record_enabled} @router.post("/start") diff --git a/dashboard/routes/mask.py b/dashboard/routes/mask.py new file mode 100644 index 0000000..129ab67 --- /dev/null +++ b/dashboard/routes/mask.py @@ -0,0 +1,179 @@ +"""Mask Face tab — Shining LED face mask control (BLE). + +Routes live under /api/mask. Backed by the FaceController subsystem +(face/mask_face.py), which owns a dedicated asyncio loop + BLE connection to the +standalone Mask project's `shiningmask` library. + +Every handler is failure-safe: if the subsystem or its library is unavailable it +returns 503 (GET /status returns a degraded body) rather than crash the +dashboard. FaceController raises RuntimeError for "not connected" / "face not +started"; those map to 409. Blocking BLE calls run in a thread pool so the event +loop stays responsive. +""" + +from __future__ import annotations + +import asyncio +from typing import List, Optional + +from fastapi import APIRouter, HTTPException, Query +from pydantic import BaseModel + +from Project.Sanad.core.logger import get_logger + +log = get_logger("mask_routes") + +router = APIRouter() + + +# ── lazy subsystem accessor ───────────────────────────────── + +def _get_face(): + try: + from Project.Sanad.main import mask_face # type: ignore + return mask_face + except Exception: + return None + + +def _require(): + mf = _get_face() + if mf is None: + raise HTTPException(503, "Mask face subsystem unavailable.") + return mf + + +def _run(fn, *args, **kwargs): + """Call a FaceController method, mapping its errors to HTTP status codes.""" + try: + return fn(*args, **kwargs) + except HTTPException: + raise + except RuntimeError as exc: + raise HTTPException(409, str(exc)) + except Exception as exc: # noqa: BLE001 + log.exception("mask operation failed") + raise HTTPException(500, str(exc)) + + +# ── status ────────────────────────────────────────────────── + +@router.get("/status") +async def status(): + """Never raises — returns a degraded body if the subsystem is missing.""" + mf = _get_face() + if mf is None: + return {"available": False, "connected": False, "lib_available": False, + "last_error": "mask face subsystem not constructed"} + s = await asyncio.to_thread(mf.status) + s["available"] = True + return s + + +# ── connection ────────────────────────────────────────────── + +@router.post("/connect") +async def connect(timeout: Optional[float] = Query(None), + attempts: Optional[int] = Query(None)): + mf = _require() + return await asyncio.to_thread(_run, mf.connect, timeout, attempts) + + +@router.post("/disconnect") +async def disconnect(): + mf = _require() + return await asyncio.to_thread(_run, mf.disconnect) + + +# ── simple commands ───────────────────────────────────────── + +@router.post("/brightness") +async def brightness(level: int = Query(..., ge=0, le=255)): + mf = _require() + return await asyncio.to_thread(_run, mf.set_brightness, level) + + +class TextBody(BaseModel): + text: str = "" + color: List[int] = [255, 255, 255] + mode: Optional[int] = None + bg: Optional[List[int]] = None # background RGB (None -> black) + speed: Optional[int] = None # scroll speed 0-255 (None -> firmware default) + + +@router.post("/text") +async def text(body: TextBody): + mf = _require() + bg = tuple(body.bg) if body.bg else None + return await asyncio.to_thread(_run, mf.set_text, body.text, tuple(body.color), + body.mode, bg, body.speed) + + +@router.post("/image") +async def image(id: int = Query(...)): + mf = _require() + return await asyncio.to_thread(_run, mf.show_image, id) + + +@router.post("/animation") +async def animation(id: int = Query(...)): + mf = _require() + return await asyncio.to_thread(_run, mf.play_animation, id) + + +@router.post("/clear") +async def clear(): + mf = _require() + return await asyncio.to_thread(_run, mf.clear_diy) + + +# ── animated face ─────────────────────────────────────────── + +@router.post("/face/start") +async def face_start(reload: bool = Query(False)): + mf = _require() + return await asyncio.to_thread(_run, mf.face_start, reload) + + +@router.post("/face/stop") +async def face_stop(): + mf = _require() + return await asyncio.to_thread(_run, mf.face_stop) + + +@router.post("/face/return") +async def face_return(): + """Resume the live animated face after a text/image/animation override.""" + mf = _require() + return await asyncio.to_thread(_run, mf.return_face) + + +class FaceColorBody(BaseModel): + eye: Optional[List[int]] = None # eye/iris RGB + mouth: Optional[List[int]] = None # mouth RGB + sclera: Optional[List[int]] = None # white-of-the-eye RGB + + +@router.post("/face/color") +async def face_color(body: FaceColorBody): + """Recolor the animated face (re-uploads the frame set if the face is live).""" + mf = _require() + return await asyncio.to_thread(_run, mf.set_face_color, body.eye, body.mouth, body.sclera) + + +@router.post("/speaking") +async def speaking(on: bool = Query(...)): + mf = _require() + return await asyncio.to_thread(_run, mf.set_speaking, on) + + +@router.post("/mouth") +async def mouth(level: int = Query(..., ge=0, le=3)): + mf = _require() + return await asyncio.to_thread(_run, mf.set_mouth, level) + + +@router.post("/expression/{name}") +async def expression(name: str): + mf = _require() + return await asyncio.to_thread(_run, mf.show_expression, name) diff --git a/dashboard/routes/mask_social.py b/dashboard/routes/mask_social.py new file mode 100644 index 0000000..4ecf514 --- /dev/null +++ b/dashboard/routes/mask_social.py @@ -0,0 +1,395 @@ +"""Social-media / QR display on the LED mask. + +Renders a QR code (for a preset Instagram account) or an uploaded image onto the +mask's 46x58 display and holds it via the FaceController's reserved scratch slot +until the animated face is resumed. The shared helper :func:`show_social_on_mask` +is also called from the Gemini ``[[SHOW:account]]`` relay wired in ``main.py``. + +Routes (under /api/mask): + POST /social/{account} -> show a preset Instagram QR + POST /qr -> upload an image (QR or any picture) + show it + POST /face/resume -> stop showing the scratch image, return to the face + GET /social -> list the preset accounts +""" + +from __future__ import annotations + +import asyncio +import io +import logging +import os +import sys +from pathlib import Path + +import re + +from fastapi import APIRouter, File, HTTPException, Query, UploadFile +from fastapi.responses import FileResponse + +log = logging.getLogger("sanad.mask_social") +router = APIRouter() # prefix "/api/mask" supplied by dashboard/app.py _REST_ROUTES + +# Preset Instagram accounts the mask can show as a QR. The mask is a low-res +# 46x58 panel, so a full-URL QR is dense; the black margin acts as the quiet +# zone and we scale modules crisply (NEAREST) to give it the best chance. +SOCIAL = { + "bu_sunaidah": {"handle": "@bu.sunaidah", + "url": "https://instagram.com/bu.sunaidah", + "short": "da.gd/VMkH8J"}, # -> instagram.com/bu.sunaidah (v1 QR) + "yslootahtech": {"handle": "@yslootahtech", + "url": "https://instagram.com/yslootahtech", + "short": "da.gd/Qr8RO"}, # -> instagram.com/yslootahtech (v1 QR) +} + + +def _ensure_mask_path() -> None: + """Make the flat Mask lib (colorface) importable from this route — using the + SAME dir the FaceController resolved (the Mask lib lives outside the repo).""" + d = os.environ.get("SANAD_MASK_DIR") + if not d: + try: + from Project.Sanad.main import mask_face as _mf # type: ignore + d = getattr(_mf, "mask_dir", None) + except Exception: + d = None + if not d: + d = str(Path(__file__).resolve().parents[2] / "Mask") + if d and d not in sys.path: + sys.path.insert(0, d) + + +def _get_face(): + from Project.Sanad.main import mask_face # type: ignore + if mask_face is None: + raise HTTPException(status_code=503, detail="mask face unavailable") + return mask_face + + +_EYE_BAND = 16 # top rows reserved for the cyan eyes; the code sits below them + + +def _compose_under_eyes(inner) -> bytes: + """Draw two cyan eyes across the top and place ``inner`` (a QR / image) in the + area BELOW them, then encode for the mask. Keeps the panel looking like a face + with a code under the eyes instead of a full-screen QR.""" + _ensure_mask_path() + import colorface as cf + from PIL import Image, ImageDraw + W, H = cf.DISPLAY_W, cf.DISPLAY_H + inner = inner.convert("RGB") + iw, ih = inner.size + # keep the code a small badge under the eyes (~70% of the space below them) + target = max(20, int(min(W, H - _EYE_BAND - 1) * 0.72)) + if iw <= target and ih <= target: + s = max(1, min(target // iw, target // ih)) # crisp integer up-scale (QR) + nw, nh = iw * s, ih * s + else: + s = min(target / iw, target / ih) # scale big images down + nw, nh = max(1, int(iw * s)), max(1, int(ih * s)) + inner = inner.resize((nw, nh), Image.NEAREST) + canvas = Image.new("RGB", (W, H), (0, 0, 0)) + g = ImageDraw.Draw(canvas) + eye = cf.DEFAULT_EYE + for cx in (W // 2 - 10, W // 2 + 10): # two eyes at the top + g.ellipse([cx - 5, 3, cx + 5, 13], fill=(255, 255, 255)) + g.ellipse([cx - 3, 5, cx + 3, 11], fill=eye) + g.ellipse([cx - 1, 7, cx + 1, 10], fill=(0, 0, 0)) + x = (W - nw) // 2 + y = _EYE_BAND + (H - _EYE_BAND - nh) // 2 + canvas.paste(inner, (max(0, x), max(_EYE_BAND, y))) + return cf.encode(canvas) + + +def _qr_bytes(url: str) -> bytes: + """Render a QR for ``url`` FULL-SCREEN with the largest crisp (integer) module + size the 46-wide panel allows — the only way it has any chance of scanning. + Only a ~version-1 QR (<=17 chars) reaches ~2 px/module; longer data is denser + and won't scan. Returns (bytes, qr_version).""" + _ensure_mask_path() + import qrcode + from PIL import Image + import colorface as cf + W, H = cf.DISPLAY_W, cf.DISPLAY_H + qr = qrcode.QRCode(error_correction=qrcode.constants.ERROR_CORRECT_L, + box_size=1, border=1) + qr.add_data(url) + qr.make(fit=True) + q = qr.make_image(fill_color=(255, 255, 255), + back_color=(0, 0, 0)).convert("RGB") + scale = max(1, min(W, H) // max(1, q.width)) # largest integer that fits + if scale > 1: + q = q.resize((q.width * scale, q.width * scale), Image.NEAREST) + canvas = Image.new("RGB", (W, H), (0, 0, 0)) + canvas.paste(q, ((W - q.width) // 2, (H - q.height) // 2)) + return cf.encode(canvas) + + +def _image_bytes(img) -> bytes: + """Show an uploaded QR/image FULL-SCREEN, crisp (NEAREST) — best effort.""" + _ensure_mask_path() + import colorface as cf + from PIL import Image + W, H = cf.DISPLAY_W, cf.DISPLAY_H + s = min(W, H) + img = img.convert("RGB").resize((s, s), Image.NEAREST) + canvas = Image.new("RGB", (W, H), (0, 0, 0)) + canvas.paste(img, ((W - s) // 2, (H - s) // 2)) + return cf.encode(canvas) + + +def show_social_on_mask(account: str) -> dict: + """Show the account's **scannable** QR on the mask — a version-1 QR made from + a short (da.gd) link that redirects to the Instagram profile. Shared by the + dashboard button and the Gemini ``show_social`` tool. Raises for an unknown + account; propagates FaceController errors (e.g. not connected).""" + acc = SOCIAL.get(str(account).strip().lower()) + if not acc: + raise HTTPException(status_code=404, detail="unknown account") + data = _qr_bytes(acc.get("short") or acc["url"]) # v1 short link -> scannable + mf = _get_face() + res = mf.show_scratch_image(data) + log.info("showing scannable social QR on mask: %s (%s)", acc["handle"], acc.get("short")) + return {"ok": True, "handle": acc["handle"], "scannable": True, **(res or {})} + + +@router.get("/social") +async def list_social(): + return {"accounts": [{"id": k, "handle": v["handle"]} for k, v in SOCIAL.items()]} + + +def _friendly(exc: Exception) -> HTTPException: + """Map FaceController errors to clean HTTP responses (esp. the common + 'mask not connected' — usually the mask is off / far / held by the phone app).""" + if isinstance(exc, HTTPException): + return exc + msg = str(exc) + if "not connected" in msg or "not started" in msg or "MASK" in msg: + return HTTPException(status_code=503, detail=( + "Mask not connected — power it on, bring it close to the robot, and " + "free it from the phone app.")) + log.exception("mask scratch op failed") + return HTTPException(status_code=500, detail="%s: %s" % (type(exc).__name__, msg)) + + +@router.post("/social/{account}") +async def show_social(account: str): + try: + return await asyncio.to_thread(show_social_on_mask, account) + except Exception as exc: + raise _friendly(exc) + + +@router.post("/qr") +async def upload_qr(file: UploadFile = File(...)): + """Upload an image (a QR you generated, or any picture) and show it on the mask.""" + raw = await file.read() + if not raw: + raise HTTPException(status_code=400, detail="empty upload") + from PIL import Image + try: + img = Image.open(io.BytesIO(raw)) + img.load() + except Exception: + raise HTTPException(status_code=400, detail="not a valid image") + try: + data = await asyncio.to_thread(_image_bytes, img) + mf = _get_face() + return await asyncio.to_thread(mf.show_scratch_image, data) + except Exception as exc: + raise _friendly(exc) + + +@router.post("/face/resume") +async def resume_face(): + """Stop showing the scratch image and resume the animated face.""" + mf = _get_face() + return await asyncio.to_thread(mf.set_expression, None) + + +@router.post("/face/mouth") +async def face_mouth(hidden: bool = Query(...)): + """Show (hidden=false) or hide (hidden=true) the mouth on the animated face.""" + mf = _get_face() + return await asyncio.to_thread(mf.set_mouth_hidden, hidden) + + +@router.post("/link") +async def face_link(on: bool = Query(...)): + """Link (on=true) / unlink (on=false) Gemini <-> the mask. + + ON connects the mask + lets Gemini drive its emotions/social. + OFF tears the link down (no BLE churn) and Gemini stops touching the mask. + Default state is OFF. Runs in a thread — a link-on may briefly block while it + makes its first connect attempt.""" + mf = _get_face() + return await asyncio.to_thread(mf.set_gemini_linked, on) + + +# ── saved QR library ──────────────────────────────────────────────── +# Upload QR/images, save them by name, list/show/delete them. Stored as PNGs +# under data/qr_codes so they persist across restarts. + +_QR_DIR = None + + +def _qr_dir() -> Path: + global _QR_DIR + if _QR_DIR is None: + try: + from Project.Sanad.config import BASE_DIR + base = Path(BASE_DIR) + except Exception: + base = Path(__file__).resolve().parents[2] + _QR_DIR = base / "data" / "qr_codes" + _QR_DIR.mkdir(parents=True, exist_ok=True) + return _QR_DIR + + +def _safe_name(name: str) -> str: + n = re.sub(r"[^A-Za-z0-9_.-]", "_", (name or "").strip())[:40].strip("._") + return n or "qr" + + +@router.post("/qr/save") +async def qr_save(name: str = Query(...), file: UploadFile = File(...)): + """Save an uploaded QR/image into the library under ``name``.""" + raw = await file.read() + if not raw: + raise HTTPException(status_code=400, detail="empty upload") + from PIL import Image + try: + img = Image.open(io.BytesIO(raw)) + img.load() + except Exception: + raise HTTPException(status_code=400, detail="not a valid image") + sn = _safe_name(name) + await asyncio.to_thread(img.convert("RGB").save, str(_qr_dir() / (sn + ".png"))) + return {"ok": True, "name": sn} + + +@router.post("/qr/save_link") +async def qr_save_link(name: str = Query(...), url: str = Query(...)): + """Generate a QR from ``url`` and save it to the library. Returns the QR + version + whether it's short enough to actually scan on the mask (version 1).""" + u = (url or "").strip() + if not u: + raise HTTPException(status_code=400, detail="empty url") + _ensure_mask_path() + import qrcode + qr = qrcode.QRCode(error_correction=qrcode.constants.ERROR_CORRECT_L, + box_size=10, border=2) + qr.add_data(u) + qr.make(fit=True) + img = qr.make_image(fill_color=(255, 255, 255), + back_color=(0, 0, 0)).convert("RGB") + sn = _safe_name(name or u) + await asyncio.to_thread(img.save, str(_qr_dir() / (sn + ".png"))) + return {"ok": True, "name": sn, "version": qr.version, + "scannable_on_mask": qr.version <= 1, + "note": ("scannable" if qr.version <= 1 else + "too dense to scan on the mask — use a shorter link")} + + +@router.get("/qr/library") +async def qr_library(): + """List the saved QR names.""" + return {"qr": sorted(p.stem for p in _qr_dir().glob("*.png"))} + + +@router.get("/qr/thumb/{name}") +async def qr_thumb(name: str): + """Serve a saved QR image (for the dashboard thumbnail).""" + p = _qr_dir() / (_safe_name(name) + ".png") + if not p.exists(): + raise HTTPException(status_code=404, detail="not found") + return FileResponse(str(p), media_type="image/png") + + +@router.post("/qr/show/{name}") +async def qr_show(name: str): + """Show a saved QR (under the eyes) on the mask.""" + p = _qr_dir() / (_safe_name(name) + ".png") + if not p.exists(): + raise HTTPException(status_code=404, detail="not found") + from PIL import Image + try: + img = Image.open(p) + data = await asyncio.to_thread(_image_bytes, img) + mf = _get_face() + return await asyncio.to_thread(mf.show_scratch_image, data) + except Exception as exc: + raise _friendly(exc) + + +@router.delete("/qr/{name}") +async def qr_delete(name: str): + """Delete a saved QR from the library.""" + p = _qr_dir() / (_safe_name(name) + ".png") + if p.exists(): + p.unlink() + return {"ok": True, "deleted": _safe_name(name)} + + +# ── saved TEXT library ────────────────────────────────────────────── +# Save words/phrases and scroll any of them across the mask on demand. + +_TEXT_DIR = None + + +def _text_dir() -> Path: + global _TEXT_DIR + if _TEXT_DIR is None: + try: + from Project.Sanad.config import BASE_DIR + base = Path(BASE_DIR) + except Exception: + base = Path(__file__).resolve().parents[2] + _TEXT_DIR = base / "data" / "mask_texts" + _TEXT_DIR.mkdir(parents=True, exist_ok=True) + return _TEXT_DIR + + +@router.post("/texts/save") +async def text_save(text: str = Query(...), name: str = Query("")): + """Save a word/phrase to the text library (name defaults to the text).""" + t = (text or "").strip()[:200] + if not t: + raise HTTPException(status_code=400, detail="empty text") + nm = _safe_name(name or t) + await asyncio.to_thread((_text_dir() / (nm + ".txt")).write_text, t) + return {"ok": True, "name": nm, "text": t} + + +@router.get("/texts/library") +async def text_library(): + """List the saved texts.""" + out = [] + for p in sorted(_text_dir().glob("*.txt")): + try: + out.append({"name": p.stem, "text": p.read_text()[:80]}) + except Exception: + pass + return {"texts": out} + + +@router.post("/texts/show/{name}") +async def text_show(name: str): + """Scroll a saved text across the mask.""" + p = _text_dir() / (_safe_name(name) + ".txt") + if not p.exists(): + raise HTTPException(status_code=404, detail="not found") + txt = p.read_text() + mf = _get_face() + try: + return await asyncio.to_thread(mf.set_text, txt, (255, 255, 255), None, None, 38) + except Exception as exc: + raise _friendly(exc) + + +@router.delete("/texts/{name}") +async def text_delete(name: str): + """Delete a saved text.""" + p = _text_dir() / (_safe_name(name) + ".txt") + if p.exists(): + p.unlink() + return {"ok": True, "deleted": _safe_name(name)} diff --git a/dashboard/routes/navigation.py b/dashboard/routes/navigation.py new file mode 100644 index 0000000..0a2ef8b --- /dev/null +++ b/dashboard/routes/navigation.py @@ -0,0 +1,402 @@ +"""Navigation tab — proxy to the web_nav3 Nav2 stack. + +Routes live under /api/nav (the prefix is applied centrally in dashboard/app.py, +NOT here). This router is a thin HTTP proxy: it forwards dashboard requests to a +single module-level WebNav3Client, which itself talks to the standalone web_nav3 +FastAPI service (default http://127.0.0.1:8765 + rosbridge on :9090). + +Fault isolation, two layers: + 1. The `from ...navigation import WebNav3Client` import is GUARDED. If the + navigation package can't be imported (missing dep, syntax error), this + module still imports cleanly — `_CLIENT` is None and every handler degrades + (GET /status returns {"available": False}; actions raise 503). This mirrors + how app.py loads each router in isolation. + 2. WebNav3Client never raises into us by contract — every method returns a + clean dict / NavStatus even when web_nav3 is unreachable — so handlers just + forward the result. Blocking HTTP calls run off the event loop. +""" + +from __future__ import annotations + +import asyncio + +from fastapi import APIRouter, HTTPException, Query +from pydantic import BaseModel + +from Project.Sanad.core.logger import get_logger + +from Project.Sanad.dashboard.routes import _arbiter + +log = get_logger("navigation_route") + +# Module-level router with NO prefix and NO tags — those are supplied by +# app.include_router(prefix="/api/nav", tags=["navigation"]) at registration time. +router = APIRouter() + + +# ── guarded optional import ───────────────────────────────── +# A broken navigation package must NOT stop this route module from importing — +# app.py would otherwise log the whole router as failed. Guard it and degrade. +try: + from Project.Sanad.navigation import WebNav3Client # type: ignore + _IMPORT_ERROR: str | None = None +except Exception as exc: # noqa: BLE001 + WebNav3Client = None # type: ignore[assignment,misc] + _IMPORT_ERROR = f"{type(exc).__name__}: {exc}" + log.warning("navigation client unavailable — nav routes degraded: %s", _IMPORT_ERROR) + + +# ── config (env var -> dashboard config section -> default) ── +def _nav_config() -> dict: + """Resolve nav connection config. Precedence: env var -> config -> default.""" + import os + + from Project.Sanad.core.config_loader import section as _cfg_section + + cfg = _cfg_section("dashboard", "navigation") + web_nav3_url = ( + os.environ.get("WEB_NAV3_URL") + or cfg.get("web_nav3_url") + or "http://127.0.0.1:8765" + ) + rosbridge_url = ( + os.environ.get("ROSBRIDGE_URL") + or cfg.get("rosbridge_url") + or "ws://127.0.0.1:9090" + ) + robot = os.environ.get("SANAD_ROBOT_NAME") or cfg.get("robot") or "sanad" + return { + "web_nav3_url": str(web_nav3_url), + "rosbridge_url": str(rosbridge_url), + "robot": str(robot), + } + + +_CFG = _nav_config() + +# ── single module-level client ────────────────────────────── +# One WebNav3Client for the whole dashboard, built from config. If the import +# was guarded out (above), or construction fails, _CLIENT stays None and every +# handler degrades gracefully. +if WebNav3Client is not None: + try: + _CLIENT = WebNav3Client(base_url=_CFG["web_nav3_url"], robot=_CFG["robot"]) + log.info("WebNav3Client ready → %s (robot=%s)", _CFG["web_nav3_url"], _CFG["robot"]) + except Exception as exc: # noqa: BLE001 + _CLIENT = None + _IMPORT_ERROR = f"construct failed: {type(exc).__name__}: {exc}" + log.warning("WebNav3Client construction failed — nav routes degraded: %s", exc) +else: + _CLIENT = None + + +def _require(): + """Return the live client or raise 503 (for ACTION endpoints).""" + if _CLIENT is None: + raise HTTPException(503, f"Navigation client unavailable. {_IMPORT_ERROR or ''}".strip()) + return _CLIENT + + +def _claim_nav(): + """Arbitration gate: refuse to start a Nav2 goal while manual loco owns legs.""" + if not _arbiter.acquire_nav(): + raise HTTPException( + 409, "Manual movement (Controller) is armed. Disarm it before navigating." + ) + + +# ── request bodies ────────────────────────────────────────── +class _NameBody(BaseModel): + name: str + + +class _IdBody(BaseModel): + id: object # mission ids may be int or str; forward as-is + + +class _StartBody(BaseModel): + mode: int = 2 # web_nav3 launch mode (e.g. 3 = localize against a saved map) + db_path: str | None = None # saved map to load (None = build fresh) + + +class _PoseBody(BaseModel): + name: str + x: float + y: float + yaw: float = 0.0 + + +class _RenameBody(BaseModel): + old: str + new: str + + +# ── status (never raises — degraded body when unavailable) ── +@router.get("/status") +async def status(): + if _CLIENT is None: + return {"available": False, "error": _IMPORT_ERROR} + nav = await asyncio.to_thread(_CLIENT.status) + # WebNav3Client.status() returns a NavStatus dataclass. + body = nav.as_dict() if hasattr(nav, "as_dict") else dict(nav) + body["available"] = True + return body + + +# ── places / navigation ───────────────────────────────────── +@router.get("/places") +async def places(map_name: str | None = Query(None, alias="map")): + """List saved places. Per-MAP when ?map= is given (each map keeps + its own places); else the legacy per-robot store.""" + client = _require() + return await asyncio.to_thread(client.list_places, map_name) + + +@router.post("/goto") +async def goto(body: _NameBody): + client = _require() + _claim_nav() + res = await asyncio.to_thread(client.goto, body.name) + # A failed dispatch never drove the legs — release the gate so manual loco + # isn't locked out by a goto that never started. + if isinstance(res, dict) and not res.get("ok", True): + _arbiter.release_nav() + return res + + +@router.post("/start") +async def start(body: _StartBody): + client = _require() + return await asyncio.to_thread(client.start, body.mode, body.db_path) + + +class _DbBody(BaseModel): + db_path: str + + +@router.post("/load_map") +async def load_map(body: _DbBody): + """View a saved map: stop any running bringup, then localize against it.""" + client = _require() + return await asyncio.to_thread(client.load_map, body.db_path) + + +@router.post("/cancel") +async def cancel(): + client = _require() + res = await asyncio.to_thread(client.cancel) + # WebNav3Client.cancel() is a no-op server-side (it only returns a note), + # so releasing the arbiter without truly stopping Nav2 would let the robot + # keep driving while manual loco re-acquires the legs (double-drive). Send a + # REAL goal-cancel over rosbridge first, and disarm the arrival monitor so a + # stale terminal can't fire, THEN release. + try: + from Project.Sanad.navigation.goal_monitor import request_cancel, disarm + disarm() + cancelled = await asyncio.to_thread(request_cancel) + if isinstance(res, dict): + res = {**res, "cancel_sent": bool(cancelled)} + except Exception as exc: # noqa: BLE001 + log.debug("goal cancel skipped: %s", exc) + _arbiter.release_nav() + return res + + +@router.post("/save_here") +async def save_here(body: _NameBody): + client = _require() + return await asyncio.to_thread(client.save_here, body.name) + + +@router.post("/save_at") +async def save_at(body: _PoseBody, map_name: str | None = Query(None, alias="map")): + """Save a named place at a map coordinate (from clicking the map). Per-MAP + when ?map= given. Re-saving an existing name MOVES the place.""" + client = _require() + return await asyncio.to_thread(client.save_at, body.name, body.x, body.y, body.yaw, map_name) + + +@router.post("/places/delete") +async def delete_place(body: _NameBody, map_name: str | None = Query(None, alias="map")): + """Delete a saved place (per-map).""" + client = _require() + return await asyncio.to_thread(client.delete_place, body.name, map_name) + + +@router.post("/places/rename") +async def rename_place(body: _RenameBody, map_name: str | None = Query(None, alias="map")): + """Rename a saved place (per-map).""" + client = _require() + return await asyncio.to_thread(client.rename_place, body.old, body.new, map_name) + + +class _MapEditsBody(BaseModel): + edits: list # [[world_x, world_y, value], ...] value 0=free/erase, 100=wall + + +@router.get("/map_edits") +async def get_map_edits(map_name: str = Query(..., alias="map")): + """Saved edit overlay for a map (erased points + painted walls).""" + client = _require() + return await asyncio.to_thread(client.get_map_edits, map_name) + + +@router.post("/map_edits") +async def save_map_edits(body: _MapEditsBody, map_name: str = Query(..., alias="map")): + """Persist a map's edit overlay (Map Editor).""" + client = _require() + return await asyncio.to_thread(client.save_map_edits, map_name, body.edits) + + +class _VoiceGotoBody(BaseModel): + place: str + + +def _resolve_place(client, spoken: str) -> dict: + """Resolve a spoken place name against the ACTIVE map's places. + + Strategy: exact (case-insensitive) → single substring candidate → + ambiguous / unknown. Returns a dict the caller (and ultimately Gemini) + can act on. Never raises. + """ + try: + st = client.status() + body = st.as_dict() if hasattr(st, "as_dict") else dict(st) + except Exception as exc: # noqa: BLE001 + return {"ok": False, "reason": "status_error", "detail": str(exc)[:160]} + if not body.get("bringup_alive"): + return {"ok": False, "reason": "no_map", + "detail": "No navigation session is running — load a map first."} + active_map = body.get("active_map") + try: + places = client.list_places(active_map) or [] + except Exception: # noqa: BLE001 + places = [] + names = [p.get("name") for p in places if isinstance(p, dict) and p.get("name")] + sl = (spoken or "").strip().lower() + if not sl: + return {"ok": False, "reason": "no_place", "map": active_map, "places": names} + exact = [n for n in names if n.lower() == sl] + if exact: + return {"ok": True, "resolved": exact[0], "map": active_map} + subs = [] + for n in names: + nl = n.lower() + if sl in nl or nl in sl: + subs.append(n) + subs = list(dict.fromkeys(subs)) # de-dup, preserve order + if len(subs) == 1: + return {"ok": True, "resolved": subs[0], "map": active_map} + if len(subs) > 1: + return {"ok": False, "reason": "ambiguous", "candidates": subs, "map": active_map} + return {"ok": False, "reason": "unknown_place", "candidates": names, "map": active_map} + + +@router.get("/active") +async def active(): + """Navigation context for Gemini: the active map, its mode, and that map's + place names — one call so the voice tools (list_places / where_am_i) don't + have to guess the active map.""" + client = _require() + st = await asyncio.to_thread(client.status) + body = st.as_dict() if hasattr(st, "as_dict") else dict(st) + places = [] + if body.get("bringup_alive"): + try: + pl = await asyncio.to_thread(client.list_places, body.get("active_map")) + places = [p.get("name") for p in (pl or []) + if isinstance(p, dict) and p.get("name")] + except Exception: # noqa: BLE001 + places = [] + return { + "map": body.get("active_map"), + "mode": body.get("mode"), + "mode_label": body.get("mode_label"), + "localizing": bool(body.get("localizing")), + "bringup_alive": bool(body.get("bringup_alive")), + "places": places, + } + + +@router.post("/voice_goto") +async def voice_goto(body: _VoiceGotoBody): + """Resolve a spoken place name and drive there — Gemini's navigate_to_place. + + Arbiter-gated (claims the legs for Nav2) and arms the arrival monitor so + Gemini later hears [NAV ARRIVED]/[NAV FAILED]. Never raises into the caller; + returns a structured result the model can speak from. + """ + client = _require() + res = await asyncio.to_thread(_resolve_place, client, body.place or "") + if not res.get("ok"): + return res + # Claim the legs for Nav2 — refuse (don't raise) if manual loco is armed. + if not _arbiter.acquire_nav(): + return {"ok": False, "reason": "manual_armed", + "detail": "Manual movement (Controller) is armed — disarm it to navigate."} + drive = await asyncio.to_thread(client.goto, res["resolved"]) + if isinstance(drive, dict) and not drive.get("ok", True): + _arbiter.release_nav() + return {"ok": False, "reason": "dispatch_failed", + "resolved": res["resolved"], "detail": drive} + # Arm arrival monitoring (best-effort; absence must not fail the drive). + try: + from Project.Sanad.navigation.goal_monitor import arm_goal + arm_goal(res["resolved"]) + except Exception as exc: # noqa: BLE001 + log.debug("goal monitor arm skipped: %s", exc) + return {"ok": True, "resolved": res["resolved"], "map": res.get("map")} + + +@router.post("/goto_pose") +async def goto_pose(body: _PoseBody): + """Arbiter-gate a coordinate nav goal (click-to-drive). + + The browser publishes the actual /goal_pose over rosbridge; this only + CLAIMS the legs for Nav2 (409 if manual loco is armed) so the two stacks + never both drive. The frontend sends the goal only after this returns ok. + """ + _require() + _claim_nav() + # Arm the arrival monitor so this click-to-drive goal releases the arbiter + # when it ends — without this, nav_active stays True forever after the goal + # completes (the browser publishes the goal but never arms anything). + try: + from Project.Sanad.navigation.goal_monitor import arm_goal + arm_goal(f"({body.x:.1f}, {body.y:.1f})") + except Exception as exc: # noqa: BLE001 + log.debug("goal monitor arm skipped: %s", exc) + return {"ok": True, "x": body.x, "y": body.y, "yaw": body.yaw} + + +# ── maps / missions ───────────────────────────────────────── +@router.get("/maps") +async def maps(): + client = _require() + return await asyncio.to_thread(client.list_maps) + + +@router.get("/missions") +async def missions(): + client = _require() + return await asyncio.to_thread(client.list_missions) + + +@router.post("/missions/run") +async def run_mission(body: _IdBody): + client = _require() + _claim_nav() + res = await asyncio.to_thread(client.run_mission, body.id) + if isinstance(res, dict) and not res.get("ok", True): + _arbiter.release_nav() + return res + + +# ── config (what the SPA needs to render links / connect) ─── +@router.get("/config") +async def config(): + return { + "web_nav3_url": _CFG["web_nav3_url"], + "rosbridge_url": _CFG["rosbridge_url"], + "robot": _CFG["robot"], + } diff --git a/dashboard/routes/records.py b/dashboard/routes/records.py index 8846e6d..f9212c1 100644 --- a/dashboard/routes/records.py +++ b/dashboard/routes/records.py @@ -23,6 +23,12 @@ router = APIRouter() RECORDS_INDEX = AUDIO_RECORDINGS_DIR / "records.json" _INDEX_LOCK = threading.Lock() +# Strong refs to fire-and-forget playback tasks. The event loop only keeps a +# weak reference to tasks, so an unreferenced create_task() result can be +# garbage-collected (cancelling playback) before it finishes. Mirror replay.py. +import asyncio as _asyncio # noqa: E402 +_BG_TASKS: set[_asyncio.Task] = set() + def _load_index() -> dict[str, Any]: if not RECORDS_INDEX.exists(): @@ -110,15 +116,19 @@ async def play_record(payload: RecordPlay): raise HTTPException(404, f"File not found: {raw_path.name}") from Project.Sanad.main import audio_mgr - import asyncio - # Fire-and-forget — play_wav blocks for the clip duration on the G1 - # DDS path, and the dashboard's pause / resume / stop / status calls - # need to be served while it's running. Without this, /play wouldn't - # return until the clip finished and the UI couldn't interact with - # the in-flight playback. - asyncio.create_task(asyncio.to_thread( - audio_mgr.play_wav, raw_path, payload.record_name, - )) + import threading + # Fire-and-forget on a DEDICATED daemon thread — NOT asyncio.to_thread. + # to_thread runs on the shared default executor, which gets starved while + # the dashboard services the live-voice child's reconnect chatter; that + # delayed record playback by several seconds (clip silent, counter parked). + # A dedicated thread starts immediately regardless of executor/event-loop + # load. play_wav blocks for the clip duration and serves pause/stop via + # _play_state; the UI stays responsive because this handler returns now. + # Python keeps running threads alive, so no ref is needed to prevent GC. + threading.Thread( + target=audio_mgr.play_wav, args=(raw_path, payload.record_name), + name="record-playback", daemon=True, + ).start() return {"ok": True, "record_name": payload.record_name, "file_kind": payload.file_kind, "path": str(raw_path)} @@ -135,6 +145,14 @@ async def resume_playback(): return audio_mgr.resume_playback() +@router.post("/seek") +async def seek_playback(position_sec: float): + """Jump to a position (seconds) in the currently-playing clip — used by the + waveform scrubber. No-op (ok=False) if nothing is playing.""" + from Project.Sanad.main import audio_mgr + return audio_mgr.seek_playback(position_sec) + + @router.post("/stop") async def stop_playback(): from Project.Sanad.main import audio_mgr @@ -149,6 +167,15 @@ async def playback_status(): return audio_mgr.playback_status() +@router.post("/live-hold") +async def set_live_hold(on: bool): + """Manual hold for the live-Gemini pause. on=True pauses the live voice and + keeps it paused (records won't resume it) until on=False is sent. Default + behaviour (on=False) is AUTO: records pause Gemini only for the clip.""" + from Project.Sanad.main import audio_mgr + return {"live_hold": audio_mgr.set_live_voice_hold(on)} + + class RecordRename(BaseModel): record_name: str new_name: str @@ -217,7 +244,12 @@ async def delete_record(payload: RecordDelete): deleted_files = [] for fi in deleted_entry.get("files", {}).values(): try: - p = Path(fi.get("path", "")).resolve() + # _resolve_path handles new-style basenames (resolved under + # AUDIO_RECORDINGS_DIR) as well as legacy absolute paths. + # A raw Path(basename) would resolve vs CWD and fall outside + # base, so the relative_to guard would skip the unlink and the + # WAV would be orphaned on disk. Mirror play_record/rename_record. + p = _resolve_path(fi.get("path", "")).resolve() p.relative_to(base) # never delete files outside recordings dir except (ValueError, OSError): continue @@ -228,3 +260,43 @@ async def delete_record(payload: RecordDelete): index["records"] = kept _save_index(index) return {"ok": True, "deleted": payload.record_name, "deleted_files": deleted_files} + + +class RecordBulkDelete(BaseModel): + record_names: list[str] | None = None + all: bool = False + + +@router.post("/delete-bulk") +async def delete_bulk(payload: RecordBulkDelete): + """Delete many records in one call. all=True wipes every record; otherwise + only those in record_names. Files are unlinked, guarded to the recordings + dir (same safety as /delete).""" + names = set(payload.record_names or []) + with _INDEX_LOCK: + index = _load_index() + base = AUDIO_RECORDINGS_DIR.resolve() + kept: list = [] + removed: list = [] + deleted_files = 0 + for r in index.get("records", []): + if payload.all or r.get("record_name") in names: + removed.append(r.get("record_name")) + for fi in r.get("files", {}).values(): + try: + p = _resolve_path(fi.get("path", "")).resolve() + p.relative_to(base) # never delete outside recordings dir + except (ValueError, OSError): + continue + if p.exists(): + try: + p.unlink() + deleted_files += 1 + except OSError: + pass + else: + kept.append(r) + index["records"] = kept + _save_index(index) + return {"ok": True, "deleted": removed, "deleted_count": len(removed), + "deleted_files": deleted_files} diff --git a/dashboard/routes/scripts.py b/dashboard/routes/scripts.py index b4c65a5..f4f18f2 100644 --- a/dashboard/routes/scripts.py +++ b/dashboard/routes/scripts.py @@ -2,6 +2,7 @@ from __future__ import annotations +import asyncio from datetime import datetime from pathlib import Path @@ -9,6 +10,7 @@ from fastapi import APIRouter, HTTPException from pydantic import BaseModel from Project.Sanad.config import SCRIPTS_DIR +from Project.Sanad.core import persona as _persona from Project.Sanad.dashboard.routes._safe_io import ( atomic_write_text, MAX_UPLOAD_BYTES, ) @@ -31,6 +33,8 @@ def _safe_path(name: str) -> Path: @router.get("/") async def list_scripts(): SCRIPTS_DIR.mkdir(parents=True, exist_ok=True) + active = _persona.active_persona_name() + default = _persona.default_persona_name() items = [] for p in sorted(SCRIPTS_DIR.iterdir(), key=lambda x: x.name.lower()): if not p.is_file(): @@ -40,8 +44,48 @@ async def list_scripts(): "name": p.name, "size_bytes": st.st_size, "modified_at": datetime.fromtimestamp(st.st_mtime).isoformat(timespec="seconds"), + "active": p.name == active, # the persona Gemini loads now + "is_default": p.name == default, # the fallback (sanad_script.txt) }) - return {"path": str(SCRIPTS_DIR), "files": items} + return {"path": str(SCRIPTS_DIR), "files": items, + "active": active, "default": default} + + +class ScriptActive(BaseModel): + name: str | None = None # None / "" / the default name → revert to default + restart: bool = False # also restart the live voice so it takes effect now + + +@router.get("/active") +async def get_active(): + """Which persona Gemini will load, and the default it falls back to.""" + return {"active": _persona.active_persona_name(), + "default": _persona.default_persona_name()} + + +@router.post("/active") +async def set_active(payload: ScriptActive): + """Select the persona script Gemini uses. With restart=true, the live voice + session is bounced so the new persona takes effect immediately; otherwise it + applies on the next voice (re)connect.""" + try: + active = _persona.set_active_persona(payload.name) + except FileNotFoundError: + raise HTTPException(404, f"Script not found: {payload.name}") + restarted = False + if payload.restart: + try: + from Project.Sanad.main import live_sub + if live_sub is not None and hasattr(live_sub, "start"): + if hasattr(live_sub, "is_running") and live_sub.is_running(): + await asyncio.to_thread(live_sub.stop) + await asyncio.sleep(1.5) + await asyncio.to_thread(live_sub.start) + restarted = True + except Exception: + pass # selection is saved regardless of restart success + return {"ok": True, "active": active, + "default": _persona.default_persona_name(), "restarted": restarted} class ScriptLoad(BaseModel): @@ -116,5 +160,9 @@ async def delete_script(payload: ScriptDelete): path = _safe_path(payload.name) if not path.exists(): raise HTTPException(404, f"Not found: {payload.name}") + if path.name == _persona.default_persona_name(): + raise HTTPException(409, f"Cannot delete the default persona ({path.name}).") path.unlink() + # If the active selection was the deleted file, resolution auto-falls-back + # to the default — no extra cleanup needed. return {"ok": True, "deleted": payload.name} diff --git a/dashboard/routes/system.py b/dashboard/routes/system.py index 4839579..aba03f4 100644 --- a/dashboard/routes/system.py +++ b/dashboard/routes/system.py @@ -5,18 +5,24 @@ from __future__ import annotations import asyncio import os import platform +import shutil import socket import sys +from pathlib import Path from typing import Any -from fastapi import APIRouter +from fastapi import APIRouter, HTTPException +from pydantic import BaseModel from Project.Sanad.config import ( + AUDIO_RECORDINGS_DIR, BASE_DIR, DASHBOARD_HOST, DASHBOARD_INTERFACE, DASHBOARD_PORT, + DATA_DIR, DDS_NETWORK_INTERFACE, + LOGS_DIR, list_network_interfaces, ) from Project.Sanad.core.logger import get_logger @@ -26,6 +32,36 @@ log = get_logger("system_route") router = APIRouter() +def _runtime_bind() -> tuple[str, int]: + """The host/port the server is ACTUALLY bound to. + + main.py launches `uvicorn.run(_app, host=args.host, port=args.port)` with + the CLI --host/--port (start_sanad.sh passes `--port $PORT`, default 8001), + which can differ from the import-time DASHBOARD_HOST/DASHBOARD_PORT config + defaults (port 8000). Reading the live argv reports the real URL instead of + a stale config value. Falls back to the config constants when an arg is + absent (e.g. argparse default in effect).""" + host = DASHBOARD_HOST + port = DASHBOARD_PORT + argv = sys.argv + for i, tok in enumerate(argv): + if tok == "--host" and i + 1 < len(argv): + host = argv[i + 1] + elif tok.startswith("--host="): + host = tok.split("=", 1)[1] + elif tok == "--port" and i + 1 < len(argv): + try: + port = int(argv[i + 1]) + except (TypeError, ValueError): + pass + elif tok.startswith("--port="): + try: + port = int(tok.split("=", 1)[1]) + except (TypeError, ValueError): + pass + return host, port + + def _safe_status(component, name: str) -> dict[str, Any]: if component is None: return {"available": False} @@ -90,8 +126,9 @@ async def system_info(): except Exception: interfaces = [] - # Determine the URL the dashboard is reachable at - bound_host = DASHBOARD_HOST + # Determine the URL the dashboard is reachable at — use the ACTUAL + # runtime bind args (argv), not the import-time config defaults. + bound_host, bound_port = _runtime_bind() if bound_host == "0.0.0.0": # Try to find the wlan0 IP for display purposes up_ifaces = [i for i in interfaces if i["is_up"] and i["ip"] and not i["ip"].startswith("127.")] @@ -112,8 +149,8 @@ async def system_info(): "interface": DASHBOARD_INTERFACE, "bound_host": bound_host, "display_host": display_host, - "port": DASHBOARD_PORT, - "url": f"http://{display_host}:{DASHBOARD_PORT}", + "port": bound_port, + "url": f"http://{display_host}:{bound_port}", }, "dds": { "interface": DDS_NETWORK_INTERFACE, @@ -131,3 +168,148 @@ async def system_info(): } return await asyncio.to_thread(_do) + + +# ───────────────────── storage tracking + cleanup ───────────────────── +# Categories surfaced in the Settings → Storage panel. `cleanable` ones get a +# Clean button + are included in "Clean all"; the rest (faces/motions/zones) +# are shown for tracking only — they're operational assets (enrollments, +# motion configs) managed in their own tabs, not disposable clutter. +_STORAGE_CATS = [ + ("recordings", "Conversation recordings", DATA_DIR / "recordings", True), + ("records", "Named records (Typed Replay)", AUDIO_RECORDINGS_DIR, True), + ("logs", "Logs", LOGS_DIR, True), + ("faces", "Enrolled faces", DATA_DIR / "faces", False), + ("motions", "Motion replays + config", DATA_DIR / "motions", False), + ("photos", "Photos", DATA_DIR / "photos", False), + ("zones", "Vision zones", DATA_DIR / "zones", False), +] +_CLEANABLE = {k for k, _l, _p, c in _STORAGE_CATS if c} + + +def _dir_stats(path: Path) -> tuple[int, int]: + """(total_bytes, file_count) of a dir tree. Missing dir → (0, 0).""" + total, n = 0, 0 + try: + for root, _dirs, files in os.walk(path): + for f in files: + try: + total += os.path.getsize(os.path.join(root, f)) + n += 1 + except OSError: + pass + except Exception: + pass + return total, n + + +def _human(b: float) -> str: + f = float(b) + for u in ("B", "KB", "MB", "GB", "TB"): + if f < 1024 or u == "TB": + return f"{f:.0f} {u}" if u == "B" else f"{f:.1f} {u}" + f /= 1024 + return f"{f:.1f} TB" + + +@router.get("/storage") +async def storage_usage(): + """Per-category data/log sizes + disk free, for the Storage panel.""" + def _do(): + cats = [] + for key, label, path, cleanable in _STORAGE_CATS: + size, files = _dir_stats(Path(path)) + cats.append({ + "key": key, "label": label, "path": str(path), + "size_bytes": size, "size_human": _human(size), + "files": files, "cleanable": cleanable, + }) + data_b, _ = _dir_stats(DATA_DIR) + logs_b, _ = _dir_stats(LOGS_DIR) + try: + du = shutil.disk_usage(str(BASE_DIR)) + disk = { + "free_human": _human(du.free), "total_human": _human(du.total), + "used_pct": round(100.0 * (du.total - du.free) / du.total, 1), + } + except Exception: + disk = {} + return { + "categories": cats, + "data_bytes": data_b, "data_human": _human(data_b), + "logs_human": _human(logs_b), + "total_human": _human(data_b + logs_b), + "disk": disk, + } + return await asyncio.to_thread(_do) + + +class _CleanReq(BaseModel): + target: str # recordings | records | logs | all + + +def _clean_recordings() -> tuple[int, int]: + d = DATA_DIR / "recordings" + freed, n = 0, 0 + for f in list(d.glob("*.wav")) + [d / "index.json"]: + if f.is_file(): + try: + freed += f.stat().st_size + f.unlink() + n += 1 + except OSError: + pass + return n, freed + + +def _clean_records() -> tuple[int, int]: + d = AUDIO_RECORDINGS_DIR + freed, n = 0, 0 + for f in list(d.glob("*.wav")) + [d / "records.json"]: + if f.is_file(): + try: + freed += f.stat().st_size + f.unlink() + n += 1 + except OSError: + pass + return n, freed + + +def _clean_logs() -> tuple[int, int]: + # Truncate (not delete) — active loggers hold append-mode handles, so + # truncating to 0 clears content cleanly without losing the fd. + freed, n = 0, 0 + for f in Path(LOGS_DIR).glob("*.log"): + try: + freed += f.stat().st_size + open(f, "w").close() + n += 1 + except OSError: + pass + return n, freed + + +@router.post("/storage/clean") +async def storage_clean(req: _CleanReq): + """Clean a disposable category (recordings | records | logs) or 'all'. + Recordings/records are deleted; logs are truncated. Assets (faces, motions, + zones) are never touched here.""" + t = (req.target or "").strip().lower() + if t != "all" and t not in _CLEANABLE: + raise HTTPException(400, f"target must be 'all' or one of {sorted(_CLEANABLE)}") + + def _do(): + targets = ["recordings", "records", "logs"] if t == "all" else [t] + fns = {"recordings": _clean_recordings, "records": _clean_records, + "logs": _clean_logs} + result, total = {}, 0 + for tg in targets: + n, freed = fns[tg]() + result[tg] = {"items": n, "freed_bytes": freed, "freed_human": _human(freed)} + total += freed + log.info("storage clean %s → freed %s", targets, _human(total)) + return {"ok": True, "cleaned": targets, + "total_freed_bytes": total, "total_freed_human": _human(total), + "result": result} + return await asyncio.to_thread(_do) diff --git a/dashboard/routes/temp_monitor.py b/dashboard/routes/temp_monitor.py index 266e7b4..81505bb 100644 --- a/dashboard/routes/temp_monitor.py +++ b/dashboard/routes/temp_monitor.py @@ -65,3 +65,17 @@ async def motors_snapshot(): except Exception: positions = [] return build_payload(temps, positions, time.time()) + + +@router.get("/battery") +async def battery_status(): + """Live G1 battery (BMS) snapshot: state-of-charge %, voltage, current, + charge/discharge status, pack temperature, cycles. `available=False` until + the BMS topic (rt/lf/bmsstate) delivers its first message.""" + arm = _get_arm() + if arm is None or not hasattr(arm, "get_battery"): + return {"available": False} + try: + return arm.get_battery() + except Exception: + return {"available": False} diff --git a/dashboard/routes/zones.py b/dashboard/routes/zones.py index 395077b..d6def31 100644 --- a/dashboard/routes/zones.py +++ b/dashboard/routes/zones.py @@ -13,6 +13,7 @@ they just record the target and feed Gemini the place's reference. from __future__ import annotations +import asyncio import io from typing import Optional @@ -110,11 +111,59 @@ def _place_to_dict(p) -> dict: def _zone_to_dict(z) -> dict: return { "id": z.id, "name": z.name, "description": z.description, + "linked_map": getattr(z, "linked_map", None), "added_at": z.added_at, "places": [_place_to_dict(p) for p in z.places], } +async def _maybe_drive_to_place(zone, place) -> Optional[dict]: + """If the place links a nav2 place AND its zone's map is the one currently + localized, actually DRIVE there (arbiter-gated + arm arrival monitor). + Returns the drive outcome, or None when the place isn't drivable (no link). + Best-effort: never raises into the caller.""" + nav_place = getattr(place, "nav_place", None) + linked_map = getattr(zone, "linked_map", None) + if not nav_place or not linked_map: + return None + try: + from Project.Sanad.dashboard.routes import navigation as navmod + from Project.Sanad.dashboard.routes import _arbiter + except Exception: + return {"ok": False, "reason": "nav_unavailable"} + client = getattr(navmod, "_CLIENT", None) + if client is None: + return {"ok": False, "reason": "nav_unavailable"} + try: + st = await asyncio.to_thread(client.status) + body = st.as_dict() if hasattr(st, "as_dict") else dict(st) + except Exception as exc: # noqa: BLE001 + return {"ok": False, "reason": "status_error", "detail": str(exc)[:120]} + if not body.get("bringup_alive"): + return {"ok": False, "reason": "no_map"} + # The robot can only drive in the currently-localized map. Require the + # zone's linked map to match (compare on the sanitized .db stem). + active = (body.get("active_map") or "").strip().lower() + want = (linked_map or "").strip().lower() + if want.endswith(".db"): + want = want[:-3] + if active and want and active != want: + return {"ok": False, "reason": "wrong_map", + "active": body.get("active_map"), "want": linked_map} + if not _arbiter.acquire_nav(): + return {"ok": False, "reason": "manual_armed"} + drive = await asyncio.to_thread(client.goto, nav_place) + if isinstance(drive, dict) and not drive.get("ok", True): + _arbiter.release_nav() + return {"ok": False, "reason": "dispatch_failed", "detail": drive} + try: + from Project.Sanad.navigation.goal_monitor import arm_goal + arm_goal(nav_place) + except Exception: + pass + return {"ok": True, "resolved": nav_place} + + def _nav_target_dict(st, gallery) -> Optional[dict]: zid, pid = st.nav_target_zone_id, st.nav_target_place_id if not zid or not pid: @@ -184,6 +233,16 @@ class FacesPayload(BaseModel): face_ids: list[int] = [] +class LinkMapPayload(BaseModel): + # nav2 map .db basename (e.g. "office.db"); None/"" unlinks. + map: Optional[str] = None + + +class NavPlacePayload(BaseModel): + # nav2 place name in the zone's linked map; None/"" unlinks. + nav_place: Optional[str] = None + + @router.get("") async def list_zones(): g = _require_zones() @@ -247,6 +306,7 @@ async def create_place( name: Optional[str] = Query(default=None), description: Optional[str] = Query(default=None), face_ids: list[int] = Query(default=[]), + nav_place: Optional[str] = Query(default=None), files: Optional[list[UploadFile]] = File(default=None), ): g = _require_zones() @@ -262,11 +322,38 @@ async def create_place( _validate_image(content, f.filename) image_bytes.append(content) p = g.create_place(zone_id, name=name, description=description, - face_ids=face_ids, image_bytes_list=image_bytes or None) + face_ids=face_ids, image_bytes_list=image_bytes or None, + nav_place=nav_place) _bump_zones_version() return {"ok": True, "place": _place_to_dict(p)} +@router.post("/{zone_id}/link_map") +async def link_zone_map(zone_id: int, payload: LinkMapPayload): + """Bind (or unbind) the zone to a nav2 map .db. Required before its places + can link to that map's nav places / before Gemini Nav can drive in it.""" + g = _require_zones() + try: + g.set_zone_map(zone_id, payload.map) + except FileNotFoundError as exc: + raise HTTPException(404, str(exc)) + _bump_zones_version() + return {"ok": True, "zone": _zone_to_dict(g.get_zone(zone_id))} + + +@router.post("/{zone_id}/places/{place_id}/nav_link") +async def link_place_nav(zone_id: int, place_id: int, payload: NavPlacePayload): + """Link (or unlink) a place to a nav2 place name in the zone's map — this is + what makes the place drivable from voice / 'Go here'.""" + g = _require_zones() + try: + g.set_place_nav(zone_id, place_id, payload.nav_place) + except FileNotFoundError as exc: + raise HTTPException(404, str(exc)) + _bump_zones_version() + return {"ok": True, "place": _place_to_dict(g.get_place(zone_id, place_id))} + + @router.post("/{zone_id}/places/{place_id}/rename") async def rename_place(zone_id: int, place_id: int, payload: NamePayload): g = _require_zones() @@ -398,9 +485,13 @@ async def download_place_zip(zone_id: int, place_id: int): @router.post("/{zone_id}/places/{place_id}/go") async def go_to_place(zone_id: int, place_id: int): - """Set this place as the active destination. Records the target and lets - the Gemini child pick it up (reference photo + goal). Actual robot motion - is wired by N2 locomotion — until then this just establishes the goal.""" + """Set this place as the active destination AND, if the place links a nav2 + place in this zone's (currently-localized) map, actually drive there. + + Two effects: (1) records nav_target so the Gemini child primes on the + reference photo + announces the destination; (2) if drivable, dispatches a + Nav2 goal (arbiter-gated, with arrival monitoring). A place with no nav link + is announce-only, as before.""" g = _require_zones() p = g.get_place(zone_id, place_id) if p is None: @@ -410,8 +501,12 @@ async def go_to_place(zone_id: int, place_id: int): nav_target_place_id=place_id) log.info("nav target set → zone_%d/place_%d (%s)", zone_id, place_id, p.name or "(unnamed)") - return {"ok": True, "nav_target": {"zone_id": zone_id, "place_id": place_id, - "place_name": p.name}} + zone = g.get_zone(zone_id) + drive = await _maybe_drive_to_place(zone, p) + return {"ok": True, + "nav_target": {"zone_id": zone_id, "place_id": place_id, + "place_name": p.name}, + "drive": drive} @router.post("/nav/clear") @@ -419,3 +514,84 @@ async def clear_nav_target(): recognition_state.mutate(STATE_PATH, nav_target_zone_id=0, nav_target_place_id=0) log.info("nav target cleared") return {"ok": True, "nav_target": None} + + +def _resolve_map_path(client, linked_map: str) -> Optional[str]: + """Find the .db path for a zone's linked map name via the nav client.""" + want = (linked_map or "").strip().lower() + want_stem = want[:-3] if want.endswith(".db") else want + try: + maps = client.list_maps() or [] + except Exception: + return None + for m in maps: + nm = (m.get("name") or "").strip().lower() + if nm == want or (nm[:-3] if nm.endswith(".db") else nm) == want_stem: + return m.get("path") + return None + + +@router.post("/{zone_id}/gemini_nav/start") +async def gemini_nav_start(zone_id: int): + """Enter 'Gemini Nav' for a zone: localize the zone's map, turn on camera + + face + zone recognition + movement, ensure the Gemini session is live, and + greet the user so they can converse to navigate. + + The robot only ever runs ONE map; this loads the zone's map in localize-only + mode (so it cannot fresh-map while driving), exactly as the user requires. + """ + g = _require_zones() + zone = g.get_zone(zone_id) + if zone is None: + raise HTTPException(404, f"zone_{zone_id} not found") + linked_map = getattr(zone, "linked_map", None) + if not linked_map: + raise HTTPException(400, "This zone has no linked nav2 map — link one first.") + + # 1) Localize the zone's map (single bringup, mode 3 — no fresh mapping). + loaded: dict = {"ok": False, "reason": "nav_unavailable"} + try: + from Project.Sanad.dashboard.routes import navigation as navmod + client = getattr(navmod, "_CLIENT", None) + if client is not None: + db_path = await asyncio.to_thread(_resolve_map_path, client, linked_map) + if db_path: + loaded = await asyncio.to_thread(client.load_map, db_path) + else: + loaded = {"ok": False, "reason": "map_not_found", "map": linked_map} + except Exception as exc: # noqa: BLE001 + loaded = {"ok": False, "reason": "load_error", "detail": str(exc)[:160]} + + # 2) Camera + face + zone recognition + movement ON for the session. + recognition_state.mutate(STATE_PATH, + vision_enabled=True, face_rec_enabled=True, + zone_rec_enabled=True, movement_enabled=True) + _bump_zones_version() + + # 3) Ensure the Gemini session is live, then greet (zone + drivable places). + session_started = False + try: + from Project.Sanad.main import live_sub + if live_sub is not None: + if hasattr(live_sub, "is_running") and not live_sub.is_running(): + await asyncio.to_thread(live_sub.start) + session_started = True + drivable = [p.name or p.nav_place for p in zone.places + if getattr(p, "nav_place", None)] + zname = zone.name or f"zone {zone_id}" + if drivable: + placelist = ", ".join(str(x) for x in drivable) + greet = (f"You are now in the '{zname}' zone. You can drive the " + f"user to: {placelist}. Greet the user warmly in your " + f"normal Khaleeji style and ask where they would like to go.") + else: + greet = (f"You are now in the '{zname}' zone, but no drivable " + f"places are linked to its map yet. Greet the user and " + f"say places still need to be linked before you can drive.") + if hasattr(live_sub, "send_state"): + live_sub.send_state("nav_zone", greet) + except Exception as exc: # noqa: BLE001 + log.warning("gemini_nav greet failed: %s", exc) + + return {"ok": True, "zone_id": zone_id, "zone": _zone_to_dict(zone), + "loaded": loaded, "session_started": session_started} diff --git a/dashboard/static/index.html b/dashboard/static/index.html index 754fac6..50dac2c 100644 --- a/dashboard/static/index.html +++ b/dashboard/static/index.html @@ -109,6 +109,124 @@ .switch input:checked+.slider::before{transform:translateX(16px)} /* Scrollbar */ ::-webkit-scrollbar{width:6px} ::-webkit-scrollbar-track{background:transparent} ::-webkit-scrollbar-thumb{background:var(--dim);border-radius:3px} ::-webkit-scrollbar-thumb:hover{background:var(--muted)} + /* WhatsApp-style voice-message record cards */ + .rec-card{display:flex;flex-direction:column;gap:.45rem;padding:.6rem .75rem;margin-bottom:.55rem;background:rgba(255,255,255,.025);border:1px solid var(--border);border-radius:.7rem;transition:background .15s,border-color .15s} + .rec-card.is-playing{background:rgba(52,211,153,.07);border-color:rgba(52,211,153,.4)} + .rec-card .rec-row{display:flex;align-items:center;gap:.6rem} + .rec-sel{width:16px;height:16px;flex:0 0 auto;cursor:pointer} + .rec-play{flex:0 0 auto;width:38px;height:38px;border-radius:50%;border:none;background:var(--accent);color:#04121f;font-size:1rem;display:flex;align-items:center;justify-content:center;cursor:pointer;transition:transform .1s,background .15s} + .rec-play:hover{transform:scale(1.06)} + .rec-card.is-playing .rec-play{background:#34d399} + .rec-replay{flex:0 0 auto;width:26px;height:26px;border-radius:50%;border:1px solid var(--border);background:transparent;color:var(--muted);font-size:.95rem;line-height:1;display:flex;align-items:center;justify-content:center;cursor:pointer;transition:color .12s,border-color .12s} + .rec-replay:hover{color:#34d399;border-color:#34d399} + .rec-wave{flex:1;min-width:70px;display:flex;align-items:center;gap:2px;height:30px;overflow:hidden;cursor:pointer;touch-action:none} + .rec-wave .wb{flex:1 1 0;min-width:2px;max-width:5px;border-radius:2px;background:rgba(255,255,255,.16);transition:background .12s,height .12s;pointer-events:none} + .rec-wave .wb.played{background:#34d399} + .rec-time{flex:0 0 auto;font-size:.7rem;color:var(--dim);font-variant-numeric:tabular-nums;min-width:74px;text-align:right} + .rec-card.is-playing .rec-time{color:#34d399} + .rec-name{font-size:.74rem;font-weight:600;color:var(--muted)} + .rec-text{font-size:.8rem;line-height:1.45;color:var(--fg,#dfe7ef);white-space:pre-wrap;word-break:break-word} + .rec-acts{flex:0 0 auto;display:flex;gap:.3rem;align-items:center} + /* Battery widget (Temperature tab) */ + .batt-icon{position:relative;width:56px;height:27px;border:2px solid var(--muted);border-radius:5px;padding:2px;flex:0 0 auto} + .batt-icon::after{content:'';position:absolute;right:-6px;top:7px;width:4px;height:9px;background:var(--muted);border-radius:0 2px 2px 0} + .batt-fill{height:100%;width:0%;border-radius:2px;background:var(--success);transition:width .45s ease,background .3s} + .batt-fill.charging{animation:battpulse 1.3s ease-in-out infinite} + @keyframes battpulse{0%,100%{opacity:1}50%{opacity:.55}} + /* ── Tablet (single column already kicks in at 900px for .grid) ── */ + @media (max-width:768px){ + header{padding:.6rem .9rem} + .tab-content{padding:.85rem .85rem} + .status-pills{padding:.3rem .9rem} + .tabs{padding:0 .8rem} + } + /* ── Phone view ──────────────────────────────────────────────── */ + @media (max-width:640px){ + /* Header wraps gracefully; controls drop to a second line if needed */ + header{flex-wrap:wrap;gap:.4rem;padding:.5rem .7rem} + header h1{font-size:1.05rem} + .hdr-right{gap:.4rem;font-size:.68rem;flex-wrap:wrap;justify-content:flex-end;width:auto} + #estop{padding:.28rem .6rem;font-size:.66rem} + .hdr-badge{font-size:.6rem;padding:2px 5px} + /* Reclaim horizontal space on the narrow viewport */ + .status-pills{padding:.3rem .55rem;gap:.28rem} + .tabs{padding:0 .45rem;-webkit-overflow-scrolling:touch} + .tab{padding:.5rem .65rem;font-size:.74rem} + .tab-content{padding:.7rem .55rem} + .grid{gap:.55rem} + .card{padding:.8rem .7rem} + .card-full,.card{min-width:0} + /* Wide tables scroll sideways instead of stretching the page */ + .card table{display:block;max-width:100%;overflow-x:auto;white-space:nowrap;-webkit-overflow-scrolling:touch} + /* Generic control rows wrap rather than overflow */ + .row{flex-wrap:wrap} + .toast{max-width:calc(100vw - 24px)} + /* Voice-message record cards: larger touch targets, acts wrap below */ + .rec-card{padding:.55rem .6rem} + .rec-card .rec-row{flex-wrap:wrap;gap:.45rem} + .rec-play{width:44px;height:44px;font-size:1.15rem} + .rec-replay{width:34px;height:34px;font-size:1.05rem} + .rec-wave{flex:1 1 110px;min-width:60px;height:34px} + .rec-time{min-width:0;font-size:.72rem;flex:0 0 auto} + .rec-acts{margin-left:auto} + .rec-text{font-size:.82rem} + } + /* ── Small phone ─────────────────────────────────────────────── */ + @media (max-width:380px){ + header h1{font-size:.95rem} + .tab{padding:.45rem .55rem;font-size:.72rem} + .rec-card .rec-sel{order:-1} + .rec-wave{flex:1 1 100%;order:5} + .rec-time{order:4} + } + /* ── Mobile audit pass 2 — per-tab phone fixes ───────────────── */ + @media (max-width:768px){ + #livemapFrame{height:62vh} #temp3d-frame{height:60vh} #medWrap{max-height:60vh} + } + @media (max-width:640px){ + /* Operations · Audio Control: fixed-width mute buttons + 5-item profile row */ + .mute-btn{min-width:unset;width:100%;display:block} + #audio-profile{flex:1 1 100%} #audio-profile~.btn{flex:1 1 auto} + #action-speed{width:auto;flex:0 0 auto} + /* Voice tab: nested gaps, packed subprocess row, key-input labels */ + #tab-voice .row[style*="gap:1.2rem"]{gap:.6rem} + #tab-voice .btn.btn-sm.btn-ghost{min-width:unset;flex:0 1 auto} + #tab-voice label[style*="min-width:70px"]{min-width:unset;display:block;margin-bottom:.3rem} + /* Motion & Replay: two-column min-width panes + fixed-width selects */ + #tab-motion .card > div[style*="min-width:260px"]{min-width:0} + #tab-motion #action-speed-2,#tab-motion #replay-speed{width:55px} + #tab-motion #combo-speed{width:60px} + #tab-motion #teach-duration{width:auto;flex:0 0 45px} + /* Navigation control bar → stack vertically, hide meta spans */ + #tab-navigation .card-full:has(#navMapSelect) > div:first-child{flex-direction:column;align-items:stretch} + #tab-navigation .card-full:has(#navMapSelect) > div:first-child > h3, + #tab-navigation .card-full:has(#navMapSelect) > div:first-child > select, + #tab-navigation .card-full:has(#navMapSelect) > div:first-child > .action-btn{width:100%} + #tab-navigation .card-full:has(#navMapSelect) > div:first-child > span{display:none} + /* Map Editor control bar → stack */ + #tab-mapeditor .card-full > div:first-child{flex-direction:column;align-items:stretch} + #tab-mapeditor .card-full .action-btn{flex:1 1 auto;min-width:auto} + #medMapSelect,#medBrush{width:100%} + /* Embeds shrink so they don't eat the phone viewport (but the Temperature + 3D viewer IS the tab's main content → keep it tall enough for the model + + its compact overlay panels). */ + #livemapFrame{height:50vh} #temp3d-frame{height:72vh;min-height:360px} #medWrap{max-height:40vh} + /* Recognition gallery: smaller tiles */ + .gallery-grid{grid-template-columns:repeat(auto-fill,minmax(70px,1fr));gap:.3rem} + .gallery-grid img{height:60px} + /* Mask: slightly larger color swatches (touch) + flexible number inputs */ + #tab-mask input[type="color"]{width:50px} #mask-bright-val{width:auto} + #mask-img-id,#mask-anim-id{width:50px} + /* Temperature battery widget gaps */ + #battery-card .row{gap:.6rem} + /* Terminal: wrap header, shorter min-height */ + #tab-terminal .row{flex-wrap:wrap} #tab-terminal h3{font-size:.85rem} #tab-terminal .card{min-height:240px} + /* Recordings bulk-delete label fits */ + #rec-del-selected{white-space:nowrap;font-size:.65rem;padding:.2rem .35rem} + /* Settings: shorter log box + compact header buttons */ + #log-box{height:150px} + #tab-settings .card-full > .row button{font-size:.65rem;padding:.2rem .3rem;white-space:nowrap} + } @@ -142,7 +260,11 @@
Voice & Audio
Motion & Replay
Controller
+
Navigation
+
Live Map
+
Map Editor
Recognition
+
Mask Face
Recordings
Temperature
Terminal
@@ -153,25 +275,14 @@
- -
-

Quick Voice

-
- -
- - - -
-
-
-

System Info

Loading...
+ +
Network interfaces
@@ -191,7 +302,7 @@
- + –
@@ -203,7 +314,7 @@
- Persisted in data/motions/config.json · applies live via DDS + Controls the ACTIVE speaker — G1 chest (DDS) + the selected PulseAudio sink (JBL/Anker). Applies live.
@@ -295,7 +406,9 @@ + + Pause: Auto
@@ -628,6 +741,26 @@
+ +
+

🔋 Battery

+
+
+
+
+
--%
+
--
+
+
+
+
Voltage
--
+
Current
--
+
Pack temp
--
+
Cycles
--
+
+
+
Reading battery…
+
+
+
+ Full web_nav3 dashboard (live map, set-pose, manual goals, missions) embedded from the robot. + Also available standalone at + :8765. + If it stays blank, check that bringup + rosbridge are alive (see the Navigation tab) and that the + robot is reachable on the network. +
+ +
+ + +
+
+
+

MAP EDITOR

+ + + map: — + MODE — + + Tool: + + + + Brush + + + + + + + + + load a map… +
+
+ +
+
+ Edit a SAVED map. Pick a map → Load & Edit. 🧽 Erase removes black phantom obstacles (paints them free); ⬛ Wall paints virtual walls / keep-outs. Click-drag to paint (brush size above). Save stores the edits per-map and applies them to the robot's navigation — it stops avoiding erased points and treats painted walls as keep-outs. Yellow = your edits. The original map .db is never modified. +
+
+
+ + +
+ + +
+
+
+ + unknown + FACE — + SPEAK — +
+
+ + + +
+
+
+ LED face mask over Bluetooth. Check Link Gemini (below) to connect it + let Gemini show emotions; leave it off and the mask stays idle (no reconnecting). + Once linked it self-heals dropped links — keep the mask near the Jetson and free it from the phone app first. Faces upload once (~25 s) then animate via PLAY. +
+
+ +
+ +
+

Animated Face

+
+ + + + +
+
+ + + +
+
+ Mouth + + 0 +
+
+ + + + + + + + + + + + + + + + +
+
+ Face colors + + + + +
+
Run face → it blinks/glances on its own and lip-syncs while Gemini speaks. Colors re-upload the frames (~30–60s). Auto-reconnect keeps the face alive through BLE drops.
+
+ + +
+

Brightness & Text

+
+ Brightness + + 95 +
+
Keep ≤100 to avoid LED flicker (battery-limited).
+
+ + + + +
+
+ + Speed + + +
+
Text overrides the animated face until you Run face / ↩ Face again.
+
+ + +
+

Social / QR on Face

+
+ + + + +
+
Social buttons show a scannable QR (short da.gd link → Instagram). Full-URL / dense QRs show full-screen but only scan if short (use QR from link below).
+ +
+ Saved QR codes + + + +
+
+ QR from link + + + +
+
+
+ + +
+

Saved Text / Words

+
+ + + +
+
Save words/phrases, then scroll any of them across the mask on demand.
+
+
+ + +
+

Built-in Images / Animations

+
+ Image + + + + +
+
+ Anim + + + + +
+
+ +
+
Built-in IMAG ids ~0–105, ANIM ids ~0–69 (values above range show garbled frames).
+
+
+
+
@@ -754,15 +1194,25 @@

Saved Records

- - + + + +