Sanad_lite

A small FastAPI dashboard that speaks in the exact voices of the robots — Unitree G1, Unitree R1 and Agibot x2 — and lets you hold a live spoken conversation with any of them from a browser.

It is a stripped fork of Sanad: the arm, macros, camera, wake words and on-robot audio are gone. What remains is text → speech, saved recordings, and a realtime voice tab — with all audio in the user's own browser, never on the host.

Live at https://sanadlite.yslootahtech.com (cPanel host, see Production).

┌──────────────────────────────────────────────────────────────────────┐
│  Dashboard (FastAPI)                                                 │
│  ├─ /login              Cookie session + sign-in history             │
│  ├─ Voice & Audio       Typed Replay — type text, hear the robot     │
│  ├─ Live Gemini         Talk to a robot voice in realtime            │
│  ├─ Recordings          Saved WAVs: search, filter by robot, play    │
│  └─ Settings & Logs     Persona/rule files, log tail, sign-in log    │
└──────────────────────────────────────────────────────────────────────┘

Robot	Gemini voice	Character
Unitree G1	Charon	Sanad / Bousandah — male, Emirati
Unitree R1	Puck	Super Dubai — male, Emirati
Agibot x2	Kore	Muza — female, Emirati

The one thing this project is about

The site must sound identical to the robots. Sanadv3 (G1), SanadR1 (R1) and the Agibot stack each open a Gemini Live session and speak the reply audio. If the dashboard renders the same sentence through a different engine — even one that offers the same voice name — it does not sound like the robot. Measured: the REST TTS model's "Charon" sits roughly 50 Hz above the Live one.

So typed replay uses the robots' own path and refuses to quietly downgrade:

Gemini Live (same model, voice and prompt as the robot)   ← up to 6 attempts
   └─ each take is verified before it is served
REST TTS (gemini-2.5-flash-preview-tts)                   ← only if Live
                                                             returned NO audio

Every Live take is checked three ways before it is accepted (voice/typed_replay.py):

Gate	What it catches	Default
Coverage — output transcript vs the text you typed	a take that stops mid-sentence	1.0 (all of it)
Believable duration — chars ÷ seconds	audio too short to contain the words	14 chars/s Arabic, 20 Latin
Pitch band — median F0 of the take	the wrong voice / wrong engine	Charon & Puck 100–140 Hz, Kore 165–225 Hz

A take that fails is retried. If no attempt passes, the fullest Live take is served — the robot's voice with a missing word beats a clean read in the wrong voice — and the UI shows a warning naming the gate that failed. If a take is complete but ends early, up to 3 continuations are requested and joined.

Why words used to cut off

The Live socket sends generationComplete before the last audio frames have arrived; turnComplete is the real end of turn. Breaking on the first one truncated tails and left stale frames in the socket for the next request to pick up. gemini/client.py now breaks only on turnComplete, keeps a 3.0 s post-generation grace window, and drains the socket before closing.

The robots (Sanadv3 / SanadR1 / Sanad_Package_4) still carry the original generationComplete behaviour. Porting this fix to them has not been done.

Why every replay opens a fresh session

fresh_session_per_replay: true in config/voice_config.json. A warm session carries conversational context and the model drifts — same text, same voice, different tone and a weaker Emirati accent. One session per replay removed the drift. Voices that should keep a warm session can be listed in warm_session_voices.

voice/pitch.py is a dependency-free median-F0 estimator (autocorrelation over 48 frames, decimated to 8 kHz) — there is no numpy on the cPanel host. It was validated to within ~4 Hz of a numpy reference, ~48 ms worst case.

Quick start (laptop)

pip install --user fastapi 'uvicorn[standard]' itsdangerous python-multipart \
                   pydantic websockets

cd Project/Other/Sanad_lite
SANAD_DASHBOARD_HOST=127.0.0.1 python3 main.py

Open http://127.0.0.1:8000. Credentials live in config/core_config.json under auth — see Security below.

Omit SANAD_DASHBOARD_HOST to auto-bind to wlan0's IP so colleagues on the LAN can reach http://<your-ip>:8000.

A Gemini API key is required and none ships with the repo. gemini_defaults.api_key in config/core_config.json is deliberately empty. Supply one by:

• pasting it in Voice & Audio → Gemini API Key (hot-swap, no restart), or
• export SANAD_GEMINI_API_KEY=... before starting, or
• filling gemini_defaults.api_key.

Get one at https://aistudio.google.com/apikey. Keys beginning AQ. are AI Studio express keys — they work, with one consequence for the Live tab (below).

pyaudio, torch and transformers appear in requirements.txt but are not needed; they are leftovers from full Sanad and their import failures are swallowed at startup.

Where the audio actually plays

Two different paths, and the distinction matters:

Feature	Audio generated by	Audio played by
Typed Replay → Generate & Play	server (Gemini Live → WAV)	the viewer's browser
Typed Replay → Replay Last	cached WAV on the server	the viewer's browser
Recordings → Play / Raw / Download	server file	the viewer's browser
Live Gemini	Google, direct to the browser	the viewer's browser

Server-side ALSA/PulseAudio is never touched. If you host on machine A and a colleague opens http://A:8000 from machine B, the sound comes out of B's speakers.

Live Gemini tab

Pick a robot, press Connect, and talk. Speech goes to Gemini and the reply comes back in that robot's voice, in realtime — the same interaction as Sanad Package 5 on the G1, but in a browser.

Why the browser talks to Google directly

This deployment sits behind an Apache [P] rewrite that cannot upgrade a WebSocket — measured: the same handshake answers 101 straight to uvicorn and 404 through the proxy. A server-side relay is therefore impossible here, so the page opens its own socket to Gemini.

The intended way to do that is an ephemeral auth token minted by /api/live/token, so the real key never reaches the page. That path is implemented (v1beta + access_token) but Google refuses tokens minted from AQ. express keys — verified in all four documented forms. With such a key, set live.allow_direct_key: true in config/voice_config.json and the server hands the key itself to the browser (v1alpha + key).

While allow_direct_key is on, anyone who can sign in to the dashboard can read the Gemini API key. It is on in this deployment, deliberately. Turn it off the moment the key becomes a standard AIza... key.

Personas

A persona library, not a single editable prompt:

• 3 built-ins in code (dashboard/routes/live_personas.py) — one per robot.
• Any number of saved personas in data/live_personas.json: Save, Save as new, Delete, and Use for this robot to bind one to a voice.
• The active persona is shown above the picker and persists across restarts.
• Shipped saved personas: unitree-g1, unitree-r1, agibot-x2, plus t800 — an English-speaking persona written for the G1 voice.

Microphone and speaker selection

Live audio broke on external speakers and headsets for a structural reason: an AudioContext is bound to whichever output was default when it was created, and getUserMedia({audio:true}) takes the system default input. Plug a device in afterwards and audio keeps going to the old one — silently, with no error.

The tab now has explicit Mic and Speaker pickers:

• Capture opens the chosen deviceId exactly, and can be switched mid-session without dropping the connection.
• Playback is routed through a MediaStreamAudioDestinationNode into a hidden <audio> element so setSinkId() can move it to the chosen speaker.
• Both lists refresh on devicechange, and Windows' duplicate default / communications pseudo-devices are collapsed so each device appears once.
• Allow microphone re-requests permission if it was dismissed or denied; Test speaker plays a tone so output is proven before you spend credits.
• Choices are remembered in localStorage.

Capture uses an AudioWorklet (off the main thread; built from a Blob, so no extra file has to be served) and falls back to ScriptProcessorNode where it is unavailable. A compatibility line under the controls reports what the current browser actually supports, and a diagnostics line shows sent / peak / received / played / mic ctx / out ctx / out sink / capture kind — enough to tell "not listening" from "not speaking" without a debugger.

Browser support

Verified end to end with Playwright in both engines (chromium and firefox): tab renders, 3 robots, model loaded, persona library populated, device lists populated, test tone plays, no page errors. Also verified against real hardware: setSinkId matched the selection, and a named microphone opened by deviceId.

Caveats that are not code bugs:

• Bluetooth speakerphones switch from A2DP to HFP when a mic opens; the device may briefly disappear and return as a different endpoint. Prefer USB.
• Browser echo cancellation is weakest when output is routed to a non-default sink. A speakerphone with hardware AEC (e.g. Anker PowerConf) handles it; a plain speaker may echo — lower the volume or use the system default output.

Recordings

Saved WAVs from Save Last, with:

• a search box matching name or the original text. Arabic is folded before matching (diacritics stripped, alef forms unified), so a search for سلام also matches السلام and سَلام;
• a filter by robot — G1 / R1 / Agibot — built from the voice recorded with each entry;
• a count line (Showing N of M), Clear, and per-record Play / Raw / Download / Rename / Delete, plus Delete All.

Each record stores voice, voice_label, pitch_hz and any voice_warning from the gates, so an old file can be traced back to how it was produced.

Settings & Logs

• Persona and rule script editor (scripts/*.txt).
• Live log tail. The WebSocket (/ws/logs) cannot cross the production proxy, so the UI falls back to cursor-based polling of /api/logs/live.
• Sign-in history — who signed in, when, from which device and IP (data/logins.json; the device string is parsed from the user agent, and the IP prefers cf-connecting-ip because the site is behind Cloudflare). Exportable as CSV (UTF-8 with BOM, so Excel renders Arabic) or JSON.

Configuration

Environment variables

Var	Default	Effect
SANAD_DASHBOARD_HOST	wlan0's IP	Bind address. 127.0.0.1 for localhost only.
SANAD_DASHBOARD_INTERFACE	wlan0	Which interface to take the IP from.
SANAD_GEMINI_API_KEY	(empty)	Gemini API key.
PORT	8000	Listen port.

Keys worth knowing

config/core_config.json → gemini_defaults

Key	Meaning
model_live	Live model — gemini-2.5-flash-native-audio-preview-12-2025.
voice_name	Default voice (Charon).
voice_options	The robot ↔ voice table used by both tabs.
default_system_prompt	Emirati persona used for TTS.
voice_system_prompts	Per-voice overrides. Charon and Puck are deliberately absent so they keep the robots' verbatim prompt — that is what makes the site match Sanadv3/SanadR1.
auth	Dashboard username/password (cleartext).

config/voice_config.json

Key	Meaning
typed_replay.fresh_session_per_replay	One Live session per replay (tone stability).
typed_replay.warm_session_voices	Voices exempt from that.
live.allow_direct_key	Hand the API key to the browser for the Live tab.

config/gemini_config.json → client

Key	Meaning
post_generation_grace_sec	Wait after generationComplete for trailing audio (3.0 — 1.2 was measurably too short and the cuts came back).
output_transcription	Ask for the spoken transcript, which the coverage gate needs.
tts_preamble, tts_language_code	Emirati steering for the REST fallback (ar-AE).

Gate thresholds (live_attempts, min_spoken_coverage, max_chars_per_sec_ar, max_chars_per_sec_latin, max_continuations, pitch_gate, pitch_bands, short_text_chars) all live under voice.typed_replay and fall back to the defaults in voice/typed_replay.py.

API

GET    /                                  dashboard SPA (auth-gated, no-store)
GET    /login                             login page
POST   /api/auth/login                    {username,password} -> session cookie
POST   /api/auth/logout
GET    /api/auth/me
GET    /api/auth/logins?limit=20          sign-in history
GET    /api/auth/logins/export?format=csv|json

GET    /api/health                        {status, brain}
GET    /api/status                        brain + voice
GET    /api/system/info                   host / interfaces / subsystems

GET    /api/voice/status                  Gemini connection state
POST   /api/voice/connect | /disconnect
GET    /api/voice/api-key                 masked
POST   /api/voice/api-key                 {key} -> persisted, hot-swapped
GET    /api/voice/voices                  robot <-> voice table
POST   /api/voice/voice                   {voice} -> switch (reconnects)
POST   /api/voice/generate                legacy offline TTS route

POST   /api/typed-replay/say              {text,record,record_name}
GET    /api/typed-replay/audio/last       streams the cached WAV
POST   /api/typed-replay/replay-last
POST   /api/typed-replay/save-last
GET    /api/typed-replay/status           engine, session, voice, pitch, warning
GET    /api/typed-replay/records
GET    /api/typed-replay/records/{name}
POST   /api/typed-replay/records/{name}/rename
DELETE /api/typed-replay/records/{name}

GET    /api/records/                      saved records + voice metadata
GET    /api/records/audio/{name}?kind=speaker|raw
POST   /api/records/rename | /delete | /delete-all

GET    /api/live/config                   voices, model, active personas
GET    /api/live/personas
POST   /api/live/personas                 save / save-as-new
POST   /api/live/personas/select          bind a persona to a voice
POST   /api/live/personas/delete
POST   /api/live/token                    {voice} -> token + api_version + auth_param

GET    /api/scripts/                      persona/rule files
POST   /api/scripts/load | /save | /create | /rename | /delete
GET    /api/prompt/                       resolved system prompt
POST   /api/prompt/update | /reload
GET    /api/prompt/rule                   the Voice Rule text
POST   /api/prompt/rule

GET    /api/logs/                         available logs
GET    /api/logs/live?cursor=&limit=      polling tail (proxy-safe)
GET    /api/logs/tail/{filename}?lines=
POST   /api/logs/snapshot
GET    /api/logs/bundle                   zip of all logs
GET    /api/audio/*                       server-side device info (informational)
WS     /ws/logs                           live log stream (LAN only — see above)

Layout

Path	Contents
main.py	Entry point — boots subsystems, then the dashboard.
config.py	Runtime constants derived from config/*.json.
config/	core, voice, gemini, dashboard JSON.
core/	Brain (callback whitelist + status), skill registry, event bus, config loader, logger, asyncio compat.
gemini/client.py	Gemini Live WebSocket client + REST TTS fallback + quota detection.
voice/typed_replay.py	Generation, the three gates, continuations, records.
voice/pitch.py	Dependency-free median-F0 estimator.
voice/ (rest)	audio_manager.py, audio_devices.py, local_tts.py, text_utils.py — vestigial on this deployment, degrade silently.
dashboard/app.py	FastAPI app, session middleware, auth gate, route registry, cache headers.
dashboard/routes/	auth, health, system, voice, logs, audio_control, scripts, records, prompt, typed_replay, live, live_personas.
dashboard/static/	index.html (SPA), live.js (browser Gemini client), login.html.
dashboard/websockets/log_stream.py	Log ring buffer + WS endpoint + recent_since() for polling.
scripts/	sanad_script.txt (persona), sanad_rule.txt (voice rule).
data/audio/	Saved WAVs + records.json.
data/live_personas.json	Persona library.
data/logins.json	Sign-in history (runtime, gitignored).
shell_scripts/start_cpanel.sh	Production launcher / self-heal (below).
tests/test_smoke.py	Brain whitelist, skill registry, atomic IO, audio devices, isolation.

Production (cPanel)

The host cannot run a normal service manager, so:

• uvicorn runs on 127.0.0.1:8000 from the CloudLinux virtualenv (~/virtualenv/sanadlite/3.8), started by shell_scripts/start_cpanel.sh.
• Apache reverse-proxies https://sanadlite.yslootahtech.com to it. A [P] rewrite cannot upgrade WebSockets — hence log polling and the browser-direct Live socket.
• cron every 2 minutes re-runs the launcher. It is idempotent and does a real HTTP health check, not a TCP probe: a wedged uvicorn with a frozen asyncio loop still holds the port and answers a TCP connect, so a connect-only check once reported "up" while the site was down for ~2 days. If the port is bound but /api/health does not answer, the frozen process is killed and relaunched.
• Environment variables set in cPanel's Setup Python App form are stored in ~/.cl.selector/python-selector.json and are not re-exported by the venv's activate. The launcher parses and exports them itself — that is the store the Gemini API key must go into for a restart to keep it.
• passenger_wsgi.py is the alternative Passenger/WSGI entry point. It works, but WSGI has no WebSockets at all; the uvicorn path above is what runs.

Deployment is scp of changed files plus a launcher restart. Static assets are served with a 7-day max-age and the page itself with no-store, so index.html references live.js?v=<mtime> — bump that query string when you change a static file, or browsers will keep the old one.

Security

• config/core_config.json → auth holds the dashboard username and password in cleartext, and that file is committed. Change both before any deployment you do not control, and consider replacing the check in dashboard/routes/auth.py with a bcrypt hash.
• The session cookie is signed with a secret regenerated at every start, so a restart signs everyone out.
• live.allow_direct_key exposes the Gemini API key to any signed-in user.
• No API key is committed; gemini_defaults.api_key ships empty.

Troubleshooting

Symptom	Cause / fix
Replay sounds like a different voice	The REST fallback ran — Live returned no audio at all. Check logs/ for served REST TTS. Usually quota, or a very short input.
"NOT the robot voice — … Hz outside …" warning	The pitch gate caught an off-voice take. Generate again; if it persists, the Live path is failing and everything is coming from REST.
Words cut off at the end	Should be fixed (post_generation_grace_sec: 3.0). If it returns, raise that value before touching anything else.
Tone/accent drifts between generations	A warm session leaked context — confirm fresh_session_per_replay is true and the voice is not in warm_session_voices.
Everything 503s with a quota message	GeminiQuotaExhausted — the key is out of credits. It fails fast and says so rather than retrying six times.
Live tab connects but never replies	Read the diagnostics line. peak near 0 = the mic is not being heard (wrong input device, or muted). received climbing but played flat = output routing.
Live tab: token minted but connect fails	An AQ. key with allow_direct_key: false. Ephemeral tokens do not work for those keys.
A UI fix "did not deploy"	Browser cache. Bump the ?v= on the changed asset in index.html.
Arabic in the CSV export looks like mojibake	Open it through Excel's import dialog as UTF-8; the file already carries a BOM.
ModuleNotFoundError: itsdangerous / websockets	pip install itsdangerous websockets — required by SessionMiddleware and gemini/client.py.
Failed to construct audio_mgr — pyaudio not installed	Harmless. Nothing user-facing needs PyAudio.
Redirected to /login on every call	The server restarted and the cookie secret changed. Sign in again.

Known limits

• Accent quality is not measured. Pitch and coverage are verified automatically; how Emirati a take sounds is not. That needs a blind listening test.
• Typed replay is single-user. The session, the selected voice and audio/last are process globals — two people generating at once will interfere.
• Very short inputs (a single word) can still fall through to the REST engine, because the Live model sometimes returns text instead of audio for them.
• The generationComplete truncation fix has not been ported to the robots.

Attribution

Internal project for YS Lootah Robotics. Trimmed from Sanad; the original reuses patterns from SanadVoice/gemini_interact and Unitree unitree_sdk2py.