Live Gemini could not use an external speaker or headset. An AudioContext
is bound to whichever output was default when it was created, and
getUserMedia({audio:true}) takes the system default input, so plugging a
device in afterwards left audio going to the old one — silently, with no
error to explain it.
* Explicit Mic and Speaker pickers. Capture opens the chosen deviceId
exactly and can be switched mid-session; playback is routed through a
MediaStreamAudioDestinationNode into a hidden <audio> element so
setSinkId() can move it to the chosen sink. Both lists refresh on
devicechange and are remembered in localStorage.
* Windows reports each device three times (default, communications, and
the real one), so an Anker would have appeared three times with no way
to tell them apart. The pseudo-devices are now collapsed.
* Capture moved to an AudioWorklet (Blob-built, no extra file served)
with the ScriptProcessor kept as a fallback.
* A compatibility line reports what the browser actually supports, and
the diagnostics line now shows the output sink and capture kind — the
difference between "not listening" and "not speaking" without a
debugger.
Verified in both engines with Playwright (chromium PASS, firefox PASS: no
page errors, personas and device lists populated, test tone plays) and
against real hardware, where setSinkId matched the selection and a named
microphone opened by deviceId. Two engine-specific bugs fell out of that
run and are fixed here: reading AudioContext.prototype.audioWorklet invokes
the getter and throws in both browsers, which aborted init and left the tab
empty, and the init steps are now isolated so one failure cannot take the
rest down.
Also in this commit:
* README rewritten against what the code does today — the voice-fidelity
rationale and its three gates, why words used to cut off, the Live tab
and why the browser talks to Google directly, personas, device
selection, recordings search/filter, sign-in history, a current API
list, the cPanel production setup, and the known limits.
* shell_scripts/start_cpanel.sh — the launcher that actually keeps the
site up (HTTP health check, not a TCP probe) was only on the host.
* data/live_personas.json — the persona library (G1, R1, Agibot, T800)
existed only on the server; it is user-written content worth keeping.
* .gitignore covers runtime state (logins, generated WAVs, .env, backups).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Replay now matches the robots and no longer cuts words:
- read the turn to turnComplete, not generationComplete, and drain the
socket before each send; breaking early truncated every sentence and
left frames that the next turn mis-read as its own reply
- accept a take only if the model's own transcript covers the text AND
the audio is long enough to contain it (the transcript reports the
full text even for a 0.8s clip)
- pitch gate: reject an off-tone take and re-ask, per voice, using a
pure-Python F0 estimator (no numpy on the host)
- continuation: speak the words a voice skipped instead of retrying a
line it stops on deterministically
- fresh Live session per replay; delivery drifts as turns accumulate
Live Gemini tab: browser talks to Gemini directly (the reverse proxy
cannot upgrade a WebSocket), with a persona library - named personas,
per-robot selection, built-ins that cannot be overwritten.
Dashboard: records search + voice filter, log panel falls back to
polling, sign-in history with CSV/JSON export, and JS errors now show
on the page instead of silently blanking a tab.
