Update 2026-04-21 10:34:53
This commit is contained in:
parent
f9ad4aedd5
commit
0df2f2eb11
57
README.md
57
README.md
@ -218,62 +218,5 @@ The reliable path is **serial**: `speak()` → `wait_for_audio_done()` →
|
|||||||
want). Parallel is kept behind `audio_lead_s` for latency-sensitive
|
want). Parallel is kept behind `audio_lead_s` for latency-sensitive
|
||||||
demos where occasional dropped audio is acceptable.
|
demos where occasional dropped audio is acceptable.
|
||||||
|
|
||||||
## Current status (2026-04-21)
|
|
||||||
|
|
||||||
Snapshot of what is and isn't working, so whoever picks this up next
|
|
||||||
doesn't have to rediscover it.
|
|
||||||
|
|
||||||
**Working:**
|
|
||||||
|
|
||||||
- End-to-end flow: RealSense → YOLO11n → tracker → compliance →
|
|
||||||
UNSAFE/SAFE/PARTIAL events → robot voice + recorded arm motion.
|
|
||||||
- Wireless-remote gated start/stop (R2+X / R2+Y) via `rt/lowstate`.
|
|
||||||
- Recorded arm replay from [assets/motions/adnoc1.jsonl](assets/motions/)
|
|
||||||
via [robot/arm_replay.py](robot/arm_replay.py) — custom teach-and-replay
|
|
||||||
trajectory instead of the canned `reject` action.
|
|
||||||
- Recorded WAV library under [assets/audio/](assets/audio/) with a
|
|
||||||
`recorded_or_tts` mode that can be toggled via `tts.mode` in config.
|
|
||||||
- TtsMaker path hardened: `AUDIO_STOP_PLAY` reset + retry + 3 s RPC
|
|
||||||
timeout + index-bug workaround (see [Audio pipeline](#audio-pipeline)).
|
|
||||||
- MJPEG stream on port 8080 for remote visual QA.
|
|
||||||
- Fully config-driven — no hard-coded IPs, paths, or thresholds in code
|
|
||||||
beyond `SAQR_ROOT` fallback and firmware API ids.
|
|
||||||
- systemd unit + `start_saqr.sh` = single entry point, survives reboots.
|
|
||||||
|
|
||||||
**Known broken / in tension:**
|
|
||||||
|
|
||||||
- **Audio ↔ arm firmware contention** (see [Known limitations](#known-limitations)).
|
|
||||||
Current default is parallel via `audio_lead_s=0.3` — ~50 % of UNSAFE
|
|
||||||
alerts drop their audio when arm sdk is publishing. Mitigations tested
|
|
||||||
and **none of them fix the root cause** (it's a firmware limitation).
|
|
||||||
The deterministic fix is to serialise audio-then-arm; pending decision.
|
|
||||||
- **Track-ID churn** — each re-acquisition of the same person generates a
|
|
||||||
new track id, which bypasses the per-`(track_id, status)` cooldown and
|
|
||||||
triggers a fresh UNSAFE alert. Not yet quantified, but visible in
|
|
||||||
rapid-fire `ID 0001 → 0002 → 0003` sequences for what appears to be
|
|
||||||
one worker.
|
|
||||||
|
|
||||||
**Configuration right now:**
|
|
||||||
|
|
||||||
- `tts.mode = "tts_only"` (firmware TTS, no recorded WAVs used).
|
|
||||||
- `motion.enabled = true`, `motion.unsafe_file = "adnoc1.jsonl"` — the
|
|
||||||
custom 7-second trajectory with 60 frames of smooth move-in and 180
|
|
||||||
frames of smooth return-to-home.
|
|
||||||
- `bridge.audio_lead_s = 0.3` — parallel mode.
|
|
||||||
- `bridge.cooldown = 8.0` — per (track_id, status).
|
|
||||||
|
|
||||||
**Next-session picks (prioritised):**
|
|
||||||
|
|
||||||
1. Decide parallel-vs-serial for audio+arm and wire it in. Serial is
|
|
||||||
safer for safety-critical alerts.
|
|
||||||
2. Stabilise track-IDs or make the cooldown per-identity instead of
|
|
||||||
per-track so one worker doesn't trigger N alerts.
|
|
||||||
3. Add a boot-time self-check (camera reachable, DDS iface up, model
|
|
||||||
file present, motion JSONL parseable) so failures surface before the
|
|
||||||
first event rather than during it.
|
|
||||||
4. Rotate `logs/` — currently grows unbounded.
|
|
||||||
5. Add a `logs/events.jsonl` audit trail (one structured line per
|
|
||||||
UNSAFE/SAFE event) for post-hoc compliance review.
|
|
||||||
|
|
||||||
See [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md) for the full component
|
See [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md) for the full component
|
||||||
map and data flow.
|
map and data flow.
|
||||||
|
|||||||
Loading…
x
Reference in New Issue
Block a user