Saqr/DEPLOY.md

Saqr PPE Detection - Deployment Guide

Unitree G1 Robot + Intel RealSense D435I

---

Robot Details

Item	Value
Robot	Unitree G1 Humanoid
IP	192.168.123.164
User	unitree
OS	Ubuntu 20.04 (aarch64 / Jetson)
Python	3.10 (conda env: teleimager)
Camera	Intel RealSense D435I
Serial	243622073459
Port	USB 3.2 @ /dev/video0

---

Step 1: Train the Model (Dev Machine)

cd ~/Robotics_workspace/AI/Saqr
conda activate AI_MSI_yolo
python train.py --dataset dataset --epochs 100 --batch 16

Verify model exists:

ls -lh models/saqr_best.pt
# Expected: ~5.3 MB

---

Step 2: Deploy to Robot (Dev Machine)

Option A: Auto deploy

cd ~/Robotics_workspace/AI/Saqr
./deploy.sh

Option B: Manual SCP

# Create folders
ssh unitree@192.168.123.164 "mkdir -p ~/Saqr/{models,captures/{SAFE,PARTIAL,UNSAFE},Config,Logs}"

# Copy project files
scp saqr.py saqr_g1_bridge.py controller.py detect.py manager.py logger.py gui.py requirements.txt deploy.sh DEPLOY.md \
    unitree@192.168.123.164:~/Saqr/

# Copy config
scp Config/logging.json unitree@192.168.123.164:~/Saqr/Config/

# Copy trained model (5.3 MB)
scp models/saqr_best.pt unitree@192.168.123.164:~/Saqr/models/

---

Step 3: Install Dependencies (Robot)

ssh unitree@192.168.123.164

Fix system clock (required for SSL/pip):

sudo date -s "2026-04-10 15:00:00"

Install into teleimager conda env:

conda activate teleimager
python -m pip install ultralytics opencv-python-headless numpy PyYAML

If pip fails (SSL errors), install offline from dev machine:

# On dev machine:
mkdir -p /tmp/saqr_pkgs
pip download ultralytics opencv-python-headless numpy PyYAML \
    -d /tmp/saqr_pkgs --python-version 3.10 --platform manylinux2014_aarch64 --only-binary=:all:
scp -r /tmp/saqr_pkgs unitree@192.168.123.164:/tmp/saqr_pkgs

# On robot:
conda activate teleimager
python -m pip install --no-index --find-links=/tmp/saqr_pkgs ultralytics opencv-python-headless numpy PyYAML

Install Jetson GPU PyTorch (for CUDA acceleration):

# Remove pip PyTorch (wrong CUDA version)
python -m pip uninstall torch torchvision -y

# Install Jetson-specific PyTorch for JetPack 5.1 / CUDA 11.4
python -m pip install --no-cache-dir \
  https://developer.download.nvidia.com/compute/redist/jp/v51/pytorch/torch-2.1.0a0+41361538.nv23.06-cp310-cp310-linux_aarch64.whl

python -m pip install --no-cache-dir \
  https://developer.download.nvidia.com/compute/redist/jp/v51/pytorch/torchvision-0.16.1a0+5e8e2f1-cp310-cp310-linux_aarch64.whl

Fix Qt / Display (choose one):

A) At the robot's physical terminal (monitor connected):

xhost +local:
export DISPLAY=:0
export QT_QPA_PLATFORM=xcb

B) Via SSH with X11 forwarding:

# From dev machine:
ssh -X unitree@192.168.123.164
export QT_QPA_PLATFORM=xcb

C) Headless / no display (SSH without -X):

export QT_QPA_PLATFORM=offscreen
# Always add --headless flag when running saqr.py

Make permanent:

echo 'export QT_QPA_PLATFORM=offscreen' >> ~/.bashrc
source ~/.bashrc

Common error: Invalid MIT-MAGIC-COOKIE-1 key or could not connect to display :0 This means you're in SSH without X11 auth. Either use ssh -X, run xhost +local: on the physical terminal, or switch to headless mode.

Fix system clock (required for pip/SSL):

sudo date -s "2026-04-10 16:00:00"

Verify install:

python -c "from ultralytics import YOLO; print('ultralytics OK')"
python -c "import torch; print('CUDA:', torch.cuda.is_available())"
python -c "import cv2; print('opencv OK')"

---

Step 4: Run PPE Detection (Robot)

Option A: OpenCV + RealSense RGB (recommended, no pyrealsense2 needed):

conda activate teleimager
cd ~/Saqr

# === WITH DISPLAY (physical monitor on robot) ===
xhost +
export DISPLAY=:0
python saqr.py --source /dev/video2 --model models/saqr_best.pt

# === HEADLESS via SSH (no display, saves captures + CSV) ===
export QT_QPA_PLATFORM=offscreen
python saqr.py --source /dev/video2 --model models/saqr_best.pt --headless

Note: /dev/video2 is the RealSense D435I RGB camera accessed directly via OpenCV V4L2. No pyrealsense2 SDK needed. Pure OpenCV frames (640x480 BGR).

Option B: RealSense SDK (pyrealsense2):

python saqr.py --source realsense --model models/saqr_best.pt --headless
python saqr.py --source realsense:243622073459 --model models/saqr_best.pt --headless

Option C: GUI (dev machine only, not on robot):

# On your dev machine (not the robot):
python gui.py --source 0 --model models/saqr_best.pt

Note: gui.py requires PySide6 and a display. It will NOT work on the headless Jetson robot.

With OpenCV camera index:

python saqr.py --source 0 --model models/saqr_best.pt --headless

With V4L2 device path:

python saqr.py --source /dev/video0 --model models/saqr_best.pt --headless

With GUI (if display connected):

python gui.py --source realsense --model models/saqr_best.pt

Simple detection (no tracking):

python detect.py --source realsense --model models/saqr_best.pt

---

Step 4b: Run with G1 TTS + Reject Action (Bridge)

saqr_g1_bridge.py is the production entry point. It does not run Saqr itself — it sits idle, watches the G1 wireless remote, spawns saqr.py as a subprocess on demand, and drives the G1 onboard TTS + arm action client from Saqr's event stream.

Wireless-remote workflow

Press	Action
R2 + X	Start saqr.py, robot says "Saqr activated."
R2 + Y	Stop saqr.py (SIGINT → SIGTERM → SIGKILL escalation), robot says "Saqr deactivated." Bridge stays running, ready for the next R2+X.
Ctrl+C in the terminal	Stop saqr (if running) and exit the bridge cleanly.

Each press is rising-edge debounced (release-wait), so holding the button only fires once. Track IDs and per-id status state are reset on every start, so a leftover SAFE from one session never suppresses an UNSAFE in the next.

Per-detection behavior (while saqr is running)

Transition	TTS (speaker_id=2, English)	Arm action
→ UNSAFE	"Please stop. Wear your proper safety equipment. You are missing {items}." (or generic text if no items reported)	reject (id=13) + auto release arm
→ SAFE	"Safe to enter. Have a good day."	—
→ PARTIAL	—	—

The missing-item list is parsed live from Saqr's event line and joined in natural English: "vest", "helmet and vest", "helmet, vest, and gloves".

Architecture

• One ChannelFactoryInitialize(0, eth0) is shared by all DDS clients:
  • G1ArmActionClient — runs the reject arm action.
  • G1 AudioClient — TtsMaker(text, speaker_id) for English speech.
  • ChannelSubscriber("rt/lowstate", LowState_) — receives the wireless remote button bits.
• controller.py exposes LowStateHub + UnitreeRemote, parses the wireless_remote byte field, and provides combo_r2x() / combo_r2y().
• A 50 Hz daemon thread polls the hub for rising edges and calls Bridge.start_saqr() / Bridge.stop_saqr().

Requires unitree_sdk2py installed on the robot and a reachable DDS bus on eth0. Deploy controller.py alongside saqr_g1_bridge.py — without it the trigger loop is skipped and the bridge falls back to legacy auto-start mode.

Recommended: production run with R2+X / R2+Y

conda activate marcus      # or teleimager — whichever env has unitree_sdk2py
cd ~/Saqr
python3 saqr_g1_bridge.py --iface eth0 --source realsense --headless -- --stream 8080

Boot output should include:

[BRIDGE] G1ArmActionClient ready (iface=eth0)
[BRIDGE] G1 AudioClient ready (speaker_id=2)
[BRIDGE] Subscribed to rt/lowstate (wireless remote)
[BRIDGE] trigger loop ready — press R2+X to start, R2+Y to stop.

Then press R2+X to begin, R2+Y to stop. The MJPEG stream is at http://192.168.123.164:8080 (only while saqr is running).

If pyrealsense2 reports No device connected, fall back to the V4L2 path:

python3 saqr_g1_bridge.py --iface eth0 --source /dev/video2 --headless -- --stream 8080

Live OpenCV window on the robot's physical monitor

xhost +local: >/dev/null 2>&1
DISPLAY=:0 python3 saqr_g1_bridge.py --iface eth0 --source realsense

q in the OpenCV window quits the current saqr session (same as R2+Y).

Legacy / dev mode (no controller, no trigger)

--no-trigger skips the wireless-remote subscription entirely and starts saqr immediately. Use this on the workstation or when you want the old "always running" behavior.

# On the workstation, no robot, no SDK:
python3 saqr_g1_bridge.py --no-trigger --dry-run --source 0 --headless

# On the robot, but skipping the trigger:
python3 saqr_g1_bridge.py --no-trigger --iface eth0 --source realsense --headless

--dry-run automatically implies --no-trigger (no SDK = no LowState).

Bridge CLI flags

Flag	Default	Description
--iface	(default DDS)	DDS network interface, e.g. eth0
--timeout	10.0	Arm/Audio/LowState client timeout (seconds)
--cooldown	8.0	Per-(track_id, status) seconds before re-triggering TTS/arm
--release-after	2.0	Seconds before auto release arm (0 = never)
--speaker-id	2	G1 TtsMaker speaker_id (2 = English on current firmware)
--dry-run	off	Parse events but never call the SDK; implies --no-trigger
--no-trigger	off	Skip the R2+X/R2+Y trigger loop and start saqr immediately
--source	—	Pass through to saqr (0 / realsense / /dev/video2 / path)
--headless	off	Pass --headless to saqr
--saqr-conf	—	Pass --conf to saqr
--imgsz	—	Pass --imgsz to saqr
--device	—	Pass --device to saqr (cpu / 0 / cuda:0)
-- <extra>	—	Everything after -- is forwarded raw to saqr (use this for --stream 8080, --half, etc.)

Speaker-id reference

speaker_ids are locked to a language — they do NOT auto-detect input text. On current G1 firmware, speaker_id=0 is Chinese regardless of what you feed it. Speaker 2 was confirmed English by running Sanad mode 6 (voice_example.py 6). If the robot's firmware changes, re-scan:

# On the robot (in a conda env with unitree_sdk2py):
python3 ~/Sanad/voice_example.py 6

and pass the new id with --speaker-id N.

What a successful run looks like

[BRIDGE] G1ArmActionClient ready (iface=eth0)
[BRIDGE] G1 AudioClient ready (speaker_id=2)
[BRIDGE] Subscribed to rt/lowstate (wireless remote)
[BRIDGE] trigger loop ready — press R2+X to start, R2+Y to stop.
[BRIDGE] R2+X pressed -> start saqr
[BRIDGE] starting saqr: /.../python3 -u /home/unitree/Saqr/saqr.py --source realsense --headless --stream 8080
[BRIDGE] tts -> 'Saqr activated.'
...
ID 0002 | NEW | UNSAFE | wearing: none | missing: vest | ...
[BRIDGE] tts -> 'Please stop. Wear your proper safety equipment. You are missing vest.'
[BRIDGE] -> reject
[BRIDGE] -> release arm
ID 0003 | STATUS_CHANGE | SAFE | wearing: helmet, vest | missing: none | ...
[BRIDGE] tts -> 'Safe to enter. Have a good day.'
[BRIDGE] R2+Y pressed -> stop saqr
[BRIDGE] stopping saqr (SIGINT)
[BRIDGE] saqr exited rc=-2
[BRIDGE] tts -> 'Saqr deactivated.'

Note on the SIGINT traceback: when R2+Y stops saqr, you may see a Python KeyboardInterrupt traceback unwinding from inside YOLO. This is expected — saqr.py doesn't catch SIGINT explicitly, so Python prints the stack on its way out. The bridge correctly detects the exit, announces "Saqr deactivated.", and stays alive ready for the next R2+X.

---

Step 5: Check Results (Robot)

Live status:

cat ~/Saqr/captures/result.csv

Event history (audit log):

cat ~/Saqr/captures/events.csv

Captured photos:

ls ~/Saqr/captures/SAFE/
ls ~/Saqr/captures/PARTIAL/
ls ~/Saqr/captures/UNSAFE/

Export CSV report:

cd ~/Saqr
python manager.py --export

Download results to dev machine:

# From dev machine
scp -r unitree@192.168.123.164:~/Saqr/captures/ ./captures_from_robot/
scp unitree@192.168.123.164:~/Saqr/captures/events.csv ./events_robot.csv

---

Camera Source Options

Source	Command	Description
/dev/video2	--source /dev/video2	RGB camera via OpenCV (recommended)
realsense	--source realsense	RealSense D435I via pyrealsense2 SDK
realsense:SERIAL	--source realsense:243622073459	Specific RealSense by serial
/dev/video4	--source /dev/video4	Second RGB stream (if available)
0	--source 0	First OpenCV camera index
video.mp4	--source video.mp4	Video file
image.jpg	--source image.jpg	Single image

G1 Robot V4L2 Device Map (RealSense D435I):

/dev/video0  - Stereo module (infrared) - won't open with OpenCV
/dev/video1  - Stereo metadata
/dev/video2  - RGB camera (640x480) ← USE THIS
/dev/video3  - RGB metadata
/dev/video4  - RGB camera (secondary stream)

Detect cameras on robot:

# Find working RGB cameras
python -c "
import cv2
for i in range(10):
    cap = cv2.VideoCapture(f'/dev/video{i}', cv2.CAP_V4L2)
    if cap.isOpened():
        ret, frame = cap.read()
        if ret and frame is not None:
            print(f'/dev/video{i}: {frame.shape} OK')
        else:
            print(f'/dev/video{i}: opened but no frame')
        cap.release()
"

# RealSense devices
rs-enumerate-devices | grep "Serial Number"

---

Tuning Parameters

Parameter	Default	Flag	Description
Confidence	0.35	--conf 0.35	Lower = more detections, higher = fewer false positives
Max Missing	90	--max-missing 90	Frames before track deleted (~3s at 30fps)
Match Distance	250	--match-distance 250	Pixels for track matching
Confirm Frames	5	--status-confirm-frames 5	Frames to confirm a status change

Recommended for G1 patrol:

python saqr.py --source realsense --model models/saqr_best.pt --headless \
    --conf 0.30 --max-missing 120 --match-distance 300 --status-confirm-frames 7

---

Compliance Rules

Status	Condition	Color
SAFE	Helmet AND vest detected, no violations	Green
PARTIAL	Only helmet OR only vest detected	Yellow
UNSAFE	no-helmet or no-vest detected, or nothing detected	Red

---

Output Files

File	Location	Description
result.csv	captures/result.csv	Current state of all tracked persons
events.csv	captures/events.csv	Audit log (NEW / STATUS_CHANGE events)
Person crops	captures/SAFE/*.jpg	Cropped images of compliant workers
Person crops	captures/PARTIAL/*.jpg	Workers with incomplete PPE
Person crops	captures/UNSAFE/*.jpg	Workers violating PPE rules
Logs	Logs/Inference/saqr.log	Runtime log

---

Project Files

File	Purpose
saqr.py	Main PPE tracking + detection (RealSense + OpenCV)
saqr_g1_bridge.py	Saqr → G1 bridge (R2+X/R2+Y trigger, onboard TTS + reject arm action on UNSAFE/SAFE transitions)
controller.py	G1 wireless-remote DDS reader (LowStateHub, combo_r2x(), combo_r2y()); required by the bridge for trigger keys
detect.py	Simple detection without tracking
gui.py	PySide6 desktop GUI
manager.py	Photo management CLI + CSV export
train.py	YOLO model training
logger.py	Centralized logging
deploy.sh	One-command deploy to robot
Config/logging.json	Log settings

---

Troubleshooting

RealSense not detected

# Check USB connection
lsusb | grep Intel

# Re-enumerate
rs-enumerate-devices | head -10

# Reset USB (if needed)
sudo usbreset /dev/bus/usb/002/002

Camera not opening

# Test RealSense directly
python -c "
import pyrealsense2 as rs
pipe = rs.pipeline()
cfg = rs.config()
cfg.enable_stream(rs.stream.color, 640, 480, rs.format.bgr8, 30)
pipe.start(cfg)
frames = pipe.wait_for_frames()
print('Frame:', frames.get_color_frame().get_width(), 'x', frames.get_color_frame().get_height())
pipe.stop()
"

# Test OpenCV fallback
python -c "import cv2; c=cv2.VideoCapture(0); print('OK' if c.isOpened() else 'FAIL'); c.release()"

# Try different source
python saqr.py --source /dev/video0 --model models/saqr_best.pt --headless

ModuleNotFoundError: ultralytics

# Check you're in the right conda env
which python
# Should show: /home/unitree/miniconda3/envs/teleimager/bin/python

# Install to the correct env
python -m pip install ultralytics

System clock wrong (SSL errors)

sudo date -s "2026-04-10 15:00:00"

Model not found

ls ~/Saqr/models/
# Should show: saqr_best.pt (~5.3 MB)

Low FPS on Jetson

# Use smaller confidence to reduce load
python saqr.py --source realsense --conf 0.5 --headless

# Or use headless opencv
export DISPLAY=
python saqr.py --source realsense --headless

Too many duplicate track IDs

# Increase tolerance
python saqr.py --source realsense --max-missing 150 --match-distance 300 --headless