Marcus — Control & Startup Guide
Updated: 2026-04-06
Quick Start
Prerequisites (Jetson Orin NX)
# Terminal 1 — Start Holosoma (locomotion policy)
source ~/.holosoma_deps/miniconda3/bin/activate hsinference
cd ~/holosoma
~/.holosoma_deps/miniconda3/envs/hsinference/bin/python3 \
src/holosoma_inference/holosoma_inference/run_policy.py \
inference:g1-29dof-loco \
--task.model-path src/holosoma_inference/holosoma_inference/models/loco/g1_29dof/fastsac_g1_29dof.onnx \
--task.velocity-input zmq \
--task.state-input zmq \
--task.interface eth0
Option A — Terminal Mode (on Jetson)
# Terminal 2 — Start Marcus Brain
conda activate Marcus
ollama serve & sleep 3
cd ~/Marcus
python3 run_marcus.py
Direct keyboard control. All commands typed locally.
Option B — Server + Client (remote)
# Terminal 2 (Jetson) — Start Server
cd ~/Marcus
python3 -m Server.marcus_server
# Terminal 3 (Workstation) — Connect Client
cd ~/Robotics_workspace/yslootahtech/Project/Marcus
python3 -m Client.marcus_cli
Client prompts for connection:
Connection options:
1) eth0 — 192.168.123.164:8765
2) wlan0 — 10.255.254.86:8765
3) custom
Choose [1/2/3] or IP:
Or skip prompt: python3 -m Client.marcus_cli --ip 192.168.123.164 --port 8765
Command Reference
Movement
| Command |
Action |
turn left / turn right |
Rotate (2s default) |
walk forward / move back |
Walk (2s default) |
walk 1 meter |
Precise odometry walk |
walk backward 2 meters |
Precise backward walk |
turn right 90 degrees |
Precise odometry turn |
turn right then walk forward |
Multi-step compound |
come to me / come here |
Forward 2s (instant, no AI) |
stop |
Gradual stop |
Vision
| Command |
Action |
what do you see |
LLaVA describes camera view |
describe the room |
LLaVA scene description |
is anyone here |
LLaVA person check |
yolo |
Show YOLO detection status |
Goal Navigation
| Command |
Action |
goal/ stop when you see a person |
YOLO fast search + stop |
goal/ find a laptop |
YOLO + LLaVA search |
goal/ stop when you see a guy holding a phone |
YOLO + LLaVA compound verification |
find a person |
Auto-detected as goal (no prefix needed) |
look for a bottle |
Auto-detected as goal |
Place Memory
| Command |
Action |
remember this as door |
Save current position |
go to door |
Navigate to saved place |
places |
List all saved places |
forget door |
Delete place |
rename door to entrance |
Rename place |
where am I |
Show odometry position |
go home |
Return to start position |
Patrol
| Command |
Action |
patrol |
Autonomous patrol (prompts for duration) |
patrol: door → desk → exit |
Named waypoint patrol |
Image Search
| Command |
Action |
search/ /path/to/photo.jpg |
Find target from reference image |
search/ /path/to/photo.jpg person in blue shirt |
Image + hint |
search/ person in blue shirt |
Text-only search |
Session Memory
| Command |
Action |
last command |
Show last typed command |
do that again |
Repeat last command |
undo |
Reverse last movement |
last session |
Previous session summary |
session summary |
Current session stats |
System
| Command |
Action |
help |
Command reference |
example |
Usage examples |
q / quit |
Shutdown |
Client-Only Commands (CLI)
| Command |
Action |
status |
Ping server + LiDAR status |
camera |
Get camera configuration |
profile low/medium/high/full |
Switch camera profile |
capture |
Take a photo |
Network Configuration
| Interface |
IP |
Use |
eth0 |
192.168.123.164 |
Robot internal network (Jetson - G1 - LiDAR) |
wlan0 |
10.255.254.86 |
Office WiFi (Jetson - Workstation) |
| Service |
Port |
Protocol |
| Marcus WebSocket |
8765 |
ws:// |
| ZMQ Velocity |
5556 |
tcp:// (PUB/SUB) |
| Ollama API |
11434 |
HTTP |
| LiDAR |
192.168.123.120 |
Livox Mid360 |
All configurable in Config/config_Network.json.
Subsystem Status
On startup, the server/brain shows:
YOLO : active (19 tracked classes, CPU, yolov8m.pt)
Odometry : active (dead reckoning, +/-10cm)
Memory : active (session_016_2026-04-06)
Camera : 424x240@15 (RealSense D435I)
LiDAR : ALIVE (Livox Mid360 at 192.168.123.120)
Arms : pending (GR00T N1.5 not yet integrated)
Troubleshooting
| Issue |
Cause |
Fix |
ModuleNotFoundError: No module named 'Server' |
Wrong directory |
cd ~/Marcus then run |
| Robot doesn't move |
Holosoma not running |
Start Holosoma first (Terminal 1) |
| Robot doesn't move |
ZMQ port conflict |
Only run one of Server or Brain, not both |
Camera: {e} reconnecting |
USB bandwidth |
Reduce to low profile |
| LLaVA slow (>10s) |
GPU VRAM full |
Kill other GPU processes, or use qwen2.5vl:3b |
YOLO not available |
ultralytics not installed |
pip install ultralytics |
| Client can't connect |
Wrong IP or server not running |
Check status command, verify IP |
File Locations
| What |
Path |
| Brain code |
~/Marcus/Brain/ |
| Server |
~/Marcus/Server/marcus_server.py |
| Config |
~/Marcus/Config/ |
| Prompts |
~/Marcus/Config/marcus_prompts.yaml |
| YOLO model |
~/Marcus/Models/yolov8m.pt |
| Session data |
~/Marcus/Data/Brain/Sessions/ |
| Places |
~/Marcus/Data/History/Places/places.json |
| Logs |
~/Marcus/logs/ |
See Doc/architecture.md for full project structure and file-by-file documentation.
