miketsai
cd96e33c62
bt_uart.c: - Reader thread: accept plain-text (non-JSON) lines from BLE app; routes directly to extra_cmd_cb — test commands no longer need JSON wrapping - bt_uart_probe_and_upgrade(): fix AT+BAUD7 sequence — reopen fd at 115200 before sending AT+NAME/AT+RESET (was sending on stale 9600 fd) - bt_uart_send_json(): replace blocking mutex_lock with 200 ms trylock retry to avoid deadlock if writer thread dies holding the queue mutex - Reader select timeout 500 ms → 200 ms for faster plain-text flush buzzer.c: - buzzer_set_pattern(): write s_pattern directly then trylock+signal, so callers never block on the buzzer thread's mutex can_bus.c: - Add debug printf in can_bus_send_control_cmd() (temporary) event_recorder.c: - TEST_MODE_TIMEOUT_SEC 60 → 300 (5 min, enough for full test run) kp_firmware.c: - Persist INI keys via sed in-place instead of iniparser_dump_ini, preserving comments and original file structure - setvbuf stdout/stderr to line-buffered so tee log appears immediately Add GF_AI_Box_Test_Guide (.md/.docx/.pdf) and uclibc_compat.c
97 lines
4.6 KiB
Markdown
97 lines
4.6 KiB
Markdown
# CLAUDE.md
|
||
|
||
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
|
||
|
||
## Project Overview
|
||
|
||
KL630 Host Stream Firmware — a real-time semantic segmentation system running on a Kneron KL630 AI SoC. It processes video from a Sony IMX662 DOL-HDR dual-exposure camera for golf course monitoring (pedestrian/vehicle detection via STDC semantic segmentation). The binary runs on embedded Linux (ARMv7-A, uClibc 1.0.34) and outputs via RTSP streaming, HDMI display, BLE (iPad app), and CAN bus (golf cart control).
|
||
|
||
## Build
|
||
|
||
Cross-compilation is Docker-based (Ubuntu 20.04 + `arm-linux-gnueabihf-gcc`). **Docker Desktop must be running.**
|
||
|
||
```bash
|
||
# Install Python dependencies (one-time)
|
||
pip install -r requirements.txt
|
||
|
||
# Compile via web UI (recommended)
|
||
python web_serve.py # then open http://localhost:8080/ and click "Compile"
|
||
|
||
# Compile via CLI
|
||
python build_and_serve.py # outputs build/kp_firmware_host_stream + build/host_stream.ini
|
||
```
|
||
|
||
`compile.sh` is the script executed inside the Docker container — it performs the actual `arm-linux-gnueabihf-gcc` invocation. The `Dockerfile` defines the build image.
|
||
|
||
## Deploy
|
||
|
||
```bash
|
||
# Via web UI: click "Deploy to KL630" (requires device IP configured)
|
||
|
||
# Manual: telnet into device, then pull from HTTP server on host:
|
||
wget http://<HOST_IP>:8080/deploy.sh -O /tmp/deploy.sh && sh /tmp/deploy.sh
|
||
```
|
||
|
||
Device paths:
|
||
- Binary: `/mnt/flash/vienna/kp_firmware_host_stream`
|
||
- Config: `/mnt/flash/plus/kp_firmware/kp_firmware_0/kp_firmware/bin/ini/host_stream.ini`
|
||
- Model (NEF): `/mnt/flash/plus/kp_firmware/kp_firmware_0/kp_firmware/bin/nef/`
|
||
|
||
## Testing
|
||
|
||
No automated test framework. Manual verification:
|
||
- **Web UI** (`web_serve.py`): live RTSP preview, terminal, compile/deploy
|
||
- **Mock server** (`tools/mock_server/server.py`): simulates device events without hardware
|
||
- **On-device**: `ps | grep firmware`, `cat /tmp/fw.log`, `curl http://localhost:8080/api/stdc/stats`
|
||
|
||
## Architecture
|
||
|
||
**Video pipeline:**
|
||
```
|
||
IMX662 (1920×1080, DOL-HDR)
|
||
→ KL630 ISP (YUV420)
|
||
→ VMF (Vatics Media Framework)
|
||
├─ Stream 0 (1920×1080) → H.264 encoder → RTSP
|
||
└─ Stream 1 (724×362) → NPU (STDC inference)
|
||
→ Post-processing
|
||
→ HDMI overlay / BLE / CAN events
|
||
```
|
||
|
||
**Threading model** (all threads in `kp_firmware.c`):
|
||
- **Image thread**: reads ISP output from SSM → enqueues into FifoQ
|
||
- **Inference thread**: FifoQ dispatcher → NPU (STDC model, ModelId=32769)
|
||
- **Results thread**: FifoQ dequeue → post-processing → event dispatch
|
||
- **Video/VOC thread**: H.264 encoding + HDMI draw-box overlay
|
||
|
||
**Key source directories:**
|
||
- `src/host_stream/` — main firmware logic, BLE/UART (`bt_uart.c`), CAN bus (`can_bus.c`, `mcp2515.c`), event recording, FEC API, GPIO
|
||
- `src/app_flow/` — NPU inference orchestration (`stdc_inf_single_model.c`)
|
||
- `src/pre_post_proc/` — STDC post-processing: motion detection, class distribution, ROI analysis
|
||
- `include/common/` — shared data structures (KP_STRUCT, model types)
|
||
- `include/fake/` — stub headers for SDK components unavailable at compile time
|
||
- `ini/host_stream.ini` — runtime config template (model, ISP, FEC, RTSP, event thresholds)
|
||
- `tools/` — deploy scripts, mock server, Python control scripts
|
||
|
||
**Communication interfaces:**
|
||
- **BLE**: JSON over UART at 115200 baud (`/dev/ttyS1`) via DX-BT24 module → iPad app
|
||
- **CAN**: Standard CAN frames at 250 kbps via MCP2515 SPI → golf cart
|
||
- **RTSP**: H.264 from VMF encoder
|
||
- **Web API**: Flask HTTP server (`web_serve.py`) exposes compile/deploy/stats endpoints
|
||
|
||
## Code Conventions
|
||
|
||
- Global variables: `g_` prefix (e.g., `g_dwVocEnable`, `g_stdc_seg_map`)
|
||
- Boolean flags: `_bl` suffix; uint32_t: `_dw` suffix; struct pointers: `pt` prefix
|
||
- Config driven by INI file (iniparser library); sections: `[output]`, `[event]`, `[isp]`, `[fec]`
|
||
- Return codes: `KP_SUCCESS = 0`, negative values for errors
|
||
- Thread safety: mutex `g_stdc_seg_mutex` guards the shared segmentation map
|
||
- Logging: stdout/stderr + `/tmp/fw.log` on device; set `verbose_log=1` in INI for per-frame output
|
||
|
||
## Important Runtime Notes
|
||
|
||
- **ISP one-time setup**: DOL-HDR requires running firmware with `--setup` flag on first boot
|
||
- **Host IP**: configured in `.web_config.json` or via web UI; device uses it for `wget` downloads
|
||
- **FEC mode**: fish-eye correction mode 0–5 in INI; Mode 4 recommended for ceiling-mount
|
||
- **Model switching**: change `ModelId`/`JobId` in INI and redeploy
|
||
- **Startup mode**: `demo_rtsp.sh`, `demo_hdmi.sh`, or `demo_rtsp_hdmi.sh` on device sets output mode
|