Compare commits

...

6 Commits

Author SHA1 Message Date
a7bc67baba chore: ignore .autoflow / .mcp.json / graphify-out
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 19:27:45 +08:00
744283bd28 feat(build): 安裝包只打包指定模型並改用中文名稱
安裝包從 8 個 .nef 縮減為 2 個,models.json 保留完整定義以便日後
加回。

- Makefile 加 BUNDLED_NEFS 白名單,三平台共用 copy_bundled_data
  helper;用 POSIX find/cp 而非 rsync(Windows CI 的 Git Bash 沒有
  rsync)
- 白名單檔案不存在時 build 直接失敗,並在複製後驗證 models.json
  存在且 .nef 數量相符 —— 避免產出「安裝後 0 個模型」卻回報成功
- FCOS Detection (KL520) 改名為「物件辨識」
- Tiny YOLOv3 (KL520) 改名為「人型監測」
  (只改 name/description,id 不動以免影響既有設定與紀錄)
- Repository 啟動時過濾 .nef 不存在的模型,否則使用者會看到未打包
  的模型並在選取後拿到莫名錯誤

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 19:27:40 +08:00
5c1c37d151 feat(frontend): classification 結果呈現與推論期控制項
detection 畫框、classification 改在影像右上角疊標籤(不畫框),
並在推論面板提供即時切換解析方式與上傳 label 檔。

- 新增 InferenceOverlay 依 taskType 分派;ClassificationOverlay 用
  DOM 而非 canvas,讓 CJK 排版與 aria-live 交給瀏覽器處理
- 標籤防閃爍:挑戰者需連續 3 幀居冠才切換,或信心度領先 15% 直接切;
  低於門檻立即清空(顯示過期標籤比空白更糟)
- 身分比對優先用 classIndex,避免換 label 檔時被誤判為換類別
- 推論設定卡片:即時切 detection/classification、上傳 .txt label、
  清除 label,皆不需重燒
- classification-result 原本無條件渲染,導致 detection 模式下側欄
  永遠顯示「分類結果」與空圖表,改為依 taskType 切換
- 清掉 camera-overlay 每幀執行的 debug console.log
- 補 ResizeObserver stub,原本任何渲染 InferencePanel 的測試都會
  在 jsdom 直接拋錯

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 19:27:28 +08:00
4583406efa feat(api): 推論期切換解析方式與上傳 label 檔
新增 POST /api/devices/:id/inference/options,讓已燒錄的模型不重燒
即可切換 detection/classification 與更換 label。

KL520 一次只能載一個模型(換模型必須重燒),但解析方式與 label 都
只是後處理與顯示層,不碰 NPU,因此可即時生效。

- JSON: {taskType?, labels?};multipart: taskType + labelFile
- 欄位用指標型別以區分「省略 = 不動」與「labels:[] = 清空」
- label 解析器(internal/labelfile):格式 <index> <名稱>
  略過空行與 # 註解、trim CRLF、名稱含空白只 split 第一個空白、
  index 非整數/負數/重複則整檔拒絕並回報行號、稀疏 index 補空字串
- index 上限 4095、檔案上限 256KB,防記憶體耗盡
- 非法 taskType 一律拒絕(含舊別名 detection),因為 bridge 對無法
  辨識的值是靜默 fallback,忽略會回 200 但用錯解析方式
- 走窄能力介面 InferenceOptionsDriver,不動 DeviceDriver 簽章

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 19:27:16 +08:00
ddd1aae5d1 feat(server): 把 model metadata 傳進推論鏈路
models.json 宣告的 taskType/labels/inputSize 原本在 flash 時被丟棄
(只傳 modelPath),導致 bridge 只能靠檔名猜測模型類型與尺寸。

- FlashOptions 帶 TaskType/Labels/InputWidth/InputHeight
- 抽出 buildLoadModelCommand,四處 load_model 呼叫點(初次 + 三條
  retry 路徑)統一走它,並加測試釘住呼叫點數量與「不得有手寫 payload」
  —— 讓漏改 retry 路徑在結構上不可能發生
- ClassResult 加 ClassIndex(不加 omitempty,index 0 是合法值)

用 FlashOptions struct 而非裸參數,未來擴充欄位不需再動 interface
與所有 test fake。

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 19:27:02 +08:00
f14d24bd7b feat(bridge): classification 自適應解析 + 推論期動態切換
新增 classification 推論支援,核心是「不預設模型長相」:

- output shape 自適應:支援 (1,C)/(C,)/(1,C,1,1)/(C,1,1)/(1,1,1,C)
  及任何 squeeze 後為一維的張量;類別數從 shape 動態取得,
  移除原本寫死的 num_classes=1000
- 無法解析時明確報錯(附實際 shape),不靜默回空結果
- logits vs 已 softmax 自動偵測:sum≈1.0 且全非負才跳過 softmax
  (容差 1e-4,實測 float32 softmax 偏差最大僅 ~4e-7)
- label 為純顯示層:有注入用 label、沒有則輸出原始 enum class_N
- input size 三層來源:SDK > 宣告值 > 檔名猜測,log 標示來源
- 新增 handle_set_inference_options:不重載模型即可改解析方式與 label

修正:
- _detect_model_type 改為外部指定優先,解決未知檔名被誤判成
  tiny_yolov3 而回傳空結果的根因
- handle_disconnect/reset 未清 label 狀態,導致換模型後沿用舊 label 表
- load_model 失敗路徑會污染全域 metadata,改為 defer-until-success
- taskType 值域統一為 object_detection(原本回 detection)
- 檔名解析原本只取 width 丟棄 height,非正方形模型會被壓成正方形

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 19:26:51 +08:00
47 changed files with 7262 additions and 157 deletions

View File

@ -78,3 +78,7 @@ desktop.ini
*.pem *.pem
*.key *.key
.autoflow/
.mcp.json
graphify-out/

View File

@ -18,6 +18,61 @@ OS := $(shell uname -s | tr A-Z a-z)
DIST := dist DIST := dist
PAYLOAD := visiona-local/payload PAYLOAD := visiona-local/payload
# ── 打包用 .nef 白名單M5-a────────────────────────────────────────
#
# server/data/ 底下有 8 個 .nefKL520 五個、KL720 三個),但安裝包只帶
# 白名單內的這幾個,其餘不進 payload藉此縮小安裝檔體積。
#
# models.json 不做任何過濾7 個 model 定義全部照原樣複製。執行期由
# server/internal/model/repository.go 的 NewRepository() 檢查每個 model 的
# filePath 是否實際存在,不存在的直接不載入(見 M5-c。因此使用者在 UI
# 只會看到白名單內的 model未打包的不會出現、也不會選到後拿到莫名錯誤。
#
# 未來要把某個 model 加回安裝包:把對應的 .nef 相對路徑加進下面這個變數即可,
# models.json 不用動。
#
# 路徑相對於 server/data/。
BUNDLED_NEFS := \
nef/kl520/kl520_20004_fcos-drk53s_w512h512.nef \
nef/kl520/kl520_tiny_yolo_v3.nef
# copy_bundled_data把 server/data/ 複製到 $(1),但 nef/ 只帶 BUNDLED_NEFS 白名單。
# $(1) = 目標 data 目錄payload/darwin/data
#
# 步驟:(a) 複製 server/data/ 下除了 nef/ 以外的所有東西models.json 等)
# (b) 再逐一複製白名單內的 .nef
# (c) 白名單檔案不存在就直接 fail避免安靜地產出缺 model 的安裝包
#
# 只用 POSIX find / cp不用 rsync —— Windows CI 跑在 Git Bashwindows-2022 +
# shell: bash該環境沒有 rsync。
define copy_bundled_data
@set -e; \
echo "==> 複製 server/data → $(1).nef 白名單:$(words $(BUNDLED_NEFS)) 個)"; \
if [ ! -d server/data ]; then echo "!! ERROR: server/data 不存在 !!"; exit 1; fi; \
mkdir -p "$(1)"; \
dest="$$(cd "$(1)" && pwd)"; \
( cd server/data && \
find . -path ./nef -prune -o -type d -print | while read -r d; do mkdir -p "$$dest/$$d"; done && \
find . -path ./nef -prune -o -type f -print | while read -r f; do cp "$$f" "$$dest/$$f"; done ); \
if [ ! -f "$$dest/models.json" ]; then \
echo "!! ERROR: models.json 沒有被複製到 $$dest !!"; exit 1; \
fi; \
for nef in $(BUNDLED_NEFS); do \
if [ ! -f "server/data/$$nef" ]; then \
echo "!! ERROR: BUNDLED_NEFS 列出的 server/data/$$nef 不存在 !!"; \
exit 1; \
fi; \
mkdir -p "$$dest/$$(dirname $$nef)"; \
cp "server/data/$$nef" "$$dest/$$nef"; \
echo " + $$nef"; \
done; \
copied=$$(find "$$dest" -name '*.nef' | wc -l | tr -d ' '); \
if [ "$$copied" != "$(words $(BUNDLED_NEFS))" ]; then \
echo "!! ERROR: 預期 $(words $(BUNDLED_NEFS)) 個 .nef實際 $$copied 個 !!"; exit 1; \
fi; \
echo " models.json + $$copied 個 .nef 已就位"
endef
.PHONY: help \ .PHONY: help \
vendor-sync vendor-python vendor-wheels vendor-ffmpeg vendor-ffmpeg-macos-build \ vendor-sync vendor-python vendor-wheels vendor-ffmpeg vendor-ffmpeg-macos-build \
vendor-python-windows vendor-wheels-windows vendor-ffmpeg-windows \ vendor-python-windows vendor-wheels-windows vendor-ffmpeg-windows \
@ -245,7 +300,7 @@ payload-macos: build-server vendor-python vendor-wheels vendor-ffmpeg ## 準備
cp vendor/ffmpeg/macos/ffprobe payload/darwin/bin/ cp vendor/ffmpeg/macos/ffprobe payload/darwin/bin/
cp vendor/ffmpeg/macos/COPYING.LGPLv3 payload/darwin/bin/ffmpeg-COPYING.LGPLv3 cp vendor/ffmpeg/macos/COPYING.LGPLv3 payload/darwin/bin/ffmpeg-COPYING.LGPLv3
chmod +x payload/darwin/bin/ffmpeg payload/darwin/bin/ffprobe chmod +x payload/darwin/bin/ffmpeg payload/darwin/bin/ffprobe
cp -R server/data/* payload/darwin/data/ $(call copy_bundled_data,payload/darwin/data)
cp -R server/scripts/* payload/darwin/scripts/ cp -R server/scripts/* payload/darwin/scripts/
cp vendor/python/darwin/python.tar.gz payload/darwin/python/ cp vendor/python/darwin/python.tar.gz payload/darwin/python/
@cp vendor/wheels/darwin/*.whl payload/darwin/wheels/ 2>/dev/null || true @cp vendor/wheels/darwin/*.whl payload/darwin/wheels/ 2>/dev/null || true
@ -360,7 +415,7 @@ payload-windows: build-server-windows vendor-python-windows vendor-wheels-window
@# LGPL 授權條款BtbN build 自帶 LICENSE.txtCOPYING.LGPLv3 不一定在壓縮檔內,失敗不致命) @# LGPL 授權條款BtbN build 自帶 LICENSE.txtCOPYING.LGPLv3 不一定在壓縮檔內,失敗不致命)
@cp vendor/ffmpeg/windows/LICENSE.txt payload/windows/bin/ffmpeg-LICENSE.txt 2>/dev/null || true @cp vendor/ffmpeg/windows/LICENSE.txt payload/windows/bin/ffmpeg-LICENSE.txt 2>/dev/null || true
@cp vendor/ffmpeg/windows/COPYING.LGPLv3 payload/windows/bin/ffmpeg-COPYING.LGPLv3 2>/dev/null || true @cp vendor/ffmpeg/windows/COPYING.LGPLv3 payload/windows/bin/ffmpeg-COPYING.LGPLv3 2>/dev/null || true
cp -R server/data/. payload/windows/data/ $(call copy_bundled_data,payload/windows/data)
cp -R server/scripts/. payload/windows/scripts/ cp -R server/scripts/. payload/windows/scripts/
cp vendor/python/windows/python.tar.gz payload/windows/python/ cp vendor/python/windows/python.tar.gz payload/windows/python/
@cp vendor/wheels/windows/*.whl payload/windows/wheels/ 2>/dev/null || true @cp vendor/wheels/windows/*.whl payload/windows/wheels/ 2>/dev/null || true
@ -444,7 +499,7 @@ payload-linux: build-server-linux vendor-python-linux vendor-wheels-linux vendor
@cp vendor/ffmpeg/linux/ffmpeg payload/linux/bin/ 2>/dev/null && chmod +x payload/linux/bin/ffmpeg || echo "!! WARN: ffmpeg 缺失" @cp vendor/ffmpeg/linux/ffmpeg payload/linux/bin/ 2>/dev/null && chmod +x payload/linux/bin/ffmpeg || echo "!! WARN: ffmpeg 缺失"
@cp vendor/ffmpeg/linux/ffprobe payload/linux/bin/ 2>/dev/null && chmod +x payload/linux/bin/ffprobe || echo "!! WARN: ffprobe 缺失" @cp vendor/ffmpeg/linux/ffprobe payload/linux/bin/ 2>/dev/null && chmod +x payload/linux/bin/ffprobe || echo "!! WARN: ffprobe 缺失"
@cp vendor/ffmpeg/linux/LICENSE.txt payload/linux/bin/ffmpeg-LICENSE.txt 2>/dev/null || true @cp vendor/ffmpeg/linux/LICENSE.txt payload/linux/bin/ffmpeg-LICENSE.txt 2>/dev/null || true
@if [ -d server/data ]; then cp -R server/data/. payload/linux/data/; fi $(call copy_bundled_data,payload/linux/data)
@if [ -d server/scripts ]; then cp -R server/scripts/. payload/linux/scripts/; fi @if [ -d server/scripts ]; then cp -R server/scripts/. payload/linux/scripts/; fi
@if [ ! -f vendor/python/linux/python.tar.gz ]; then \ @if [ ! -f vendor/python/linux/python.tar.gz ]; then \
echo "!! ERROR: vendor/python/linux/python.tar.gz 不存在vendor-python-linux 應該已先跑過 !!"; \ echo "!! ERROR: vendor/python/linux/python.tar.gz 不存在vendor-python-linux 應該已先跑過 !!"; \

View File

@ -8,6 +8,7 @@ import { InferencePanel } from '@/components/inference/inference-panel';
import { FlashDialog } from '@/components/devices/flash-dialog'; import { FlashDialog } from '@/components/devices/flash-dialog';
import { useDeviceStore } from '@/stores/device-store'; import { useDeviceStore } from '@/stores/device-store';
import { useInferenceStore } from '@/stores/inference-store'; import { useInferenceStore } from '@/stores/inference-store';
import { useInferenceOptionsStore } from '@/stores/inference-options-store';
import { useInferenceStream } from '@/hooks/use-inference-stream'; import { useInferenceStream } from '@/hooks/use-inference-stream';
import { useCameraStore } from '@/stores/camera-store'; import { useCameraStore } from '@/stores/camera-store';
import { useResolvedParams } from '@/hooks/use-resolved-params'; import { useResolvedParams } from '@/hooks/use-resolved-params';
@ -17,6 +18,7 @@ export default function WorkspaceClient() {
const { deviceId } = useResolvedParams(); const { deviceId } = useResolvedParams();
const { selectedDevice, fetchDevice } = useDeviceStore(); const { selectedDevice, fetchDevice } = useDeviceStore();
const { isRunning, setRunning, reset } = useInferenceStore(); const { isRunning, setRunning, reset } = useInferenceStore();
const resetInferenceOptions = useInferenceOptionsStore((s) => s.reset);
const { isStreaming, sourceType } = useCameraStore(); const { isStreaming, sourceType } = useCameraStore();
// For image/video mode, inference runs automatically as part of the pipeline // For image/video mode, inference runs automatically as part of the pipeline
@ -41,8 +43,11 @@ export default function WorkspaceClient() {
} }
return () => { return () => {
reset(); reset();
// Options are per-session and per-device; leaving them set would apply a
// label mapping uploaded for one device to the next one opened.
resetInferenceOptions();
}; };
}, [deviceId, fetchDevice, fetchCameras, reset]); }, [deviceId, fetchDevice, fetchCameras, reset, resetInferenceOptions]);
const handleStartInference = async () => { const handleStartInference = async () => {
await api.post(`/devices/${deviceId}/inference/start`); await api.post(`/devices/${deviceId}/inference/start`);
@ -89,7 +94,7 @@ export default function WorkspaceClient() {
<CameraInferenceView deviceId={deviceId} /> <CameraInferenceView deviceId={deviceId} />
</div> </div>
<div className="w-80 shrink-0"> <div className="w-80 shrink-0">
<InferencePanel /> <InferencePanel deviceId={deviceId} />
</div> </div>
</div> </div>
</div> </div>

View File

@ -2,7 +2,7 @@
import { useState, useCallback } from 'react'; import { useState, useCallback } from 'react';
import { CameraFeed } from './camera-feed'; import { CameraFeed } from './camera-feed';
import { CameraOverlay } from './camera-overlay'; import { InferenceOverlay } from './inference-overlay';
import { SourceSelector } from './source-selector'; import { SourceSelector } from './source-selector';
import { BatchImageThumbnails } from './batch-image-thumbnails'; import { BatchImageThumbnails } from './batch-image-thumbnails';
import { useCameraStore } from '@/stores/camera-store'; import { useCameraStore } from '@/stores/camera-store';
@ -25,11 +25,10 @@ export function CameraInferenceView({ deviceId }: CameraInferenceViewProps) {
setRenderedSize((prev) => (prev && prev.w === w && prev.h === h ? prev : { w, h })); setRenderedSize((prev) => (prev && prev.w === w && prev.h === h ? prev : { w, h }));
}, []); }, []);
// In batch mode, show the selected image's detections // In batch mode, show the selected image's result
const selectedResult = isBatchMode const selectedResult = isBatchMode
? batchResults[batchSelectedIndex] ? batchResults[batchSelectedIndex]
: result; : result;
const detections = selectedResult?.detections || [];
// In batch mode, use static image endpoint for viewing selected image // In batch mode, use static image endpoint for viewing selected image
const batchImageUrl = isBatchMode const batchImageUrl = isBatchMode
@ -48,8 +47,8 @@ export function CameraInferenceView({ deviceId }: CameraInferenceViewProps) {
onDimensionsChange={handleDimensionsChange} onDimensionsChange={handleDimensionsChange}
overlay={ overlay={
isStreaming && renderedSize ? ( isStreaming && renderedSize ? (
<CameraOverlay <InferenceOverlay
detections={detections} result={selectedResult}
width={renderedSize.w} width={renderedSize.w}
height={renderedSize.h} height={renderedSize.h}
confidenceThreshold={confidenceThreshold} confidenceThreshold={confidenceThreshold}

View File

@ -25,16 +25,6 @@ export function CameraOverlay({ detections, width, height, confidenceThreshold }
const filtered = detections.filter((d) => d.confidence >= confidenceThreshold); const filtered = detections.filter((d) => d.confidence >= confidenceThreshold);
if (typeof window !== 'undefined') {
// TEMP debug: 驗證 bbox coordinate space 對齊問題
// eslint-disable-next-line no-console
console.log('[bbox-debug] canvas=%dx%d total=%d filtered=%d threshold=%s', width, height, detections.length, filtered.length, confidenceThreshold, filtered.map((d) => ({
label: d.label,
bbox: d.bbox,
conf: d.confidence,
})));
}
filtered.forEach((det, i) => { filtered.forEach((det, i) => {
const color = COLORS[i % COLORS.length]; const color = COLORS[i % COLORS.length];
// Convert normalized coordinates (0-1) to pixel values // Convert normalized coordinates (0-1) to pixel values
@ -68,6 +58,7 @@ export function CameraOverlay({ detections, width, height, confidenceThreshold }
height={height} height={height}
style={{ width, height }} style={{ width, height }}
className="absolute left-0 top-0 pointer-events-none" className="absolute left-0 top-0 pointer-events-none"
data-testid="camera-overlay"
/> />
); );
} }

View File

@ -0,0 +1,67 @@
'use client';
import type { ClassResult } from '@/types/inference';
import { classResultLabel } from '@/lib/classification';
import { useStableTopClass } from '@/hooks/use-stable-top-class';
import { useTranslation } from '@/lib/i18n';
interface ClassificationOverlayProps {
classifications: ClassResult[];
width: number;
height: number;
confidenceThreshold: number;
}
/**
* Classification overlay no bounding boxes.
*
* Classification models produce a whole-image verdict rather than localised
* objects, so we annotate the frame with a single top-1 chip instead of
* drawing rectangles. Rendered as absolutely positioned DOM (not canvas) so
* that CJK label text, rounded corners and theming come from CSS rather than
* hand-rolled `fillText` layout, and so screen readers can announce the result.
*
* The chip sits on the top-right, because CameraFeed already occupies the
* top-left with the source-type badge.
*/
export function ClassificationOverlay({
classifications,
width,
height,
confidenceThreshold,
}: ClassificationOverlayProps) {
const { t } = useTranslation();
const top = useStableTopClass(classifications, { confidenceThreshold });
return (
<div
style={{ width, height }}
className="absolute left-0 top-0 pointer-events-none"
data-testid="classification-overlay"
>
<div
className="absolute right-2 top-2 max-w-[70%] rounded-md bg-black/70 px-3 py-2 text-white shadow-lg"
// Announce the verdict for screen-reader users; `polite` avoids
// interrupting on every frame of a video stream.
role="status"
aria-live="polite"
aria-atomic="true"
>
{top ? (
<div className="flex items-baseline gap-2">
<span className="truncate text-base font-semibold" data-testid="classification-overlay-label">
{classResultLabel(top)}
</span>
<span className="shrink-0 text-sm tabular-nums opacity-80" data-testid="classification-overlay-confidence">
{(top.confidence * 100).toFixed(1)}%
</span>
</div>
) : (
<span className="text-sm opacity-80" data-testid="classification-overlay-empty">
{t('inference.unrecognized')}
</span>
)}
</div>
</div>
);
}

View File

@ -0,0 +1,53 @@
'use client';
import type { InferenceResult } from '@/types/inference';
import { isClassificationTask } from '@/lib/classification';
import { CameraOverlay } from './camera-overlay';
import { ClassificationOverlay } from './classification-overlay';
interface InferenceOverlayProps {
result: InferenceResult | null | undefined;
width: number;
height: number;
confidenceThreshold: number;
}
/**
* Dispatches the frame overlay based on the task type reported by the
* inference result itself.
*
* `result.taskType` is preferred over the model metadata because it reflects
* what the Python post-processor actually did metadata can disagree with the
* executed code path.
*
* Note the branch is written as "is this classification?" rather than "is this
* detection?" on purpose: the detection task type string has been inconsistent
* across the stack (`detection` vs `object_detection`, plan R-4), so detection
* is the fall-through default and stays correct under either spelling.
*/
export function InferenceOverlay({
result,
width,
height,
confidenceThreshold,
}: InferenceOverlayProps) {
if (isClassificationTask(result?.taskType)) {
return (
<ClassificationOverlay
classifications={result?.classifications || []}
width={width}
height={height}
confidenceThreshold={confidenceThreshold}
/>
);
}
return (
<CameraOverlay
detections={result?.detections || []}
width={width}
height={height}
confidenceThreshold={confidenceThreshold}
/>
);
}

View File

@ -1,6 +1,6 @@
'use client'; 'use client';
import { useEffect, useMemo, useState } from 'react'; import { useEffect, useState } from 'react';
import { import {
Dialog, Dialog,
DialogContent, DialogContent,
@ -39,11 +39,16 @@ export function FlashDialog({ deviceId }: FlashDialogProps) {
const device = devices.find((d) => d.id === deviceId); const device = devices.find((d) => d.id === deviceId);
const selectedModel = models.find((m) => m.id === selectedModelId); const selectedModel = models.find((m) => m.id === selectedModelId);
// S2: 資料載入前預設 compatible=true避免在 model/device 還沒載入時就顯示不相容警告 // 載入模型時不選推論種類解析方式改由推論頁InferenceOptions即時切換
const compatible = useMemo(() => { // 那裡不需要重燒就能改,功能完全涵蓋燒錄時選一次的舊做法。
if (!selectedModel || !device) return true;
return isModelCompatible(selectedModel.supportedHardware, device.type); // S2: 資料載入前預設 compatible=true避免在 model/device 還沒載入時就顯示不相容警告。
}, [selectedModel, device]); // 這裡不再包 useMemo —— isModelCompatible 只是一次陣列比對,手動 memo 反而讓
// React Compiler 整個元件跳過優化react-hooks/preserve-manual-memoization
const compatible =
!selectedModel || !device
? true
: isModelCompatible(selectedModel.supportedHardware, device.type);
useEffect(() => { useEffect(() => {
if (open) { if (open) {

View File

@ -2,6 +2,7 @@
import { BarChart, Bar, XAxis, YAxis, CartesianGrid, ResponsiveContainer, Cell } from 'recharts'; import { BarChart, Bar, XAxis, YAxis, CartesianGrid, ResponsiveContainer, Cell } from 'recharts';
import type { ClassResult } from '@/types/inference'; import type { ClassResult } from '@/types/inference';
import { classResultLabel } from '@/lib/classification';
import { useTranslation } from '@/lib/i18n'; import { useTranslation } from '@/lib/i18n';
interface ClassificationResultProps { interface ClassificationResultProps {
@ -18,8 +19,10 @@ export function ClassificationResult({ results, confidenceThreshold }: Classific
.sort((a, b) => b.confidence - a.confidence) .sort((a, b) => b.confidence - a.confidence)
.slice(0, 8); .slice(0, 8);
// Labels may be missing when the model ships without a label mapping — fall
// back to the raw class index (`class_<n>`) rather than rendering blank bars.
const data = filtered.map((r) => ({ const data = filtered.map((r) => ({
label: r.label, label: classResultLabel(r),
confidence: +(r.confidence * 100).toFixed(1), confidence: +(r.confidence * 100).toFixed(1),
})); }));
@ -32,6 +35,7 @@ export function ClassificationResult({ results, confidenceThreshold }: Classific
} }
return ( return (
<div data-testid="classification-result-chart">
<ResponsiveContainer width="100%" height={250}> <ResponsiveContainer width="100%" height={250}>
<BarChart data={data} layout="vertical" margin={{ left: 80, right: 20, top: 5, bottom: 5 }}> <BarChart data={data} layout="vertical" margin={{ left: 80, right: 20, top: 5, bottom: 5 }}>
<CartesianGrid strokeDasharray="3 3" /> <CartesianGrid strokeDasharray="3 3" />
@ -44,5 +48,6 @@ export function ClassificationResult({ results, confidenceThreshold }: Classific
</Bar> </Bar>
</BarChart> </BarChart>
</ResponsiveContainer> </ResponsiveContainer>
</div>
); );
} }

View File

@ -0,0 +1,54 @@
'use client';
import type { DetectionResult } from '@/types/inference';
import { useTranslation } from '@/lib/i18n';
interface DetectionResultListProps {
results: DetectionResult[];
confidenceThreshold: number;
}
/**
* Detected object list for the side panel.
*
* Detection results are spatial, so the primary read is the bounding boxes on
* the frame itself; this list is the secondary, textual read (what was found,
* how many, how confident) which the canvas cannot convey to assistive tech.
*/
export function DetectionResultList({ results, confidenceThreshold }: DetectionResultListProps) {
const { t } = useTranslation();
const filtered = results
.filter((d) => d.confidence >= confidenceThreshold)
.sort((a, b) => b.confidence - a.confidence);
if (filtered.length === 0) {
return (
<div className="flex h-24 items-center justify-center text-sm text-muted-foreground">
{t('inference.noResultsAboveThreshold')}
</div>
);
}
return (
<div className="space-y-2" data-testid="detection-result-list">
<div className="flex justify-between text-sm">
<span className="text-muted-foreground">{t('inference.detectedCount')}</span>
<span className="tabular-nums">{filtered.length}</span>
</div>
<ul className="space-y-1">
{filtered.map((det, i) => (
<li
key={`${det.label}-${i}`}
className="flex items-center justify-between gap-2 text-sm"
>
<span className="truncate">{det.label}</span>
<span className="shrink-0 tabular-nums text-muted-foreground">
{(det.confidence * 100).toFixed(1)}%
</span>
</li>
))}
</ul>
</div>
);
}

View File

@ -0,0 +1,227 @@
'use client';
import { useRef, useState } from 'react';
import {
Select,
SelectContent,
SelectItem,
SelectTrigger,
SelectValue,
} from '@/components/ui/select';
import { Button } from '@/components/ui/button';
import {
useInferenceOptionsStore,
validateLabelFile,
MAX_LABEL_FILE_BYTES,
ACCEPTED_LABEL_EXTENSIONS,
} from '@/stores/inference-options-store';
import { FLASH_TASK_TYPES, normalizeTaskType, type FlashTaskType } from '@/lib/task-type';
import { useTranslation } from '@/lib/i18n';
interface InferenceOptionsProps {
deviceId: string;
/**
* Task type reported by the most recent inference result. Used as the
* selector's value until the user overrides it, so the control opens showing
* what the pipeline is actually doing rather than a guess.
*/
resultTaskType: string | undefined;
}
/** Local-only errors that never reach the server (size / extension). */
type LocalError = 'too-large' | 'wrong-type' | null;
/** Preview of the first few label names, so the user can spot a wrong file. */
const LABEL_PREVIEW_COUNT = 3;
/**
* Live inference controls (M4).
*
* Lets the user swap the parsing mode and attach a display-only label mapping
* mid-session, without re-flashing the model. Sits in the right-hand
* InferencePanel next to the confidence slider the other control that
* re-interprets results already on screen so all the "how do I read this
* output" knobs live together and none of them block the video area.
*
* The label section is only rendered for classification: labels map class
* indices to names, which is meaningless for detection (whose labels come from
* the model metadata per box).
*/
export function InferenceOptions({ deviceId, resultTaskType }: InferenceOptionsProps) {
const { t } = useTranslation();
const fileInputRef = useRef<HTMLInputElement>(null);
const [localError, setLocalError] = useState<LocalError>(null);
const { taskType, labels, applying, error, setTaskType, uploadLabels, clearLabels } =
useInferenceOptionsStore();
// The user's explicit choice wins; otherwise mirror what the pipeline last
// reported. Both go through normalizeTaskType so an unknown/legacy spelling
// (plan R-4) falls through to object detection instead of rendering blank.
const effectiveTaskType: FlashTaskType = taskType ?? normalizeTaskType(resultTaskType);
const isClassification = effectiveTaskType === 'classification';
const taskTypeLabel = (value: FlashTaskType) =>
value === 'classification'
? t('devices.flash.taskTypeClassification')
: t('devices.flash.taskTypeObjectDetection');
const handleTaskTypeChange = (value: string) => {
setLocalError(null);
void setTaskType(deviceId, normalizeTaskType(value));
};
const handleFileChange = (e: React.ChangeEvent<HTMLInputElement>) => {
const file = e.target.files?.[0];
// Reset the input so picking the SAME file again still fires a change
// event — otherwise a user who fixed their labels.txt on disk and re-picked
// it would get no feedback at all.
e.target.value = '';
if (!file) return;
const check = validateLabelFile(file);
if (!check.ok) {
setLocalError(check.reason);
return;
}
setLocalError(null);
void uploadLabels(deviceId, file);
};
const handleClearLabels = () => {
setLocalError(null);
void clearLabels(deviceId);
};
const localErrorMessage =
localError === 'too-large'
? t('inference.options.labelFileTooLarge', {
limit: `${Math.round(MAX_LABEL_FILE_BYTES / 1024)} KB`,
})
: localError === 'wrong-type'
? t('inference.options.labelFileWrongType')
: null;
// Server errors carry a line number for parse failures — surfacing it is the
// whole reason parsing happens server-side.
const serverErrorMessage = error
? error.line != null
? t('inference.options.labelParseErrorLine', { line: error.line, message: error.message })
: t('inference.options.applyFailed', { message: error.message })
: null;
const errorMessage = localErrorMessage ?? serverErrorMessage;
return (
<div className="space-y-3" data-testid="inference-options">
<div className="space-y-1.5">
<label
htmlFor="inference-task-type"
className="text-sm font-medium text-muted-foreground"
>
{t('inference.options.taskType')}
</label>
<Select
value={effectiveTaskType}
onValueChange={handleTaskTypeChange}
disabled={applying}
>
<SelectTrigger
id="inference-task-type"
aria-describedby="inference-task-type-hint"
data-testid="inference-task-type-trigger"
>
<SelectValue />
</SelectTrigger>
<SelectContent>
{FLASH_TASK_TYPES.map((value) => (
<SelectItem key={value} value={value}>
{taskTypeLabel(value)}
</SelectItem>
))}
</SelectContent>
</Select>
<p id="inference-task-type-hint" className="text-xs text-muted-foreground">
{t('inference.options.taskTypeHint')}
</p>
</div>
{isClassification && (
<div className="space-y-1.5 border-t pt-3" data-testid="inference-label-section">
<p className="text-sm font-medium text-muted-foreground">
{t('inference.options.labels')}
</p>
<p className="text-xs text-muted-foreground">{t('inference.options.labelsHint')}</p>
<input
ref={fileInputRef}
type="file"
accept={ACCEPTED_LABEL_EXTENSIONS.join(',')}
className="sr-only"
onChange={handleFileChange}
data-testid="inference-label-file-input"
aria-label={t('inference.options.selectLabelFile')}
/>
<div className="flex flex-wrap gap-2">
<Button
type="button"
variant="outline"
size="sm"
disabled={applying}
onClick={() => fileInputRef.current?.click()}
data-testid="inference-label-upload-btn"
>
{labels
? t('inference.options.replaceLabelFile')
: t('inference.options.selectLabelFile')}
</Button>
{labels && (
<Button
type="button"
variant="ghost"
size="sm"
disabled={applying}
onClick={handleClearLabels}
data-testid="inference-label-clear-btn"
>
{t('inference.options.clearLabels')}
</Button>
)}
</div>
<p className="text-xs text-muted-foreground">
{t('inference.options.labelsFormatHint')}
</p>
<p
className="text-xs text-muted-foreground"
// The applied/cleared state changes as a result of an async request,
// so announce it rather than leaving screen-reader users guessing.
role="status"
aria-live="polite"
data-testid="inference-label-status"
>
{applying
? t('inference.options.applying')
: labels
? t('inference.options.labelsApplied', {
count: labels.length,
names: labels.slice(0, LABEL_PREVIEW_COUNT).join('、'),
})
: t('inference.options.noLabels')}
</p>
</div>
)}
{errorMessage && (
<p
className="text-xs text-destructive"
role="alert"
data-testid="inference-options-error"
>
{errorMessage}
</p>
)}
</div>
);
}

View File

@ -2,14 +2,21 @@
import { Card, CardContent, CardHeader, CardTitle } from '@/components/ui/card'; import { Card, CardContent, CardHeader, CardTitle } from '@/components/ui/card';
import { ClassificationResult } from './classification-result'; import { ClassificationResult } from './classification-result';
import { DetectionResultList } from './detection-result';
import { PerformanceMetrics } from './performance-metrics'; import { PerformanceMetrics } from './performance-metrics';
import { ConfidenceSlider } from './confidence-slider'; import { ConfidenceSlider } from './confidence-slider';
import { VideoProgress } from './video-progress'; import { VideoProgress } from './video-progress';
import { InferenceOptions } from './inference-options';
import { useInferenceStore } from '@/stores/inference-store'; import { useInferenceStore } from '@/stores/inference-store';
import { useCameraStore } from '@/stores/camera-store'; import { useCameraStore } from '@/stores/camera-store';
import { isClassificationTask } from '@/lib/classification';
import { useTranslation } from '@/lib/i18n'; import { useTranslation } from '@/lib/i18n';
export function InferencePanel() { interface InferencePanelProps {
deviceId: string;
}
export function InferencePanel({ deviceId }: InferencePanelProps) {
const { t } = useTranslation(); const { t } = useTranslation();
const { result, fps, avgLatency, isRunning, confidenceThreshold, batchResults } = const { result, fps, avgLatency, isRunning, confidenceThreshold, batchResults } =
useInferenceStore(); useInferenceStore();
@ -22,6 +29,10 @@ export function InferencePanel() {
? batchResults[batchSelectedIndex] ? batchResults[batchSelectedIndex]
: result; : result;
const classifications = displayResult?.classifications || []; const classifications = displayResult?.classifications || [];
const detections = displayResult?.detections || [];
// Branch on "is classification?" so detection stays the safe fall-through
// under either taskType spelling (see lib/classification.ts).
const isClassification = isClassificationTask(displayResult?.taskType);
return ( return (
<div className="w-80 space-y-4"> <div className="w-80 space-y-4">
@ -62,13 +73,33 @@ export function InferencePanel() {
<Card> <Card>
<CardHeader className="pb-2"> <CardHeader className="pb-2">
<CardTitle className="text-sm">{t('inference.classificationResults')}</CardTitle> <CardTitle className="text-sm">{t('inference.options.title')}</CardTitle>
</CardHeader> </CardHeader>
<CardContent> <CardContent>
<InferenceOptions deviceId={deviceId} resultTaskType={displayResult?.taskType} />
</CardContent>
</Card>
<Card>
<CardHeader className="pb-2">
<CardTitle className="text-sm">
{isClassification
? t('inference.classificationResults')
: t('inference.detectionResults')}
</CardTitle>
</CardHeader>
<CardContent>
{isClassification ? (
<ClassificationResult <ClassificationResult
results={classifications} results={classifications}
confidenceThreshold={confidenceThreshold} confidenceThreshold={confidenceThreshold}
/> />
) : (
<DetectionResultList
results={detections}
confidenceThreshold={confidenceThreshold}
/>
)}
</CardContent> </CardContent>
</Card> </Card>

View File

@ -0,0 +1,72 @@
'use client';
import { useState } from 'react';
import type { ClassResult } from '@/types/inference';
import {
DEFAULT_HYSTERESIS,
INITIAL_HYSTERESIS_STATE,
nextHysteresisState,
type HysteresisState,
} from '@/lib/classification';
interface UseStableTopClassOptions {
confidenceThreshold: number;
streakFrames?: number;
immediateMargin?: number;
}
interface InternalState {
hysteresis: HysteresisState;
/** The `classifications` array identity the hysteresis state was derived from. */
seenClassifications: ClassResult[] | undefined;
seenConfidenceThreshold: number;
}
const INITIAL_INTERNAL: InternalState = {
hysteresis: INITIAL_HYSTERESIS_STATE,
seenClassifications: undefined,
seenConfidenceThreshold: NaN,
};
/**
* Returns the anti-flicker top-1 class for the current frame.
*
* The hysteresis state machine advances once per inference result the store
* always hands us a fresh `classifications` array, so array identity is a
* reliable "new frame" signal. Static image sources produce a single result and
* therefore a single transition, which the state machine handles (the first
* result above the threshold is displayed immediately).
*
* Uses React's "adjust state while rendering" pattern rather than a ref or an
* effect: an effect-driven version would render one frame behind, which on a
* 15 FPS stream is visible lag, and a ref mutated during render is unsafe under
* concurrent rendering.
* See https://react.dev/reference/react/useState#storing-information-from-previous-renders
*/
export function useStableTopClass(
classifications: ClassResult[] | undefined,
{ confidenceThreshold, streakFrames, immediateMargin }: UseStableTopClassOptions,
): ClassResult | null {
const [state, setState] = useState<InternalState>(INITIAL_INTERNAL);
let current = state;
// Recompute when a new frame arrives, or when the user moves the confidence
// slider (the currently displayed label may no longer qualify).
if (
state.seenClassifications !== classifications ||
state.seenConfidenceThreshold !== confidenceThreshold
) {
current = {
hysteresis: nextHysteresisState(state.hysteresis, classifications, {
streakFrames: streakFrames ?? DEFAULT_HYSTERESIS.streakFrames,
immediateMargin: immediateMargin ?? DEFAULT_HYSTERESIS.immediateMargin,
confidenceThreshold,
}),
seenClassifications: classifications,
seenConfidenceThreshold: confidenceThreshold,
};
setState(current);
}
return current.hysteresis.displayed;
}

View File

@ -1,12 +1,22 @@
import { getApiBaseUrl, getRelayToken, fetchAndCacheRelayToken } from './constants'; import { getApiBaseUrl, getRelayToken, fetchAndCacheRelayToken } from './constants';
export interface ApiResponse<T> { export interface ApiError {
success: boolean;
data?: T;
error?: {
code: string; code: string;
message: string; message: string;
}; }
/**
* Standard response envelope.
*
* `E` widens the error shape for endpoints that attach extra diagnostics the
* label upload returns a `line` number on parse failures, and that field has to
* be declared rather than cast in, or the type system stops protecting the one
* piece of information the user actually needs.
*/
export interface ApiResponse<T, E extends ApiError = ApiError> {
success: boolean;
data?: T;
error?: E;
} }
// Ensure relay token is available before making API requests. // Ensure relay token is available before making API requests.
@ -40,7 +50,10 @@ function buildHeaders(): Record<string, string> {
}; };
} }
async function request<T>(path: string, options?: RequestInit): Promise<ApiResponse<T>> { async function request<T, E extends ApiError = ApiError>(
path: string,
options?: RequestInit,
): Promise<ApiResponse<T, E>> {
// Wait for relay token to be available before first request // Wait for relay token to be available before first request
await ensureRelayToken(); await ensureRelayToken();
@ -51,10 +64,32 @@ async function request<T>(path: string, options?: RequestInit): Promise<ApiRespo
return res.json(); return res.json();
} }
/**
* Multipart POST.
*
* Deliberately does NOT go through `buildHeaders()`: setting Content-Type
* manually would omit the multipart boundary that `fetch` generates from the
* FormData body, and the server would fail to parse the request.
*/
async function postForm<T, E extends ApiError = ApiError>(
path: string,
form: FormData,
): Promise<ApiResponse<T, E>> {
await ensureRelayToken();
const res = await fetch(`${getApiBaseUrl()}${path}`, {
method: 'POST',
headers: getRelayHeaders(),
body: form,
});
return res.json();
}
export const api = { export const api = {
get: <T>(path: string) => request<T>(path), get: <T>(path: string) => request<T>(path),
post: <T>(path: string, body?: unknown) => post: <T, E extends ApiError = ApiError>(path: string, body?: unknown) =>
request<T>(path, { method: 'POST', body: body ? JSON.stringify(body) : undefined }), request<T, E>(path, { method: 'POST', body: body ? JSON.stringify(body) : undefined }),
postForm,
put: <T>(path: string, body?: unknown) => put: <T>(path: string, body?: unknown) =>
request<T>(path, { method: 'PUT', body: body ? JSON.stringify(body) : undefined }), request<T>(path, { method: 'PUT', body: body ? JSON.stringify(body) : undefined }),
del: <T>(path: string) => request<T>(path, { method: 'DELETE' }), del: <T>(path: string) => request<T>(path, { method: 'DELETE' }),

View File

@ -0,0 +1,146 @@
import type { ClassResult } from '@/types/inference';
/**
* Defensive taskType check.
*
* The taskType value set has historically been inconsistent across the stack:
* the Python bridge used to emit `"detection"` while models.json / the
* frontend TASK_TYPES constant use `"object_detection"` (see plan R-4).
*
* We therefore only ever branch on "is this classification?" anything else
* (including `undefined`) falls through to the existing detection rendering
* path, which is the safe default: a detection model rendered as detection is
* correct, whereas a mis-detected classification just means no overlay label.
*/
export function isClassificationTask(taskType: string | undefined | null): boolean {
return taskType === 'classification';
}
/**
* Display label for a class result.
*
* `classIndex` is optional because the Go layer may not forward it (M2-d is a
* "nice to have"). When the label is missing/blank we fall back to
* `class_<index>`, and if we have neither we emit a stable placeholder rather
* than rendering an empty chip.
*/
export function classResultLabel(result: Pick<ClassResult, 'label' | 'classIndex'>): string {
const label = result.label?.trim();
if (label) return label;
if (typeof result.classIndex === 'number' && Number.isFinite(result.classIndex)) {
return `class_${result.classIndex}`;
}
return 'class_?';
}
/**
* Stable identity for a class result, used by the hysteresis state machine to
* decide whether two consecutive frames refer to the same class.
*
* Prefers `classIndex` (immune to label churn when labels are re-uploaded) and
* falls back to the label string when the index is absent.
*/
export function classResultKey(result: Pick<ClassResult, 'label' | 'classIndex'>): string {
if (typeof result.classIndex === 'number' && Number.isFinite(result.classIndex)) {
return `i:${result.classIndex}`;
}
return `l:${classResultLabel(result)}`;
}
/** Highest-confidence entry, or null when there is nothing to show. */
export function pickTopClass(results: ClassResult[] | undefined): ClassResult | null {
if (!results || results.length === 0) return null;
let top = results[0];
for (let i = 1; i < results.length; i++) {
if (results[i].confidence > top.confidence) top = results[i];
}
return top;
}
export interface HysteresisState {
/** Currently displayed class (null = nothing displayed yet). */
displayed: ClassResult | null;
/** Candidate class that is trying to replace `displayed`. */
candidateKey: string | null;
/** How many consecutive frames the candidate has been the top class. */
candidateStreak: number;
}
export interface HysteresisOptions {
/**
* Frames the challenger must stay on top before it replaces the displayed
* class. 3 @ ~15 FPS 200 ms long enough to swallow single-frame noise,
* short enough that a genuine gesture change still feels instant.
*/
streakFrames: number;
/**
* Confidence margin that lets a challenger switch immediately, bypassing the
* streak counter. 0.15 means "clearly more confident, not a coin flip".
*/
immediateMargin: number;
/**
* Below this confidence nothing is displayed. Mirrors the user-facing
* confidence slider so the overlay agrees with the side panel.
*/
confidenceThreshold: number;
}
export const DEFAULT_HYSTERESIS: Pick<HysteresisOptions, 'streakFrames' | 'immediateMargin'> = {
streakFrames: 3,
immediateMargin: 0.15,
};
export const INITIAL_HYSTERESIS_STATE: HysteresisState = {
displayed: null,
candidateKey: null,
candidateStreak: 0,
};
/**
* Anti-flicker state machine for the top-1 classification label.
*
* Video/camera sources re-classify every frame, so a 50/50 boundary makes the
* raw top-1 oscillate wildly (plan risk R-9). A frame is only promoted when it
* either (a) beats the incumbent by a clear confidence margin, or (b) has been
* the top class for `streakFrames` consecutive frames.
*
* Pure function: takes the previous state, returns the next one. No React,
* no timers which makes it directly unit-testable.
*/
export function nextHysteresisState(
prev: HysteresisState,
results: ClassResult[] | undefined,
options: HysteresisOptions,
): HysteresisState {
const { streakFrames, immediateMargin, confidenceThreshold } = options;
const top = pickTopClass(results);
// Nothing above the threshold → clear immediately. Showing a stale label over
// a frame the model no longer recognises is worse than showing nothing.
if (!top || top.confidence < confidenceThreshold) {
return INITIAL_HYSTERESIS_STATE;
}
// First result, or the incumbent already matches → refresh confidence in place.
if (!prev.displayed) {
return { displayed: top, candidateKey: null, candidateStreak: 0 };
}
const topKey = classResultKey(top);
if (topKey === classResultKey(prev.displayed)) {
return { displayed: top, candidateKey: null, candidateStreak: 0 };
}
// Clear winner → switch without waiting for the streak.
if (top.confidence - prev.displayed.confidence >= immediateMargin) {
return { displayed: top, candidateKey: null, candidateStreak: 0 };
}
const streak = prev.candidateKey === topKey ? prev.candidateStreak + 1 : 1;
if (streak >= streakFrames) {
return { displayed: top, candidateKey: null, candidateStreak: 0 };
}
// Challenger not convincing enough yet — keep showing the incumbent.
return { displayed: prev.displayed, candidateKey: topKey, candidateStreak: streak };
}

View File

@ -125,6 +125,12 @@ export const en: TranslationDict = {
flashFailed: 'Flash Failed', flashFailed: 'Flash Failed',
preparingFlash: 'Preparing flash...', preparingFlash: 'Preparing flash...',
flashComplete: 'Flash complete!', flashComplete: 'Flash complete!',
selectModelFirst: 'Select a model first',
// Display names for the inference types. The flash dialog no longer
// selects a type, but the inference page's InferenceOptions still shares
// these two labels, so they stay.
taskTypeObjectDetection: 'Object Detection',
taskTypeClassification: 'Classification',
}, },
card: { card: {
fwBadge: { fwBadge: {
@ -234,6 +240,9 @@ export const en: TranslationDict = {
confidenceFilter: 'Confidence Filter', confidenceFilter: 'Confidence Filter',
confidenceThreshold: 'Confidence Threshold', confidenceThreshold: 'Confidence Threshold',
classificationResults: 'Classification Results', classificationResults: 'Classification Results',
detectionResults: 'Detection Results',
detectedCount: 'Detected',
unrecognized: 'Unrecognized',
noResultsAboveThreshold: 'No results above threshold', noResultsAboveThreshold: 'No results above threshold',
details: 'Details', details: 'Details',
model: 'Model', model: 'Model',
@ -250,6 +259,27 @@ export const en: TranslationDict = {
videoProgress: 'Video Progress', videoProgress: 'Video Progress',
frames: 'frames', frames: 'frames',
framesProcessed: 'Frames Processed', framesProcessed: 'Frames Processed',
options: {
title: 'Inference Options',
taskType: 'Parsing Mode',
taskTypeHint:
'Applies to subsequent results immediately — no need to re-flash the model.',
applying: 'Applying...',
applyFailed: 'Could not apply: {message}',
labels: 'Label Mapping',
labelsHint:
'Optional. Without a label file the raw class indices are shown (class_0, class_1…).',
labelsFormatHint: 'One "<index> <name>" per line, e.g. 0 scissors',
selectLabelFile: 'Upload label file',
replaceLabelFile: 'Replace label file',
clearLabels: 'Clear labels',
labelsApplied: 'Applied {count} labels ({names})',
labelsCleared: 'Labels cleared — raw class indices will be shown',
noLabels: 'No label file uploaded',
labelFileTooLarge: 'Label file is too large (limit {limit})',
labelFileWrongType: 'Only .txt or .names files are accepted',
labelParseErrorLine: 'Line {line}: {message}',
},
}, },
settings: { settings: {
title: 'Settings', title: 'Settings',

View File

@ -123,6 +123,10 @@ export interface TranslationDict {
flashFailed: string; flashFailed: string;
preparingFlash: string; preparingFlash: string;
flashComplete: string; flashComplete: string;
selectModelFirst: string;
// Shared with the inference page's InferenceOptions selector.
taskTypeObjectDetection: string;
taskTypeClassification: string;
}; };
card: { card: {
fwBadge: { fwBadge: {
@ -232,6 +236,9 @@ export interface TranslationDict {
confidenceFilter: string; confidenceFilter: string;
confidenceThreshold: string; confidenceThreshold: string;
classificationResults: string; classificationResults: string;
detectionResults: string;
detectedCount: string;
unrecognized: string;
noResultsAboveThreshold: string; noResultsAboveThreshold: string;
details: string; details: string;
model: string; model: string;
@ -248,6 +255,25 @@ export interface TranslationDict {
videoProgress: string; videoProgress: string;
frames: string; frames: string;
framesProcessed: string; framesProcessed: string;
options: {
title: string;
taskType: string;
taskTypeHint: string;
applying: string;
applyFailed: string;
labels: string;
labelsHint: string;
labelsFormatHint: string;
selectLabelFile: string;
replaceLabelFile: string;
clearLabels: string;
labelsApplied: string;
labelsCleared: string;
noLabels: string;
labelFileTooLarge: string;
labelFileWrongType: string;
labelParseErrorLine: string;
};
}; };
settings: { settings: {
title: string; title: string;

View File

@ -125,6 +125,11 @@ export const zhTW: TranslationDict = {
flashFailed: '燒錄失敗', flashFailed: '燒錄失敗',
preparingFlash: '準備燒錄中...', preparingFlash: '準備燒錄中...',
flashComplete: '燒錄完成!', flashComplete: '燒錄完成!',
selectModelFirst: '請先選擇模型',
// 推論種類的顯示名稱。燒錄對話框已不再選推論種類,但推論頁的
// InferenceOptions 仍共用這兩個標籤,故保留。
taskTypeObjectDetection: '物件偵測',
taskTypeClassification: '分類',
}, },
card: { card: {
fwBadge: { fwBadge: {
@ -234,6 +239,9 @@ export const zhTW: TranslationDict = {
confidenceFilter: '信心度篩選', confidenceFilter: '信心度篩選',
confidenceThreshold: '信心度門檻', confidenceThreshold: '信心度門檻',
classificationResults: '分類結果', classificationResults: '分類結果',
detectionResults: '偵測結果',
detectedCount: '偵測數量',
unrecognized: '無法辨識',
noResultsAboveThreshold: '沒有超過門檻的結果', noResultsAboveThreshold: '沒有超過門檻的結果',
details: '詳細資訊', details: '詳細資訊',
model: '模型', model: '模型',
@ -250,6 +258,25 @@ export const zhTW: TranslationDict = {
videoProgress: '影片進度', videoProgress: '影片進度',
frames: '幀', frames: '幀',
framesProcessed: '已處理幀數', framesProcessed: '已處理幀數',
options: {
title: '推論設定',
taskType: '解析方式',
taskTypeHint: '切換後立即套用到之後的推論結果,不需要重新載入模型。',
applying: '套用中...',
applyFailed: '套用失敗:{message}',
labels: '標籤對照',
labelsHint: '選用。沒有上傳時會顯示原始類別編號class_0、class_1…。',
labelsFormatHint: '每行一筆「編號 名稱」例如0 剪刀',
selectLabelFile: '上傳標籤檔',
replaceLabelFile: '更換標籤檔',
clearLabels: '清除標籤',
labelsApplied: '已套用 {count} 個標籤({names}',
labelsCleared: '已清除標籤,改用原始類別編號',
noLabels: '尚未上傳標籤檔',
labelFileTooLarge: '標籤檔過大(上限 {limit}',
labelFileWrongType: '只接受 .txt 或 .names 檔',
labelParseErrorLine: '第 {line} 行:{message}',
},
}, },
settings: { settings: {
title: '設定', title: '設定',

View File

@ -0,0 +1,29 @@
/**
* Task types the inference pipeline can actually parse.
*
* models.json also carries `segmentation` / `pose_estimation` (the upload form
* offers them), but neither the Python bridge nor the frontend has a rendering
* path for those yet. The inference-page selector therefore only exposes the
* two types that produce meaningful output.
*
* Naming note: the `FLASH_` prefix is historical the selector originally
* lived in the flash dialog. It now only backs the inference page's runtime
* switch (`components/inference/inference-options.tsx`).
*/
export const FLASH_TASK_TYPES = ['object_detection', 'classification'] as const;
export type FlashTaskType = (typeof FLASH_TASK_TYPES)[number];
/**
* Normalise an arbitrary taskType string to one of the two supported values.
*
* The value set has historically been inconsistent across the stack: the Python
* bridge used to emit `"detection"` while models.json and the frontend
* TASK_TYPES constant use `"object_detection"` (plan R-4). We therefore only
* ever test for classification and let everything else including `undefined`,
* `segmentation` and legacy `detection` fall through to object detection,
* which is the existing safe default.
*/
export function normalizeTaskType(taskType: string | undefined | null): FlashTaskType {
return taskType === 'classification' ? 'classification' : 'object_detection';
}

View File

@ -0,0 +1,198 @@
import { create } from 'zustand';
import { api } from '@/lib/api';
import { normalizeTaskType, type FlashTaskType } from '@/lib/task-type';
import type { ApiResponse } from '@/lib/api';
/**
* Live inference options the parsing mode and the display-only label mapping
* that can be changed WITHOUT re-flashing the model (M4).
*
* The whole HTTP contract lives in this file on purpose. The backend endpoint
* is being built in parallel, so keeping every request shape in one module
* means realigning with the server is a single-file edit rather than a hunt
* through components.
*
* Contract as implemented here:
*
* POST /api/devices/:id/inference/options
* Content-Type: application/json
* { "taskType": "object_detection" | "classification" }
*
* POST /api/devices/:id/inference/options (label upload)
* Content-Type: multipart/form-data
* labelFile: <labels.txt> // `<index> <name>` per line
*
* POST /api/devices/:id/inference/options (clear the mapping)
* Content-Type: application/json
* { "labels": [] } // explicit empty array; omitting the field
* // instead means "leave labels untouched"
*
* All return the standard envelope:
* { success: true, data: { deviceId, taskType, labelCount, labels, maxIndex } }
* { success: false, error: { code, message, line? } }
*
* The server rejects a request that carries neither `taskType` nor `labels`,
* so every action below sends at least one of them.
*
* Nothing is persisted: the user re-uploads the label file per session, which
* is what they asked for ("不用記,每次現場傳").
*/
/** Matches the plan §3.5 cap (3 classes ≈ 30 bytes; 10k classes < 200 KB). */
export const MAX_LABEL_FILE_BYTES = 256 * 1024;
/** Extensions accepted client-side. Content validation happens server-side. */
export const ACCEPTED_LABEL_EXTENSIONS = ['.txt', '.names'] as const;
export interface InferenceOptionsResponse {
taskType?: string;
labelCount?: number;
labels?: string[];
}
/**
* Error envelope for label parsing. `line` is optional and only present for
* `LABEL_PARSE_ERROR`, which is the whole point of letting the server parse:
* it can point at the offending line.
*/
export interface InferenceOptionsError {
code: string;
message: string;
line?: number;
}
interface InferenceOptionsState {
/**
* `null` = the user has not overridden anything this session, so the mode
* flashed with the model is still in effect. Components resolve the value to
* display by falling back to the last inference result's taskType.
*/
taskType: FlashTaskType | null;
labels: string[] | null;
applying: boolean;
error: InferenceOptionsError | null;
setTaskType: (deviceId: string, taskType: FlashTaskType) => Promise<boolean>;
uploadLabels: (deviceId: string, file: File) => Promise<boolean>;
clearLabels: (deviceId: string) => Promise<boolean>;
clearError: () => void;
reset: () => void;
}
function optionsPath(deviceId: string) {
return `/devices/${deviceId}/inference/options`;
}
type OptionsResponse = ApiResponse<InferenceOptionsResponse, InferenceOptionsError>;
/** Normalises a failed response into a fully-populated error envelope. */
function toError(res: OptionsResponse): InferenceOptionsError {
return {
code: res.error?.code || 'UNKNOWN',
message: res.error?.message || 'Request failed',
line: res.error?.line,
};
}
export const useInferenceOptionsStore = create<InferenceOptionsState>((set) => ({
taskType: null,
labels: null,
applying: false,
error: null,
setTaskType: async (deviceId, taskType) => {
set({ applying: true, error: null });
try {
const res = await api.post<InferenceOptionsResponse, InferenceOptionsError>(
optionsPath(deviceId),
{ taskType },
);
if (!res.success) {
set({ applying: false, error: toError(res) });
return false;
}
// Trust the server's echo when it sends one, so a server-side coercion
// never leaves the UI showing a mode that is not actually in effect.
set({
applying: false,
taskType: res.data?.taskType ? normalizeTaskType(res.data.taskType) : taskType,
});
return true;
} catch (e) {
set({
applying: false,
error: { code: 'NETWORK_ERROR', message: e instanceof Error ? e.message : String(e) },
});
return false;
}
},
uploadLabels: async (deviceId, file) => {
set({ applying: true, error: null });
try {
const form = new FormData();
form.append('labelFile', file);
const res = await api.postForm<InferenceOptionsResponse, InferenceOptionsError>(
optionsPath(deviceId),
form,
);
if (!res.success) {
set({ applying: false, error: toError(res) });
return false;
}
set({ applying: false, labels: res.data?.labels ?? [] });
return true;
} catch (e) {
set({
applying: false,
error: { code: 'NETWORK_ERROR', message: e instanceof Error ? e.message : String(e) },
});
return false;
}
},
clearLabels: async (deviceId) => {
set({ applying: true, error: null });
try {
// An explicit empty array clears the mapping. Omitting the field would
// mean "leave labels as they are", so the array must actually be sent.
const res = await api.post<InferenceOptionsResponse, InferenceOptionsError>(
optionsPath(deviceId),
{ labels: [] },
);
if (!res.success) {
set({ applying: false, error: toError(res) });
return false;
}
set({ applying: false, labels: null });
return true;
} catch (e) {
set({
applying: false,
error: { code: 'NETWORK_ERROR', message: e instanceof Error ? e.message : String(e) },
});
return false;
}
},
clearError: () => set({ error: null }),
reset: () => set({ taskType: null, labels: null, applying: false, error: null }),
}));
/**
* Client-side pre-checks. The server validates again this only spares the
* user a round trip for the two mistakes that need no parsing to detect.
*/
export function validateLabelFile(
file: File,
): { ok: true } | { ok: false; reason: 'too-large' | 'wrong-type' } {
const lower = file.name.toLowerCase();
if (!ACCEPTED_LABEL_EXTENSIONS.some((ext) => lower.endsWith(ext))) {
return { ok: false, reason: 'wrong-type' };
}
if (file.size > MAX_LABEL_FILE_BYTES) {
return { ok: false, reason: 'too-large' };
}
return { ok: true };
}

View File

@ -0,0 +1,54 @@
import { describe, it, expect } from 'vitest';
import { render, screen } from '@testing-library/react';
import { DetectionResultList } from '@/components/inference/detection-result';
import type { DetectionResult } from '@/types/inference';
function det(label: string, confidence: number): DetectionResult {
return { label, confidence, bbox: { x: 0.1, y: 0.1, width: 0.2, height: 0.2 } };
}
describe('DetectionResultList — M3-b detection side panel', () => {
it('lists detections sorted by confidence descending', () => {
render(
<DetectionResultList
results={[det('car', 0.6), det('person', 0.9), det('dog', 0.75)]}
confidenceThreshold={0.5}
/>,
);
const items = screen.getAllByRole('listitem').map((li) => li.textContent);
expect(items[0]).toContain('person');
expect(items[1]).toContain('dog');
expect(items[2]).toContain('car');
});
it('filters out detections below the confidence threshold', () => {
render(
<DetectionResultList
results={[det('person', 0.9), det('noise', 0.1)]}
confidenceThreshold={0.5}
/>,
);
expect(screen.getAllByRole('listitem')).toHaveLength(1);
expect(screen.queryByText('noise')).toBeNull();
});
it('shows the detected count', () => {
render(
<DetectionResultList
results={[det('person', 0.9), det('car', 0.8)]}
confidenceThreshold={0.5}
/>,
);
expect(screen.getByTestId('detection-result-list').textContent).toContain('2');
});
it('shows the empty state when nothing clears the threshold', () => {
render(<DetectionResultList results={[det('person', 0.1)]} confidenceThreshold={0.5} />);
expect(screen.queryByTestId('detection-result-list')).toBeNull();
});
it('shows the empty state for an empty result set', () => {
render(<DetectionResultList results={[]} confidenceThreshold={0.5} />);
expect(screen.queryByTestId('detection-result-list')).toBeNull();
});
});

View File

@ -0,0 +1,191 @@
import { describe, it, expect, beforeEach, vi } from 'vitest';
import { render, screen, fireEvent, waitFor } from '@testing-library/react';
import { FlashDialog } from '@/components/devices/flash-dialog';
import { useModelStore } from '@/stores/model-store';
import { useFlashStore } from '@/stores/flash-store';
import { useDeviceStore } from '@/stores/device-store';
import { api } from '@/lib/api';
import type { ModelSummary } from '@/types/model';
vi.mock('@/lib/api', () => ({
api: {
get: vi.fn().mockResolvedValue({ success: true, data: { models: [], total: 0 } }),
post: vi.fn().mockResolvedValue({ success: true }),
},
getRelayHeaders: vi.fn().mockReturnValue({}),
}));
vi.mock('@/lib/toast', () => ({
showSuccess: vi.fn(),
showError: vi.fn(),
showApiError: vi.fn(),
}));
// The dialog opens a flash-progress WebSocket before POSTing. Stub it so
// `connectAndWait` resolves immediately instead of hitting the 3s timeout.
//
// The returned callbacks must keep a STABLE identity across renders: the real
// hook wraps them in useCallback, and the dialog lists `disconnect` in a
// useEffect dependency array. Returning fresh functions each render re-fires
// that effect on every render and blows the React update-depth limit.
const connectAndWaitMock = vi.fn().mockResolvedValue(undefined);
const disconnectMock = vi.fn();
vi.mock('@/hooks/use-flash-progress', () => ({
useFlashProgress: () => ({
connectAndWait: connectAndWaitMock,
disconnect: disconnectMock,
}),
}));
const DEVICE_ID = 'dev-1';
function model(overrides: Partial<ModelSummary> & Pick<ModelSummary, 'id' | 'name'>): ModelSummary {
return {
thumbnail: '',
taskType: 'object_detection',
categories: [],
modelSize: 1024,
accuracy: 0.9,
fps: 30,
supportedHardware: ['KL520'],
...overrides,
};
}
const DETECTION_MODEL = model({ id: 'm-det', name: '物件辨識', taskType: 'object_detection' });
const CLASSIFICATION_MODEL = model({ id: 'm-cls', name: '手勢分類', taskType: 'classification' });
// Declares a different hardware family than the seeded KL520 device, so the
// incompatibility path can be exercised.
const INCOMPATIBLE_MODEL = model({ id: 'm-720', name: '高階模型', supportedHardware: ['KL720'] });
// Stable identity — replacing this per-test would change the dialog's effect
// dependencies on every render and spin the component into an update loop.
const noopFetchModels = async () => {};
beforeEach(() => {
vi.clearAllMocks();
// clearAllMocks() also drops the resolved value, which would make
// connectAndWait return undefined and break the await in handleFlash.
connectAndWaitMock.mockResolvedValue(undefined);
vi.mocked(api.post).mockResolvedValue({ success: true });
// fetchModels() runs when the dialog opens; with the mocked api it would
// overwrite the seeded list with []. Stub it once with a stable identity so
// the dialog's `useEffect([open, fetchModels, ...])` does not re-fire.
useModelStore.setState({
models: [DETECTION_MODEL, CLASSIFICATION_MODEL, INCOMPATIBLE_MODEL],
fetchModels: noopFetchModels,
});
useFlashStore.setState({
activeDeviceId: null,
isFlashing: false,
progress: null,
error: null,
lastFlashParams: null,
});
useDeviceStore.setState({
devices: [{ id: DEVICE_ID, type: 'KL520' } as ReturnType<
typeof useDeviceStore.getState
>['devices'][number]],
});
});
/**
* Radix Select cannot be opened with a plain click under jsdom (it depends on
* pointer capture), but its keyboard path works. Enter opens the listbox and a
* click on the rendered option commits the value.
*/
async function selectOption(trigger: HTMLElement, optionName: string) {
fireEvent.keyDown(trigger, { key: 'Enter', code: 'Enter' });
fireEvent.click(await screen.findByRole('option', { name: optionName }));
}
/** Opens the dialog and picks a model by its visible name. */
async function openAndSelectModel(name: string) {
render(<FlashDialog deviceId={DEVICE_ID} />);
fireEvent.click(screen.getByRole('button', { name: '載入模型' }));
const triggers = await screen.findAllByRole('combobox');
await selectOption(triggers[0], name);
}
describe('FlashDialog — model selection', () => {
it('cannot submit until a model is chosen', () => {
render(<FlashDialog deviceId={DEVICE_ID} />);
fireEvent.click(screen.getByRole('button', { name: '載入模型' }));
expect(screen.getByRole('button', { name: '請先選擇模型' })).toBeDisabled();
});
it('offers exactly one selector — the model dropdown', async () => {
// Guards the removal of the flash-time inference-type selector: parsing is
// now switched on the inference page, so a second combobox here would mean
// the duplicate entry point came back.
await openAndSelectModel('物件辨識');
expect(screen.getAllByRole('combobox')).toHaveLength(1);
});
it('enables submitting once a compatible model is selected', async () => {
await openAndSelectModel('物件辨識');
expect(screen.getByRole('button', { name: '開始載入' })).toBeEnabled();
});
});
describe('FlashDialog — hardware compatibility', () => {
it('warns and blocks flashing a model the device does not support', async () => {
await openAndSelectModel('高階模型');
expect(screen.getByText('硬體不相容')).toBeInTheDocument();
expect(screen.getByRole('button', { name: '不相容 — 無法載入' })).toBeDisabled();
});
it('shows no incompatibility warning for a supported model', async () => {
await openAndSelectModel('物件辨識');
expect(screen.queryByText('硬體不相容')).toBeNull();
});
});
describe('FlashDialog — flash request payload', () => {
it('posts only the model id', async () => {
await openAndSelectModel('物件辨識');
fireEvent.click(screen.getByRole('button', { name: '開始載入' }));
await waitFor(() => expect(api.post).toHaveBeenCalled());
expect(api.post).toHaveBeenCalledWith(`/devices/${DEVICE_ID}/flash`, { modelId: 'm-det' });
});
it('posts only the model id for a classification model too', async () => {
// The model's declared task type must never leak into the flash body: the
// server resolves it from models.json on its own.
await openAndSelectModel('手勢分類');
fireEvent.click(screen.getByRole('button', { name: '開始載入' }));
await waitFor(() => expect(api.post).toHaveBeenCalled());
expect(api.post).toHaveBeenCalledWith(`/devices/${DEVICE_ID}/flash`, { modelId: 'm-cls' });
});
it('waits for the progress socket before posting', async () => {
await openAndSelectModel('物件辨識');
fireEvent.click(screen.getByRole('button', { name: '開始載入' }));
await waitFor(() => expect(api.post).toHaveBeenCalled());
expect(connectAndWaitMock).toHaveBeenCalled();
});
it('does not post when the progress socket fails to connect', async () => {
connectAndWaitMock.mockRejectedValueOnce(new Error('socket down'));
await openAndSelectModel('物件辨識');
fireEvent.click(screen.getByRole('button', { name: '開始載入' }));
await waitFor(() => expect(useFlashStore.getState().error).toBe('socket down'));
expect(api.post).not.toHaveBeenCalled();
});
it('replays the same payload on retry', async () => {
await openAndSelectModel('物件辨識');
fireEvent.click(screen.getByRole('button', { name: '開始載入' }));
await waitFor(() => expect(api.post).toHaveBeenCalledTimes(1));
vi.mocked(api.post).mockClear();
await useFlashStore.getState().retryFlash();
expect(api.post).toHaveBeenCalledWith(`/devices/${DEVICE_ID}/flash`, { modelId: 'm-det' });
});
});

View File

@ -0,0 +1,345 @@
import { describe, it, expect, beforeEach, vi } from 'vitest';
import { render, screen, fireEvent, waitFor } from '@testing-library/react';
import { InferenceOptions } from '@/components/inference/inference-options';
import {
useInferenceOptionsStore,
MAX_LABEL_FILE_BYTES,
type InferenceOptionsResponse,
type InferenceOptionsError,
} from '@/stores/inference-options-store';
import { api, type ApiResponse } from '@/lib/api';
/**
* The store calls `api.post`/`api.postForm` with the widened error type so a
* parse error keeps its `line` number. `vi.mocked` resolves the generic to its
* default `ApiError`, so responses are built through this helper to stay in the
* shape the store actually receives.
*/
type OptionsResponse = ApiResponse<InferenceOptionsResponse, InferenceOptionsError>;
const reply = (r: OptionsResponse) => r as ApiResponse<InferenceOptionsResponse>;
vi.mock('@/lib/api', () => ({
api: {
get: vi.fn().mockResolvedValue({ success: true, data: {} }),
post: vi.fn().mockResolvedValue({ success: true, data: {} }),
postForm: vi.fn().mockResolvedValue({ success: true, data: {} }),
},
getRelayHeaders: vi.fn().mockReturnValue({}),
}));
const DEVICE_ID = 'dev-1';
const OPTIONS_PATH = `/devices/${DEVICE_ID}/inference/options`;
beforeEach(() => {
vi.clearAllMocks();
vi.mocked(api.post).mockResolvedValue({ success: true, data: {} });
vi.mocked(api.postForm).mockResolvedValue({ success: true, data: {} });
useInferenceOptionsStore.getState().reset();
});
/**
* Radix Select cannot be opened by a plain click under jsdom (pointer capture),
* but the keyboard path works: Enter opens the listbox, a click commits.
*/
async function selectOption(trigger: HTMLElement, optionName: string) {
fireEvent.keyDown(trigger, { key: 'Enter', code: 'Enter' });
fireEvent.click(await screen.findByRole('option', { name: optionName }));
}
function taskTypeTrigger() {
return screen.getByTestId('inference-task-type-trigger');
}
function labelFile(name = 'labels.txt', content = '0 剪刀\n1 石頭\n2 布\n') {
return new File([content], name, { type: 'text/plain' });
}
function uploadLabelFile(file: File) {
const input = screen.getByTestId('inference-label-file-input') as HTMLInputElement;
fireEvent.change(input, { target: { files: [file] } });
}
describe('InferenceOptions — parsing mode selector', () => {
it('mirrors the task type reported by the latest result', () => {
render(<InferenceOptions deviceId={DEVICE_ID} resultTaskType="classification" />);
expect(taskTypeTrigger()).toHaveTextContent('分類');
});
it('shows object detection for a detection result', () => {
render(<InferenceOptions deviceId={DEVICE_ID} resultTaskType="object_detection" />);
expect(taskTypeTrigger()).toHaveTextContent('物件偵測');
});
it('falls back to object detection for the legacy "detection" spelling (R-4)', () => {
render(<InferenceOptions deviceId={DEVICE_ID} resultTaskType="detection" />);
expect(taskTypeTrigger()).toHaveTextContent('物件偵測');
});
it('falls back to object detection when no result has arrived yet', () => {
render(<InferenceOptions deviceId={DEVICE_ID} resultTaskType={undefined} />);
expect(taskTypeTrigger()).toHaveTextContent('物件偵測');
});
it('POSTs the newly chosen task type', async () => {
render(<InferenceOptions deviceId={DEVICE_ID} resultTaskType="object_detection" />);
await selectOption(taskTypeTrigger(), '分類');
await waitFor(() => expect(api.post).toHaveBeenCalled());
expect(api.post).toHaveBeenCalledWith(OPTIONS_PATH, { taskType: 'classification' });
});
it('keeps showing the user choice after it is applied, not the stale result type', async () => {
// The prop still says object_detection (no new frame has arrived yet). The
// explicit choice must win, otherwise the selector snaps back and looks
// like the switch failed.
render(<InferenceOptions deviceId={DEVICE_ID} resultTaskType="object_detection" />);
await selectOption(taskTypeTrigger(), '分類');
await waitFor(() => expect(taskTypeTrigger()).toHaveTextContent('分類'));
});
it('adopts the task type echoed back by the server', async () => {
vi.mocked(api.post).mockResolvedValue({
success: true,
data: { taskType: 'object_detection' },
});
render(<InferenceOptions deviceId={DEVICE_ID} resultTaskType="object_detection" />);
await selectOption(taskTypeTrigger(), '分類');
// Server refused to switch and said "still object_detection" — the UI must
// not claim classification is in effect.
await waitFor(() => expect(taskTypeTrigger()).toHaveTextContent('物件偵測'));
});
it('surfaces a failed switch', async () => {
vi.mocked(api.post).mockResolvedValue({
success: false,
error: { code: 'DEVICE_NOT_CONNECTED', message: 'device not connected' },
});
render(<InferenceOptions deviceId={DEVICE_ID} resultTaskType="object_detection" />);
await selectOption(taskTypeTrigger(), '分類');
const err = await screen.findByTestId('inference-options-error');
expect(err.textContent).toContain('device not connected');
});
});
describe('InferenceOptions — label section visibility', () => {
it('hides the label section for object detection', () => {
render(<InferenceOptions deviceId={DEVICE_ID} resultTaskType="object_detection" />);
expect(screen.queryByTestId('inference-label-section')).toBeNull();
});
it('shows the label section for classification', () => {
render(<InferenceOptions deviceId={DEVICE_ID} resultTaskType="classification" />);
expect(screen.getByTestId('inference-label-section')).toBeInTheDocument();
});
it('reveals the label section once the user switches to classification', async () => {
render(<InferenceOptions deviceId={DEVICE_ID} resultTaskType="object_detection" />);
expect(screen.queryByTestId('inference-label-section')).toBeNull();
await selectOption(taskTypeTrigger(), '分類');
await waitFor(() =>
expect(screen.getByTestId('inference-label-section')).toBeInTheDocument(),
);
});
it('hides the label section again when switching back to detection', async () => {
render(<InferenceOptions deviceId={DEVICE_ID} resultTaskType="classification" />);
expect(screen.getByTestId('inference-label-section')).toBeInTheDocument();
await selectOption(taskTypeTrigger(), '物件偵測');
await waitFor(() => expect(screen.queryByTestId('inference-label-section')).toBeNull());
});
});
describe('InferenceOptions — label upload', () => {
it('uploads the picked file as multipart to the options endpoint', async () => {
render(<InferenceOptions deviceId={DEVICE_ID} resultTaskType="classification" />);
uploadLabelFile(labelFile());
await waitFor(() => expect(api.postForm).toHaveBeenCalled());
const [path, form] = vi.mocked(api.postForm).mock.calls[0];
expect(path).toBe(OPTIONS_PATH);
expect(form).toBeInstanceOf(FormData);
expect((form.get('labelFile') as File).name).toBe('labels.txt');
});
it('shows the applied labels returned by the server', async () => {
vi.mocked(api.postForm).mockResolvedValue({
success: true,
data: { labelCount: 3, labels: ['剪刀', '石頭', '布'] },
});
render(<InferenceOptions deviceId={DEVICE_ID} resultTaskType="classification" />);
uploadLabelFile(labelFile());
await waitFor(() => {
const status = screen.getByTestId('inference-label-status');
expect(status.textContent).toContain('3');
expect(status.textContent).toContain('剪刀');
});
});
it('says no labels are set before anything is uploaded', () => {
render(<InferenceOptions deviceId={DEVICE_ID} resultTaskType="classification" />);
expect(screen.getByTestId('inference-label-status').textContent).toContain(
'尚未上傳標籤檔',
);
});
it('offers a clear action only after labels are applied', async () => {
vi.mocked(api.postForm).mockResolvedValue({
success: true,
data: { labels: ['剪刀', '石頭', '布'] },
});
render(<InferenceOptions deviceId={DEVICE_ID} resultTaskType="classification" />);
expect(screen.queryByTestId('inference-label-clear-btn')).toBeNull();
uploadLabelFile(labelFile());
await waitFor(() =>
expect(screen.getByTestId('inference-label-clear-btn')).toBeInTheDocument(),
);
});
it('clears the mapping through the endpoint and drops back to raw indices', async () => {
vi.mocked(api.postForm).mockResolvedValue({
success: true,
data: { labels: ['剪刀', '石頭', '布'] },
});
render(<InferenceOptions deviceId={DEVICE_ID} resultTaskType="classification" />);
uploadLabelFile(labelFile());
await waitFor(() =>
expect(screen.getByTestId('inference-label-clear-btn')).toBeInTheDocument(),
);
fireEvent.click(screen.getByTestId('inference-label-clear-btn'));
await waitFor(() =>
expect(screen.getByTestId('inference-label-status').textContent).toContain(
'尚未上傳標籤檔',
),
);
// Clearing goes over JSON with an explicit empty array — an omitted field
// would leave the mapping in place on the device.
expect(api.post).toHaveBeenCalledWith(OPTIONS_PATH, { labels: [] });
});
it('lets the same file be re-picked after fixing it on disk', async () => {
render(<InferenceOptions deviceId={DEVICE_ID} resultTaskType="classification" />);
uploadLabelFile(labelFile());
await waitFor(() => expect(api.postForm).toHaveBeenCalledTimes(1));
// The input value is reset after each pick, so selecting the same filename
// again still fires a change event.
uploadLabelFile(labelFile());
await waitFor(() => expect(api.postForm).toHaveBeenCalledTimes(2));
});
});
describe('InferenceOptions — label upload errors', () => {
it('rejects a non-.txt file without hitting the network', async () => {
render(<InferenceOptions deviceId={DEVICE_ID} resultTaskType="classification" />);
uploadLabelFile(labelFile('labels.json', '{}'));
expect(await screen.findByTestId('inference-options-error')).toHaveTextContent(
'只接受 .txt 或 .names 檔',
);
expect(api.postForm).not.toHaveBeenCalled();
});
it('accepts a .names file', async () => {
render(<InferenceOptions deviceId={DEVICE_ID} resultTaskType="classification" />);
uploadLabelFile(labelFile('coco.names'));
await waitFor(() => expect(api.postForm).toHaveBeenCalled());
expect(screen.queryByTestId('inference-options-error')).toBeNull();
});
it('rejects an oversized file without hitting the network', async () => {
const huge = labelFile('labels.txt', 'x'.repeat(MAX_LABEL_FILE_BYTES + 1));
render(<InferenceOptions deviceId={DEVICE_ID} resultTaskType="classification" />);
uploadLabelFile(huge);
expect(await screen.findByTestId('inference-options-error')).toHaveTextContent('過大');
expect(api.postForm).not.toHaveBeenCalled();
});
it('shows the offending line number for a parse error', async () => {
vi.mocked(api.postForm).mockResolvedValue(
reply({
success: false,
error: {
code: 'LABEL_PARSE_ERROR',
message: "index 必須為非負整數,收到 'abc'",
line: 5,
},
}),
);
render(<InferenceOptions deviceId={DEVICE_ID} resultTaskType="classification" />);
uploadLabelFile(labelFile());
const err = await screen.findByTestId('inference-options-error');
// The line number is the whole reason parsing lives server-side — losing it
// would leave the user hunting through their file blind.
expect(err.textContent).toContain('5');
expect(err.textContent).toContain('非負整數');
});
it('shows a generic failure message when the server sends no line number', async () => {
vi.mocked(api.postForm).mockResolvedValue({
success: false,
error: { code: 'STORAGE_ERROR', message: 'could not write labels' },
});
render(<InferenceOptions deviceId={DEVICE_ID} resultTaskType="classification" />);
uploadLabelFile(labelFile());
const err = await screen.findByTestId('inference-options-error');
expect(err.textContent).toContain('could not write labels');
});
it('keeps the previous mapping when a replacement upload fails', async () => {
vi.mocked(api.postForm).mockResolvedValue({
success: true,
data: { labels: ['剪刀', '石頭', '布'] },
});
render(<InferenceOptions deviceId={DEVICE_ID} resultTaskType="classification" />);
uploadLabelFile(labelFile());
await waitFor(() =>
expect(screen.getByTestId('inference-label-status').textContent).toContain('剪刀'),
);
vi.mocked(api.postForm).mockResolvedValue(
reply({
success: false,
error: { code: 'LABEL_PARSE_ERROR', message: 'bad', line: 2 },
}),
);
uploadLabelFile(labelFile('bad.txt', 'garbage'));
await screen.findByTestId('inference-options-error');
// A failed replacement must not silently wipe the mapping that is still in
// effect on the device.
expect(screen.getByTestId('inference-label-status').textContent).toContain('剪刀');
});
it('clears a stale error once a later upload succeeds', async () => {
vi.mocked(api.postForm).mockResolvedValue(
reply({
success: false,
error: { code: 'LABEL_PARSE_ERROR', message: 'bad', line: 2 },
}),
);
render(<InferenceOptions deviceId={DEVICE_ID} resultTaskType="classification" />);
uploadLabelFile(labelFile('bad.txt'));
await screen.findByTestId('inference-options-error');
vi.mocked(api.postForm).mockResolvedValue({
success: true,
data: { labels: ['剪刀'] },
});
uploadLabelFile(labelFile());
await waitFor(() => expect(screen.queryByTestId('inference-options-error')).toBeNull());
});
});

View File

@ -0,0 +1,109 @@
import { describe, it, expect } from 'vitest';
import { render, screen } from '@testing-library/react';
import { InferenceOverlay } from '@/components/camera/inference-overlay';
import type { InferenceResult } from '@/types/inference';
function makeResult(overrides: Partial<InferenceResult> = {}): InferenceResult {
return {
deviceId: 'dev-1',
modelId: 'model-1',
taskType: 'object_detection',
timestamp: 1784620457620,
latencyMs: 12.5,
detections: [],
classifications: [],
...overrides,
};
}
const SIZE = { width: 640, height: 480, confidenceThreshold: 0.5 };
describe('InferenceOverlay — M3-a taskType dispatch', () => {
it('renders the bbox canvas for object_detection', () => {
render(<InferenceOverlay result={makeResult()} {...SIZE} />);
expect(screen.getByTestId('camera-overlay')).toBeInTheDocument();
expect(screen.queryByTestId('classification-overlay')).toBeNull();
});
it('renders the bbox canvas for the legacy "detection" spelling (R-4)', () => {
render(<InferenceOverlay result={makeResult({ taskType: 'detection' })} {...SIZE} />);
expect(screen.getByTestId('camera-overlay')).toBeInTheDocument();
});
it('falls back to the bbox canvas when taskType is unknown', () => {
render(<InferenceOverlay result={makeResult({ taskType: 'something_new' })} {...SIZE} />);
expect(screen.getByTestId('camera-overlay')).toBeInTheDocument();
});
it('falls back to the bbox canvas when there is no result at all', () => {
render(<InferenceOverlay result={null} {...SIZE} />);
expect(screen.getByTestId('camera-overlay')).toBeInTheDocument();
});
it('renders the classification chip — and NO bbox canvas — for classification', () => {
render(
<InferenceOverlay
result={makeResult({
taskType: 'classification',
classifications: [
{ label: '石頭', classIndex: 1, confidence: 0.9425 },
{ label: '布', classIndex: 2, confidence: 0.0384 },
],
})}
{...SIZE}
/>,
);
expect(screen.getByTestId('classification-overlay')).toBeInTheDocument();
// The whole point of the feature: classification must not draw boxes.
expect(screen.queryByTestId('camera-overlay')).toBeNull();
});
});
describe('ClassificationOverlay rendering', () => {
function renderClassification(classifications: InferenceResult['classifications'], threshold = 0.5) {
return render(
<InferenceOverlay
result={makeResult({ taskType: 'classification', classifications })}
{...SIZE}
confidenceThreshold={threshold}
/>,
);
}
it('shows the top-1 label and confidence', () => {
renderClassification([
{ label: '布', classIndex: 2, confidence: 0.0384 },
{ label: '石頭', classIndex: 1, confidence: 0.9425 },
]);
expect(screen.getByTestId('classification-overlay-label').textContent).toBe('石頭');
expect(screen.getByTestId('classification-overlay-confidence').textContent).toBe('94.3%');
});
it('falls back to class_<index> when the label is missing', () => {
renderClassification([{ label: '', classIndex: 1, confidence: 0.88 }]);
expect(screen.getByTestId('classification-overlay-label').textContent).toBe('class_1');
});
it('renders without crashing when classIndex is absent (M2-d not shipped)', () => {
renderClassification([{ label: 'class_1', confidence: 0.88 }]);
expect(screen.getByTestId('classification-overlay-label').textContent).toBe('class_1');
});
it('shows the unrecognized state when nothing clears the threshold', () => {
renderClassification([{ label: '石頭', classIndex: 1, confidence: 0.2 }]);
expect(screen.getByTestId('classification-overlay-empty')).toBeInTheDocument();
expect(screen.queryByTestId('classification-overlay-label')).toBeNull();
});
it('shows the unrecognized state when the classifications array is empty', () => {
renderClassification([]);
expect(screen.getByTestId('classification-overlay-empty')).toBeInTheDocument();
});
it('exposes the verdict to assistive tech via a polite live region', () => {
renderClassification([{ label: '石頭', classIndex: 1, confidence: 0.9 }]);
const status = screen.getByRole('status');
expect(status.getAttribute('aria-live')).toBe('polite');
expect(status.textContent).toContain('石頭');
});
});

View File

@ -0,0 +1,87 @@
import { describe, it, expect, beforeEach } from 'vitest';
import { render, screen } from '@testing-library/react';
import { InferencePanel } from '@/components/inference/inference-panel';
import { useInferenceStore } from '@/stores/inference-store';
import { useCameraStore } from '@/stores/camera-store';
import { useInferenceOptionsStore } from '@/stores/inference-options-store';
import type { InferenceResult } from '@/types/inference';
const DEVICE_ID = 'dev-1';
function makeResult(overrides: Partial<InferenceResult> = {}): InferenceResult {
return {
deviceId: 'dev-1',
modelId: 'model-1',
taskType: 'object_detection',
timestamp: 1784620457620,
latencyMs: 12.5,
detections: [],
classifications: [],
...overrides,
};
}
function setResult(result: InferenceResult | null) {
useInferenceStore.setState({ result, confidenceThreshold: 0.5 });
}
beforeEach(() => {
useInferenceStore.setState({
isRunning: false,
result: null,
results: [],
fps: 0,
avgLatency: 0,
confidenceThreshold: 0.5,
batchResults: {},
});
useCameraStore.setState({ sourceType: 'camera', batchSelectedIndex: 0, batchImages: [] });
useInferenceOptionsStore.getState().reset();
});
describe('InferencePanel — M3-b taskType card switching', () => {
it('shows the detection list card for object_detection', () => {
setResult(
makeResult({
detections: [
{ label: 'person', confidence: 0.9, bbox: { x: 0, y: 0, width: 0.1, height: 0.1 } },
],
}),
);
render(<InferencePanel deviceId={DEVICE_ID} />);
expect(screen.getByTestId('detection-result-list')).toBeInTheDocument();
expect(screen.queryByTestId('classification-result-chart')).toBeNull();
});
it('shows the classification chart card for classification', () => {
setResult(
makeResult({
taskType: 'classification',
classifications: [{ label: '石頭', classIndex: 1, confidence: 0.94 }],
}),
);
render(<InferencePanel deviceId={DEVICE_ID} />);
expect(screen.getByTestId('classification-result-chart')).toBeInTheDocument();
expect(screen.queryByTestId('detection-result-list')).toBeNull();
});
it('defaults to the detection card when there is no result yet', () => {
setResult(null);
render(<InferencePanel deviceId={DEVICE_ID} />);
// No result → detection branch renders its empty state, not the chart.
expect(screen.queryByTestId('classification-result-chart')).toBeNull();
});
it('defaults to the detection card for the legacy "detection" spelling (R-4)', () => {
setResult(
makeResult({
taskType: 'detection',
detections: [
{ label: 'person', confidence: 0.9, bbox: { x: 0, y: 0, width: 0.1, height: 0.1 } },
],
}),
);
render(<InferencePanel deviceId={DEVICE_ID} />);
expect(screen.getByTestId('detection-result-list')).toBeInTheDocument();
});
});

View File

@ -0,0 +1,179 @@
import { describe, it, expect } from 'vitest';
import {
isClassificationTask,
classResultLabel,
classResultKey,
pickTopClass,
nextHysteresisState,
INITIAL_HYSTERESIS_STATE,
DEFAULT_HYSTERESIS,
type HysteresisOptions,
type HysteresisState,
} from '@/lib/classification';
import type { ClassResult } from '@/types/inference';
const OPTS: HysteresisOptions = {
...DEFAULT_HYSTERESIS,
confidenceThreshold: 0.5,
};
function cls(label: string, confidence: number, classIndex?: number): ClassResult {
return classIndex === undefined ? { label, confidence } : { label, confidence, classIndex };
}
/** Feed a sequence of frames through the state machine, return displayed labels. */
function run(frames: ClassResult[][], options: HysteresisOptions = OPTS): (string | null)[] {
let state: HysteresisState = INITIAL_HYSTERESIS_STATE;
return frames.map((f) => {
state = nextHysteresisState(state, f, options);
return state.displayed ? classResultLabel(state.displayed) : null;
});
}
describe('isClassificationTask — R-4 defensive branching', () => {
it('is true only for the exact classification value', () => {
expect(isClassificationTask('classification')).toBe(true);
});
it('is false for both detection spellings, so detection stays the fall-through', () => {
expect(isClassificationTask('detection')).toBe(false);
expect(isClassificationTask('object_detection')).toBe(false);
});
it('is false for undefined/null/unknown values', () => {
expect(isClassificationTask(undefined)).toBe(false);
expect(isClassificationTask(null)).toBe(false);
expect(isClassificationTask('segmentation')).toBe(false);
});
});
describe('classResultLabel — classIndex fallback (M2-d may be absent)', () => {
it('uses the label when present', () => {
expect(classResultLabel(cls('石頭', 0.9, 1))).toBe('石頭');
});
it('falls back to class_<index> when the label is empty', () => {
expect(classResultLabel(cls('', 0.9, 7))).toBe('class_7');
});
it('falls back to class_<index> when the label is only whitespace', () => {
expect(classResultLabel(cls(' ', 0.9, 0))).toBe('class_0');
});
it('emits a placeholder when both label and classIndex are missing', () => {
expect(classResultLabel(cls('', 0.9))).toBe('class_?');
});
it('works when classIndex is undefined but the label exists (the common case)', () => {
expect(classResultLabel(cls('cat', 0.9))).toBe('cat');
});
});
describe('classResultKey — identity used by hysteresis', () => {
it('prefers classIndex so label churn does not reset the state machine', () => {
expect(classResultKey(cls('石頭', 0.9, 1))).toBe('i:1');
expect(classResultKey(cls('rock', 0.9, 1))).toBe('i:1');
});
it('falls back to the label when classIndex is absent', () => {
expect(classResultKey(cls('石頭', 0.9))).toBe('l:石頭');
});
});
describe('pickTopClass', () => {
it('returns null for empty/undefined input', () => {
expect(pickTopClass([])).toBeNull();
expect(pickTopClass(undefined)).toBeNull();
});
it('returns the highest-confidence entry regardless of array order', () => {
const top = pickTopClass([cls('a', 0.1), cls('b', 0.8), cls('c', 0.3)]);
expect(top?.label).toBe('b');
});
});
describe('nextHysteresisState — R-9 anti-flicker', () => {
it('shows the first above-threshold result immediately', () => {
expect(run([[cls('石頭', 0.9)]])).toEqual(['石頭']);
});
it('shows nothing when the top class is below the confidence threshold', () => {
expect(run([[cls('石頭', 0.3)]])).toEqual([null]);
});
it('clears the label when confidence drops below the threshold', () => {
expect(run([[cls('石頭', 0.9)], [cls('石頭', 0.2)]])).toEqual(['石頭', null]);
});
it('does NOT switch on a single dissenting frame (the core anti-flicker case)', () => {
// 石頭 established, then one frame where 布 barely wins → must hold 石頭.
const out = run([
[cls('石頭', 0.6), cls('布', 0.4)],
[cls('布', 0.62), cls('石頭', 0.58)],
[cls('石頭', 0.6), cls('布', 0.4)],
]);
expect(out).toEqual(['石頭', '石頭', '石頭']);
});
it('switches after streakFrames consecutive frames of the challenger', () => {
// Margins stay under immediateMargin (0.15) so only the streak can promote.
const out = run([
[cls('石頭', 0.60)],
[cls('布', 0.62)],
[cls('布', 0.63)],
[cls('布', 0.64)],
]);
// frame1 石頭; frames 2-3 still 石頭 (streak 1,2); frame 4 promotes 布 (streak 3).
expect(out).toEqual(['石頭', '石頭', '石頭', '布']);
});
it('switches immediately when the challenger wins by the confidence margin', () => {
const out = run([
[cls('石頭', 0.60)],
[cls('布', 0.80)], // +0.20 >= immediateMargin 0.15
]);
expect(out).toEqual(['石頭', '布']);
});
it('resets the streak when the challenger is interrupted', () => {
const out = run([
[cls('石頭', 0.60)],
[cls('布', 0.62)], // streak 1
[cls('石頭', 0.61)], // incumbent back → streak cleared
[cls('布', 0.62)], // streak restarts at 1
[cls('布', 0.62)], // streak 2
]);
expect(out).toEqual(['石頭', '石頭', '石頭', '石頭', '石頭']);
});
it('keeps refreshing confidence of the incumbent while it stays on top', () => {
let state: HysteresisState = INITIAL_HYSTERESIS_STATE;
state = nextHysteresisState(state, [cls('石頭', 0.60)], OPTS);
state = nextHysteresisState(state, [cls('石頭', 0.95)], OPTS);
expect(state.displayed?.confidence).toBe(0.95);
});
it('uses classIndex identity so a re-labelled same class is not treated as a switch', () => {
let state: HysteresisState = INITIAL_HYSTERESIS_STATE;
state = nextHysteresisState(state, [cls('石頭', 0.60, 1)], OPTS);
// Same index, different label string → same class, no hysteresis delay.
state = nextHysteresisState(state, [cls('rock', 0.61, 1)], OPTS);
expect(state.displayed?.label).toBe('rock');
expect(state.candidateStreak).toBe(0);
});
it('clears state entirely on an empty frame', () => {
let state: HysteresisState = INITIAL_HYSTERESIS_STATE;
state = nextHysteresisState(state, [cls('石頭', 0.9)], OPTS);
state = nextHysteresisState(state, [], OPTS);
expect(state).toEqual(INITIAL_HYSTERESIS_STATE);
});
it('honours a raised confidenceThreshold on the very next frame', () => {
let state: HysteresisState = INITIAL_HYSTERESIS_STATE;
state = nextHysteresisState(state, [cls('石頭', 0.6)], OPTS);
expect(state.displayed).not.toBeNull();
state = nextHysteresisState(state, [cls('石頭', 0.6)], { ...OPTS, confidenceThreshold: 0.8 });
expect(state.displayed).toBeNull();
});
});

View File

@ -6,6 +6,31 @@ afterEach(() => {
cleanup(); cleanup();
}); });
// jsdom does not implement ResizeObserver, which Radix primitives (Slider) and
// CameraFeed rely on. A no-op stub is enough: tests assert on rendered output,
// not on resize-driven behaviour.
if (!('ResizeObserver' in globalThis)) {
class ResizeObserverStub implements ResizeObserver {
observe() {}
unobserve() {}
disconnect() {}
}
globalThis.ResizeObserver = ResizeObserverStub;
}
// jsdom implements neither the Pointer Capture API nor scrollIntoView, both of
// which Radix Select calls while opening its dropdown. Without these stubs the
// trigger throws `target.hasPointerCapture is not a function` and the listbox
// never mounts, so any test that opens a <Select> is untestable.
if (!Element.prototype.hasPointerCapture) {
Element.prototype.hasPointerCapture = () => false;
Element.prototype.setPointerCapture = () => {};
Element.prototype.releasePointerCapture = () => {};
}
if (!Element.prototype.scrollIntoView) {
Element.prototype.scrollIntoView = () => {};
}
Object.defineProperty(window, 'matchMedia', { Object.defineProperty(window, 'matchMedia', {
writable: true, writable: true,
value: vi.fn().mockImplementation((query: string) => ({ value: vi.fn().mockImplementation((query: string) => ({

View File

@ -0,0 +1,169 @@
import { describe, it, expect, beforeEach, vi } from 'vitest';
import {
useInferenceOptionsStore,
validateLabelFile,
MAX_LABEL_FILE_BYTES,
type InferenceOptionsResponse,
type InferenceOptionsError,
} from '@/stores/inference-options-store';
import { api, type ApiResponse } from '@/lib/api';
/**
* The store widens the error type so a parse error keeps its `line`. `vi.mocked`
* resolves the generic to the default `ApiError`, so mock responses go through
* this helper to stay in the shape the store actually receives.
*/
type OptionsResponse = ApiResponse<InferenceOptionsResponse, InferenceOptionsError>;
const reply = (r: OptionsResponse) => r as ApiResponse<InferenceOptionsResponse>;
vi.mock('@/lib/api', () => ({
api: {
post: vi.fn(),
postForm: vi.fn(),
},
getRelayHeaders: vi.fn().mockReturnValue({}),
}));
const DEVICE_ID = 'dev-1';
const OPTIONS_PATH = `/devices/${DEVICE_ID}/inference/options`;
beforeEach(() => {
vi.clearAllMocks();
vi.mocked(api.post).mockResolvedValue({ success: true, data: {} });
vi.mocked(api.postForm).mockResolvedValue({ success: true, data: {} });
useInferenceOptionsStore.getState().reset();
});
function file(name = 'labels.txt', content = '0 a\n') {
return new File([content], name, { type: 'text/plain' });
}
describe('inference options store — request shapes', () => {
it('POSTs JSON { taskType } to the device options path', async () => {
await useInferenceOptionsStore.getState().setTaskType(DEVICE_ID, 'classification');
expect(api.post).toHaveBeenCalledWith(OPTIONS_PATH, { taskType: 'classification' });
});
it('POSTs the label file as multipart under the "labelFile" field', async () => {
await useInferenceOptionsStore.getState().uploadLabels(DEVICE_ID, file());
const [path, form] = vi.mocked(api.postForm).mock.calls[0];
expect(path).toBe(OPTIONS_PATH);
// The server reads `labelFile`; anything else is silently ignored and the
// request then fails the "neither taskType nor labels" guard.
expect((form.get('labelFile') as File).name).toBe('labels.txt');
});
it('clears via an explicit empty labels array, not multipart', async () => {
await useInferenceOptionsStore.getState().clearLabels(DEVICE_ID);
// Omitting `labels` means "leave untouched" server-side, so the empty array
// has to be on the wire for the mapping to actually be dropped.
expect(api.post).toHaveBeenCalledWith(OPTIONS_PATH, { labels: [] });
expect(api.postForm).not.toHaveBeenCalled();
});
});
describe('inference options store — state transitions', () => {
it('returns true and records the task type on success', async () => {
const ok = await useInferenceOptionsStore
.getState()
.setTaskType(DEVICE_ID, 'classification');
expect(ok).toBe(true);
expect(useInferenceOptionsStore.getState().taskType).toBe('classification');
});
it('returns false and leaves the task type untouched on failure', async () => {
vi.mocked(api.post).mockResolvedValue({
success: false,
error: { code: 'DEVICE_NOT_CONNECTED', message: 'nope' },
});
const ok = await useInferenceOptionsStore
.getState()
.setTaskType(DEVICE_ID, 'classification');
expect(ok).toBe(false);
// A rejected switch must not leave the UI claiming the new mode is active.
expect(useInferenceOptionsStore.getState().taskType).toBeNull();
expect(useInferenceOptionsStore.getState().error?.code).toBe('DEVICE_NOT_CONNECTED');
});
it('preserves the parse error line number', async () => {
vi.mocked(api.postForm).mockResolvedValue(
reply({
success: false,
error: { code: 'LABEL_PARSE_ERROR', message: 'bad index', line: 7 },
}),
);
await useInferenceOptionsStore.getState().uploadLabels(DEVICE_ID, file());
expect(useInferenceOptionsStore.getState().error?.line).toBe(7);
});
it('keeps existing labels when a replacement upload fails', async () => {
vi.mocked(api.postForm).mockResolvedValue({ success: true, data: { labels: ['a', 'b'] } });
await useInferenceOptionsStore.getState().uploadLabels(DEVICE_ID, file());
vi.mocked(api.postForm).mockResolvedValue(
reply({
success: false,
error: { code: 'LABEL_PARSE_ERROR', message: 'bad', line: 1 },
}),
);
await useInferenceOptionsStore.getState().uploadLabels(DEVICE_ID, file());
expect(useInferenceOptionsStore.getState().labels).toEqual(['a', 'b']);
});
it('turns a thrown network error into an error envelope rather than rejecting', async () => {
vi.mocked(api.post).mockRejectedValue(new Error('offline'));
const ok = await useInferenceOptionsStore
.getState()
.setTaskType(DEVICE_ID, 'classification');
expect(ok).toBe(false);
expect(useInferenceOptionsStore.getState().error?.code).toBe('NETWORK_ERROR');
expect(useInferenceOptionsStore.getState().applying).toBe(false);
});
it('drops both the task type and labels on reset', async () => {
vi.mocked(api.postForm).mockResolvedValue({ success: true, data: { labels: ['a'] } });
await useInferenceOptionsStore.getState().setTaskType(DEVICE_ID, 'classification');
await useInferenceOptionsStore.getState().uploadLabels(DEVICE_ID, file());
useInferenceOptionsStore.getState().reset();
const s = useInferenceOptionsStore.getState();
expect(s.taskType).toBeNull();
expect(s.labels).toBeNull();
expect(s.error).toBeNull();
});
});
describe('validateLabelFile', () => {
it('accepts .txt', () => {
expect(validateLabelFile(file('labels.txt'))).toEqual({ ok: true });
});
it('accepts .names', () => {
expect(validateLabelFile(file('coco.names'))).toEqual({ ok: true });
});
it('accepts an uppercase extension', () => {
expect(validateLabelFile(file('LABELS.TXT'))).toEqual({ ok: true });
});
it('rejects another extension', () => {
expect(validateLabelFile(file('labels.json'))).toEqual({
ok: false,
reason: 'wrong-type',
});
});
it('rejects a file over the size cap', () => {
const huge = file('labels.txt', 'x'.repeat(MAX_LABEL_FILE_BYTES + 1));
expect(validateLabelFile(huge)).toEqual({ ok: false, reason: 'too-large' });
});
it('accepts a file exactly at the cap', () => {
const exact = file('labels.txt', 'x'.repeat(MAX_LABEL_FILE_BYTES));
expect(validateLabelFile(exact)).toEqual({ ok: true });
});
});

View File

@ -8,6 +8,11 @@ export interface BBox {
export interface ClassResult { export interface ClassResult {
label: string; label: string;
confidence: number; confidence: number;
/**
* Raw class index from the model output. Optional the Go layer may not
* forward it (M2-d is optional), so consumers must fall back to `label`.
*/
classIndex?: number;
} }
export interface DetectionResult { export interface DetectionResult {

View File

@ -40,8 +40,8 @@
}, },
{ {
"id": "kl520-fcos-detection", "id": "kl520-fcos-detection",
"name": "FCOS Detection (KL520)", "name": "物件辨識",
"description": "FCOS (Fully Convolutional One-Stage) object detection with DarkNet53s backbone, compiled for KL520. Anchor-free detection at 512x512.", "description": "通用物件偵測模型,可辨識人、車輛、動物等常見物件,適合一般場景的多物件偵測。",
"thumbnail": "/images/models/fcos-det.png", "thumbnail": "/images/models/fcos-det.png",
"taskType": "object_detection", "taskType": "object_detection",
"categories": [ "categories": [
@ -109,8 +109,8 @@
}, },
{ {
"id": "kl520-tiny-yolov3", "id": "kl520-tiny-yolov3",
"name": "Tiny YOLOv3 (KL520)", "name": "人型監測",
"description": "Tiny YOLOv3 object detection model compiled for KL520. Compact and fast model for general-purpose multi-object detection on edge devices.", "description": "輕量快速的人員偵測模型,適合即時監控場景,可在邊緣裝置上高速偵測畫面中的人員。",
"thumbnail": "/images/models/tiny-yolov3.png", "thumbnail": "/images/models/tiny-yolov3.png",
"taskType": "object_detection", "taskType": "object_detection",
"categories": [ "categories": [

View File

@ -2,9 +2,14 @@ package handlers
import ( import (
"context" "context"
"errors"
"fmt" "fmt"
"io"
"net/http"
"os" "os"
"path/filepath"
"runtime" "runtime"
"strings"
"time" "time"
"visiona-local/server/internal/api/ws" "visiona-local/server/internal/api/ws"
@ -12,6 +17,7 @@ import (
"visiona-local/server/internal/driver" "visiona-local/server/internal/driver"
"visiona-local/server/internal/flash" "visiona-local/server/internal/flash"
"visiona-local/server/internal/inference" "visiona-local/server/internal/inference"
"visiona-local/server/internal/labelfile"
"github.com/gin-gonic/gin" "github.com/gin-gonic/gin"
) )
@ -175,6 +181,8 @@ func (h *DeviceHandler) DisconnectDevice(c *gin.Context) {
func (h *DeviceHandler) FlashDevice(c *gin.Context) { func (h *DeviceHandler) FlashDevice(c *gin.Context) {
id := c.Param("id") id := c.Param("id")
// 燒錄只需要 modelId。推論種類一律用 models.json 宣告的值;要改解析方式
// 走 POST /devices/:id/inference/options推論期即時切換、不必重燒
var req struct { var req struct {
ModelID string `json:"modelId"` ModelID string `json:"modelId"`
} }
@ -207,6 +215,289 @@ func (h *DeviceHandler) FlashDevice(c *gin.Context) {
c.JSON(200, gin.H{"success": true, "data": gin.H{"taskId": taskID}}) c.JSON(200, gin.H{"success": true, "data": gin.H{"taskId": taskID}})
} }
// InferenceOptionsDriver 是「支援推論期切換解析方式」的 driver 能力介面。
//
// 為什麼不直接加進 driver.DeviceDriver那是所有 driver 都必須實作的最小
// 契約,加一個 Kneron 特有能力進去,三個既有 test fake 全都要跟著改,而它們
// 跟這個功能完全無關。用窄介面 + type assert 是這個 repo 已建立的做法
// (見 firmware.UpgradeDriver / DeviceManagerAdapter.GetUpgradeDriver
type InferenceOptionsDriver interface {
SetInferenceOptions(opts driver.InferenceOptions) error
}
// inferenceOptionsRequest 是 JSON 形式的 request body。
//
// 兩個欄位都是指標,因為必須區分「沒帶這個欄位」與「帶了空值」:
//
// Labels == nil → 不動 label 表
// Labels == &[]string{} → 清空 label 表(回到原始 enum
//
// 用非指標 []string 的話 JSON 的 `null`、`[]` 與「欄位不存在」會全部塌成
// nil「清空」這個合法意圖就永遠表達不出來。
type inferenceOptionsRequest struct {
TaskType *string `json:"taskType"`
Labels *[]string `json:"labels"`
}
// SetInferenceOptions 在不重新燒錄的前提下,更新當前已載入模型的解析方式
// 與 label 表。
//
// POST /api/devices/:id/inference/options
//
// 支援兩種 content type
//
// application/json — {"taskType": "...", "labels": [...]}
// multipart/form-data — taskType 欄位 + labelFile 檔案(`<index> <名稱>`
//
// 刻意不做任何持久化:使用者明確要求「不用記,每次現場傳」。設定只存在於
// 當前 bridge sessiondisconnect / reset / 重新 flash 都會清掉。
func (h *DeviceHandler) SetInferenceOptions(c *gin.Context) {
id := c.Param("id")
session, err := h.deviceMgr.GetDevice(id)
if err != nil {
c.JSON(404, gin.H{
"success": false,
"error": gin.H{"code": "DEVICE_NOT_FOUND", "message": err.Error()},
})
return
}
optsDrv, ok := session.Driver.(InferenceOptionsDriver)
if !ok {
c.JSON(400, gin.H{
"success": false,
"error": gin.H{
"code": "UNSUPPORTED_DEVICE",
"message": "this device driver does not support runtime inference options",
},
})
return
}
opts, labelInfo, apiErr := parseInferenceOptionsRequest(c)
if apiErr != nil {
c.JSON(apiErr.status, gin.H{"success": false, "error": apiErr.body()})
return
}
if err := optsDrv.SetInferenceOptions(opts); err != nil {
c.JSON(400, gin.H{
"success": false,
"error": gin.H{"code": "INFERENCE_OPTIONS_FAILED", "message": err.Error()},
})
return
}
data := gin.H{
"deviceId": id,
"taskType": opts.TaskType,
}
if opts.Labels != nil {
data["labelCount"] = labelInfo.namedCount
data["labels"] = opts.Labels
if len(opts.Labels) > 0 {
data["maxIndex"] = len(opts.Labels) - 1
}
}
c.JSON(200, gin.H{"success": true, "data": data})
}
// apiError 讓 parse 階段能同時回「HTTP status + 錯誤碼 + 可選的行號」。
type apiError struct {
status int
code string
message string
// line 為 label 檔解析失敗的行號0 表示與行號無關、不放進回應。
line int
}
func (e *apiError) body() gin.H {
h := gin.H{"code": e.code, "message": e.message}
if e.line > 0 {
h["line"] = e.line
}
return h
}
// labelSummary 帶回 handler 要回報給前端的 label 統計。
type labelSummary struct {
// namedCount 是實際有名稱的筆數(不含稀疏補洞的空字串)。
namedCount int
}
// parseInferenceOptionsRequest 從 JSON 或 multipart 取出設定並完整驗證。
func parseInferenceOptionsRequest(c *gin.Context) (driver.InferenceOptions, labelSummary, *apiError) {
var opts driver.InferenceOptions
var summary labelSummary
contentType := c.ContentType()
if strings.HasPrefix(contentType, "multipart/form-data") {
var err *apiError
opts, summary, err = parseMultipartInferenceOptions(c)
if err != nil {
return opts, summary, err
}
} else {
var req inferenceOptionsRequest
if bindErr := c.ShouldBindJSON(&req); bindErr != nil {
return opts, summary, &apiError{
status: 400,
code: "BAD_REQUEST",
message: "invalid JSON body: " + bindErr.Error(),
}
}
if req.TaskType != nil {
opts.TaskType = *req.TaskType
}
if req.Labels != nil {
// 顯式給了 labels含空陣列→ 一律送出。空陣列 = 清空,
// 必須與「沒帶欄位」區分開。
labels := *req.Labels
if labels == nil {
labels = []string{}
}
if len(labels) > labelfile.MaxIndex+1 {
return opts, summary, &apiError{
status: 400,
code: "LABEL_TOO_LARGE",
message: fmt.Sprintf("labels 筆數 %d 超過上限 %d", len(labels), labelfile.MaxIndex+1),
}
}
opts.Labels = labels
summary.namedCount = countNamedLabels(labels)
}
}
// taskType 值域用 flash.IsValidTaskTypeOverride —— 這裡是目前唯一讓使用者
// 指定解析方式的入口(燒錄時不再選,一律用 models.json 宣告值)。舊別名
// detection 一樣拒絕 —— bridge 收得下,但不讓兩套命名同時出現在 wire 上R-4
if opts.TaskType != "" && !flash.IsValidTaskTypeOverride(opts.TaskType) {
return opts, summary, &apiError{
status: 400,
code: "VALIDATION_ERROR",
message: fmt.Sprintf("invalid taskType %q: must be %s or %s",
opts.TaskType, flash.TaskTypeObjectDetection, flash.TaskTypeClassification),
}
}
// 兩者都沒帶 = 呼叫端沒表達任何意圖。回 200 等於假裝做了事,正是這個
// 功能要防的靜默失敗,所以擋在這裡。
if opts.TaskType == "" && opts.Labels == nil {
return opts, summary, &apiError{
status: 400,
code: "BAD_REQUEST",
message: "至少要提供 taskType 或 labels 其中一項",
}
}
return opts, summary, nil
}
// parseMultipartInferenceOptions 處理 multipart 上傳taskType 欄位 + labelFile 檔案)。
func parseMultipartInferenceOptions(c *gin.Context) (driver.InferenceOptions, labelSummary, *apiError) {
var opts driver.InferenceOptions
var summary labelSummary
// 限制 multipart 在記憶體中的暫存量;超過的部分 gin 會落地成暫存檔,
// 但真正的防線是下方的 header.Size 檢查。
if err := c.Request.ParseMultipartForm(labelfile.MaxFileSize); err != nil {
return opts, summary, &apiError{
status: 400,
code: "BAD_REQUEST",
message: "invalid multipart form: " + err.Error(),
}
}
opts.TaskType = c.PostForm("taskType")
file, header, err := c.Request.FormFile("labelFile")
if err != nil {
// 沒有檔案是合法的(只切 taskType。其他錯誤才算壞請求。
if errors.Is(err, http.ErrMissingFile) {
return opts, summary, nil
}
return opts, summary, &apiError{
status: 400,
code: "BAD_REQUEST",
message: "failed to read labelFile: " + err.Error(),
}
}
defer file.Close()
// 大小上限在讀取「之前」擋,不能讀完再判斷 —— 那時記憶體已經吃掉了。
if header.Size > labelfile.MaxFileSize {
return opts, summary, &apiError{
status: 400,
code: "LABEL_TOO_LARGE",
message: fmt.Sprintf("標籤檔過大(%d bytes上限為 %d bytes",
header.Size, labelfile.MaxFileSize),
}
}
// 副檔名檢查純粹防呆(真正的防線是內容解析)。上傳檔名只用來看副檔名,
// 不參與任何路徑組合 —— 本 endpoint 不落地存檔,沒有路徑穿越面。
ext := strings.ToLower(filepath.Ext(header.Filename))
if ext != "" && ext != ".txt" && ext != ".names" {
return opts, summary, &apiError{
status: 400,
code: "BAD_REQUEST",
message: "標籤檔僅支援 .txt / .names",
}
}
// LimitReader 是 header.Size 之外的第二道防線Content-Length 可以造假,
// 實際串流長度才是真的。多讀 1 byte 用來偵測「宣稱小、其實大」。
data, readErr := io.ReadAll(io.LimitReader(file, labelfile.MaxFileSize+1))
if readErr != nil {
return opts, summary, &apiError{
status: 400,
code: "BAD_REQUEST",
message: "failed to read labelFile: " + readErr.Error(),
}
}
if len(data) > labelfile.MaxFileSize {
return opts, summary, &apiError{
status: 400,
code: "LABEL_TOO_LARGE",
message: fmt.Sprintf("標籤檔過大,上限為 %d bytes", labelfile.MaxFileSize),
}
}
result, parseErr := labelfile.Parse(data)
if parseErr != nil {
var pe *labelfile.ParseError
if errors.As(parseErr, &pe) {
return opts, summary, &apiError{
status: 400,
code: "LABEL_PARSE_ERROR",
message: pe.Error(),
line: pe.Line,
}
}
return opts, summary, &apiError{
status: 400,
code: "LABEL_PARSE_ERROR",
message: parseErr.Error(),
}
}
opts.Labels = result.Labels
summary.namedCount = result.LabelCount
return opts, summary, nil
}
// countNamedLabels 算出實際有名稱的筆數(稀疏補洞的空字串不計)。
func countNamedLabels(labels []string) int {
n := 0
for _, l := range labels {
if strings.TrimSpace(l) != "" {
n++
}
}
return n
}
func (h *DeviceHandler) StartInference(c *gin.Context) { func (h *DeviceHandler) StartInference(c *gin.Context) {
id := c.Param("id") id := c.Param("id")
resultCh := make(chan *driver.InferenceResult, 10) resultCh := make(chan *driver.InferenceResult, 10)

View File

@ -0,0 +1,500 @@
package handlers
// device_inference_options_test.go — M4POST /api/devices/:id/inference/options
// 的 HTTP 契約測試(推論期切換解析方式 / label 表)。
//
// 測試分兩層:
//
// 1. parseInferenceOptionsRequest — 這裡是本 endpoint 幾乎全部的邏輯
// JSON / multipart 解析、值域驗證、大小上限、nil vs 空陣列的語意)。
// 它只依賴 *gin.Context可以完整單元測試。
// 2. SetInferenceOptions handler — 只驗它自己負責的分支device 不存在、
// driver 不支援、driver 回錯。
//
// ⚠️ 測試接縫限制(與 device_flash_tasktype_test.go 同一個既有問題):
// DeviceHandler.deviceMgr 是具體的 *device.Manager其 sessions map 未匯出、
// 只能由真實硬體填入,跨 package 無法注入 fake session。因此「成功路徑打到
// driver」這段沒有 handler 級測試 —— 由 buildSetInferenceOptionsCommand 的
// 單元測試driver/kneron與實機驗收覆蓋。這是既有架構限制不是本次引入。
import (
"bytes"
"encoding/json"
"fmt"
"mime/multipart"
"net/http"
"net/http/httptest"
"strings"
"testing"
"visiona-local/server/internal/device"
"visiona-local/server/internal/driver"
"visiona-local/server/internal/driver/kneron"
"visiona-local/server/internal/labelfile"
"github.com/gin-gonic/gin"
)
// ── helpers ──────────────────────────────────────────────────────────
// errorCode 取出統一錯誤格式裡的 error.code。
//
// 原先住在 device_flash_tasktype_test.go該檔隨「燒錄時選推論種類」功能一起
// 移除後搬來這裡(本檔是目前唯一的使用者)。
func errorCode(parsed map[string]interface{}) string {
errObj, ok := parsed["error"].(map[string]interface{})
if !ok {
return ""
}
code, _ := errObj["code"].(string)
return code
}
// runParse 以指定的 body / content-type 呼叫 parseInferenceOptionsRequest。
func runParse(t *testing.T, contentType string, body []byte) (driver.InferenceOptions, labelSummary, *apiError) {
t.Helper()
var (
opts driver.InferenceOptions
summary labelSummary
apiErr *apiError
)
router := gin.New()
router.POST("/x", func(c *gin.Context) {
opts, summary, apiErr = parseInferenceOptionsRequest(c)
c.Status(http.StatusOK)
})
req := httptest.NewRequest(http.MethodPost, "/x", bytes.NewReader(body))
req.Header.Set("Content-Type", contentType)
router.ServeHTTP(httptest.NewRecorder(), req)
return opts, summary, apiErr
}
func parseJSON(t *testing.T, body string) (driver.InferenceOptions, labelSummary, *apiError) {
t.Helper()
return runParse(t, "application/json", []byte(body))
}
// buildMultipart 組出 multipart bodylabelFileName 為空表示不帶檔案)。
func buildMultipart(t *testing.T, taskType, labelFileName, labelContent string) (string, []byte) {
t.Helper()
var buf bytes.Buffer
w := multipart.NewWriter(&buf)
if taskType != "" {
if err := w.WriteField("taskType", taskType); err != nil {
t.Fatalf("WriteField: %v", err)
}
}
if labelFileName != "" {
fw, err := w.CreateFormFile("labelFile", labelFileName)
if err != nil {
t.Fatalf("CreateFormFile: %v", err)
}
if _, err := fw.Write([]byte(labelContent)); err != nil {
t.Fatalf("write file part: %v", err)
}
}
if err := w.Close(); err != nil {
t.Fatalf("close writer: %v", err)
}
return w.FormDataContentType(), buf.Bytes()
}
func requireNoAPIError(t *testing.T, err *apiError) {
t.Helper()
if err != nil {
t.Fatalf("unexpected apiError: code=%s message=%s", err.code, err.message)
}
}
func requireAPIError(t *testing.T, err *apiError, wantCode string) *apiError {
t.Helper()
if err == nil {
t.Fatalf("expected apiError with code %s, got nil", wantCode)
}
if err.code != wantCode {
t.Fatalf("error code = %q, want %q (message=%q)", err.code, wantCode, err.message)
}
return err
}
// ── JSON body ────────────────────────────────────────────────────────
func TestParseInferenceOptions_JSON_TaskTypeOnly(t *testing.T) {
opts, _, err := parseJSON(t, `{"taskType":"classification"}`)
requireNoAPIError(t, err)
if opts.TaskType != "classification" {
t.Errorf("TaskType = %q, want classification", opts.TaskType)
}
if opts.Labels != nil {
t.Errorf("Labels = %v, want nil沒帶 labels 就不該動 label 表)", opts.Labels)
}
}
func TestParseInferenceOptions_JSON_LabelsOnly(t *testing.T) {
opts, summary, err := parseJSON(t, `{"labels":["剪刀","石頭","布"]}`)
requireNoAPIError(t, err)
if opts.TaskType != "" {
t.Errorf("TaskType = %q, want 空(沒帶就不該動解析方式)", opts.TaskType)
}
want := []string{"剪刀", "石頭", "布"}
if !equalStringSlice(opts.Labels, want) {
t.Errorf("Labels = %v, want %v", opts.Labels, want)
}
if summary.namedCount != 3 {
t.Errorf("namedCount = %d, want 3", summary.namedCount)
}
}
// 關鍵語意:空陣列 = 清空 label 表,必須與「沒帶欄位」區分。
func TestParseInferenceOptions_JSON_EmptyLabelsMeansClear(t *testing.T) {
opts, summary, err := parseJSON(t, `{"labels":[]}`)
requireNoAPIError(t, err)
if opts.Labels == nil {
t.Fatal("Labels = nil —— 空陣列被塌成 nil「清空 label 表」的意圖丟失了")
}
if len(opts.Labels) != 0 {
t.Errorf("len(Labels) = %d, want 0", len(opts.Labels))
}
if summary.namedCount != 0 {
t.Errorf("namedCount = %d, want 0", summary.namedCount)
}
}
// 對照組:沒帶 labels 欄位時 Labels 必須是 nil= 不動)。
func TestParseInferenceOptions_JSON_AbsentLabelsMeansUnchanged(t *testing.T) {
opts, _, err := parseJSON(t, `{"taskType":"classification"}`)
requireNoAPIError(t, err)
if opts.Labels != nil {
t.Errorf("Labels = %v, want nil", opts.Labels)
}
}
// JSON null 與「沒帶」同義 —— 都是不動。
func TestParseInferenceOptions_JSON_NullLabelsMeansUnchanged(t *testing.T) {
opts, _, err := parseJSON(t, `{"taskType":"classification","labels":null}`)
requireNoAPIError(t, err)
if opts.Labels != nil {
t.Errorf("Labels = %v, want nil", opts.Labels)
}
}
func TestParseInferenceOptions_JSON_Both(t *testing.T) {
opts, summary, err := parseJSON(t, `{"taskType":"classification","labels":["a","b"]}`)
requireNoAPIError(t, err)
if opts.TaskType != "classification" {
t.Errorf("TaskType = %q", opts.TaskType)
}
if summary.namedCount != 2 {
t.Errorf("namedCount = %d, want 2", summary.namedCount)
}
}
func TestParseInferenceOptions_JSON_SparseLabelsCountedCorrectly(t *testing.T) {
opts, summary, err := parseJSON(t, `{"labels":["a","","c"]}`)
requireNoAPIError(t, err)
if len(opts.Labels) != 3 {
t.Errorf("len(Labels) = %d, want 3稀疏佔位要保留位置就是 class index", len(opts.Labels))
}
if summary.namedCount != 2 {
t.Errorf("namedCount = %d, want 2空字串不算一筆", summary.namedCount)
}
}
func TestParseInferenceOptions_JSON_MalformedBody(t *testing.T) {
_, _, err := parseJSON(t, `{not json`)
requireAPIError(t, err, "BAD_REQUEST")
}
// ── 值域驗證(沿用燒錄時同一套)──────────────────────────────────────
func TestParseInferenceOptions_RejectsInvalidTaskType(t *testing.T) {
// R-4舊別名 detection 也要擋 —— bridge 收得下,但不讓兩套命名同時
// 出現在 wire 上。與 POST /flash 的規則保持完全一致。
bad := []string{
"detection",
"segmentation",
"pose_estimation",
"Classification",
"classifcation",
"garbage",
}
for _, tt := range bad {
t.Run(tt, func(t *testing.T) {
body := fmt.Sprintf(`{"taskType":%q}`, tt)
_, _, err := parseJSON(t, body)
e := requireAPIError(t, err, "VALIDATION_ERROR")
if e.status != http.StatusBadRequest {
t.Errorf("status = %d, want 400", e.status)
}
if !strings.Contains(e.message, "classification") ||
!strings.Contains(e.message, "object_detection") {
t.Errorf("message = %q, 應列出合法值", e.message)
}
})
}
}
func TestParseInferenceOptions_AcceptsValidTaskType(t *testing.T) {
for _, tt := range []string{"classification", "object_detection"} {
t.Run(tt, func(t *testing.T) {
opts, _, err := parseJSON(t, fmt.Sprintf(`{"taskType":%q}`, tt))
requireNoAPIError(t, err)
if opts.TaskType != tt {
t.Errorf("TaskType = %q, want %q", opts.TaskType, tt)
}
})
}
}
// 兩者都沒帶 = 呼叫端沒表達任何意圖。回 200 等於假裝做了事。
func TestParseInferenceOptions_RejectsEmptyRequest(t *testing.T) {
cases := map[string]string{
"空物件": `{}`,
"taskType 空字串": `{"taskType":""}`,
"兩者皆 null": `{"taskType":null,"labels":null}`,
}
for name, body := range cases {
t.Run(name, func(t *testing.T) {
_, _, err := parseJSON(t, body)
requireAPIError(t, err, "BAD_REQUEST")
})
}
}
// 只帶空 labels 陣列是有意義的(清空),不該被「空請求」規則誤擋。
func TestParseInferenceOptions_EmptyLabelsAloneIsNotEmptyRequest(t *testing.T) {
_, _, err := parseJSON(t, `{"labels":[]}`)
requireNoAPIError(t, err)
}
// ── JSON labels 數量上限S-2 / R-7─────────────────────────────────
func TestParseInferenceOptions_JSON_RejectsTooManyLabels(t *testing.T) {
labels := make([]string, labelfile.MaxIndex+2)
for i := range labels {
labels[i] = "x"
}
payload, err := json.Marshal(map[string]interface{}{"labels": labels})
if err != nil {
t.Fatalf("marshal: %v", err)
}
_, _, apiErr := runParse(t, "application/json", payload)
requireAPIError(t, apiErr, "LABEL_TOO_LARGE")
}
func TestParseInferenceOptions_JSON_AcceptsExactlyMaxLabels(t *testing.T) {
labels := make([]string, labelfile.MaxIndex+1)
for i := range labels {
labels[i] = "x"
}
payload, err := json.Marshal(map[string]interface{}{"labels": labels})
if err != nil {
t.Fatalf("marshal: %v", err)
}
_, _, apiErr := runParse(t, "application/json", payload)
requireNoAPIError(t, apiErr)
}
// ── multipart 上傳 ───────────────────────────────────────────────────
func TestParseInferenceOptions_Multipart_LabelFile(t *testing.T) {
ct, body := buildMultipart(t, "classification", "labels.txt", "0 剪刀\n1 石頭\n2 布\n")
opts, summary, err := runParse(t, ct, body)
requireNoAPIError(t, err)
if opts.TaskType != "classification" {
t.Errorf("TaskType = %q", opts.TaskType)
}
want := []string{"剪刀", "石頭", "布"}
if !equalStringSlice(opts.Labels, want) {
t.Errorf("Labels = %v, want %v", opts.Labels, want)
}
if summary.namedCount != 3 {
t.Errorf("namedCount = %d, want 3", summary.namedCount)
}
}
func TestParseInferenceOptions_Multipart_TaskTypeOnlyNoFile(t *testing.T) {
ct, body := buildMultipart(t, "object_detection", "", "")
opts, _, err := runParse(t, ct, body)
requireNoAPIError(t, err)
if opts.TaskType != "object_detection" {
t.Errorf("TaskType = %q", opts.TaskType)
}
if opts.Labels != nil {
t.Errorf("Labels = %v, want nil沒上傳檔案就不該動 label 表)", opts.Labels)
}
}
func TestParseInferenceOptions_Multipart_SparseLabelFile(t *testing.T) {
ct, body := buildMultipart(t, "classification", "labels.txt", "0 a\n3 d\n")
opts, summary, err := runParse(t, ct, body)
requireNoAPIError(t, err)
want := []string{"a", "", "", "d"}
if !equalStringSlice(opts.Labels, want) {
t.Errorf("Labels = %v, want %v", opts.Labels, want)
}
if summary.namedCount != 2 {
t.Errorf("namedCount = %d, want 2", summary.namedCount)
}
}
// 解析失敗要帶行號回去,前端才能指出是哪一行。
func TestParseInferenceOptions_Multipart_ParseErrorCarriesLine(t *testing.T) {
ct, body := buildMultipart(t, "", "labels.txt", "0 a\n1 b\nabc c\n")
_, _, err := runParse(t, ct, body)
e := requireAPIError(t, err, "LABEL_PARSE_ERROR")
if e.line != 3 {
t.Errorf("line = %d, want 3", e.line)
}
if _, ok := e.body()["line"].(int); !ok {
t.Error("回應 body 應包含 line 欄位")
}
}
func TestParseInferenceOptions_Multipart_ParseErrorBodyOmitsLineWhenZero(t *testing.T) {
ct, body := buildMultipart(t, "", "labels.txt", "\n\n\n")
_, _, err := runParse(t, ct, body)
e := requireAPIError(t, err, "LABEL_PARSE_ERROR")
if _, present := e.body()["line"]; present {
t.Error("與行號無關的錯誤不應帶 line 欄位(前端會亂標第 0 行)")
}
}
func TestParseInferenceOptions_Multipart_RejectsOversizedFile(t *testing.T) {
big := strings.Repeat("0 aaaaaaaa\n", labelfile.MaxFileSize/10+100)
ct, body := buildMultipart(t, "", "labels.txt", big)
_, _, err := runParse(t, ct, body)
requireAPIError(t, err, "LABEL_TOO_LARGE")
}
func TestParseInferenceOptions_Multipart_RejectsBadExtension(t *testing.T) {
ct, body := buildMultipart(t, "", "labels.exe", "0 a\n")
_, _, err := runParse(t, ct, body)
requireAPIError(t, err, "BAD_REQUEST")
}
func TestParseInferenceOptions_Multipart_AcceptsNamesExtension(t *testing.T) {
ct, body := buildMultipart(t, "", "coco.names", "0 person\n")
opts, _, err := runParse(t, ct, body)
requireNoAPIError(t, err)
if !equalStringSlice(opts.Labels, []string{"person"}) {
t.Errorf("Labels = %v", opts.Labels)
}
}
func TestParseInferenceOptions_Multipart_RejectsInvalidTaskType(t *testing.T) {
ct, body := buildMultipart(t, "detection", "labels.txt", "0 a\n")
_, _, err := runParse(t, ct, body)
requireAPIError(t, err, "VALIDATION_ERROR")
}
func TestParseInferenceOptions_Multipart_RejectsEmptyRequest(t *testing.T) {
ct, body := buildMultipart(t, "", "", "")
_, _, err := runParse(t, ct, body)
requireAPIError(t, err, "BAD_REQUEST")
}
// 上傳的 index 超過上限 → 由 labelfile 擋下並回 LABEL_PARSE_ERROR。
// 若這道防線失效handler 會嘗試配置巨大 slice。
func TestParseInferenceOptions_Multipart_RejectsHugeIndex(t *testing.T) {
ct, body := buildMultipart(t, "", "labels.txt", "999999999 boom\n")
_, _, err := runParse(t, ct, body)
requireAPIError(t, err, "LABEL_PARSE_ERROR")
}
// ── handler 分支(不需要 device session 的部分)──────────────────────
// postOptions 呼叫 SetInferenceOptions 並回傳 status / 解析後 body。
func postOptions(t *testing.T, h *DeviceHandler, contentType string, body []byte) (int, map[string]interface{}) {
t.Helper()
router := gin.New()
router.POST("/devices/:id/inference/options", h.SetInferenceOptions)
req := httptest.NewRequest(http.MethodPost, "/devices/dev-1/inference/options", bytes.NewReader(body))
req.Header.Set("Content-Type", contentType)
w := httptest.NewRecorder()
router.ServeHTTP(w, req)
parsed := map[string]interface{}{}
_ = json.Unmarshal(w.Body.Bytes(), &parsed)
return w.Code, parsed
}
func TestSetInferenceOptions_DeviceNotFound(t *testing.T) {
h := &DeviceHandler{deviceMgr: device.NewManager(device.NewRegistry(), "")}
status, parsed := postOptions(t, h, "application/json", []byte(`{"taskType":"classification"}`))
if status != http.StatusNotFound {
t.Errorf("status = %d, want 404", status)
}
if got := errorCode(parsed); got != "DEVICE_NOT_FOUND" {
t.Errorf("error.code = %q, want DEVICE_NOT_FOUND", got)
}
}
// device 查找必須排在 body 解析之前 —— 對不存在的裝置回「body 有問題」
// 會把使用者引去改 payload而真正的問題是裝置不在。
func TestSetInferenceOptions_DeviceLookupPrecedesBodyValidation(t *testing.T) {
h := &DeviceHandler{deviceMgr: device.NewManager(device.NewRegistry(), "")}
status, parsed := postOptions(t, h, "application/json", []byte(`{not json`))
if status != http.StatusNotFound {
t.Errorf("status = %d, want 404裝置查找應先於 body 解析)", status)
}
if got := errorCode(parsed); got != "DEVICE_NOT_FOUND" {
t.Errorf("error.code = %q, want DEVICE_NOT_FOUND", got)
}
}
// ── 介面契約 ─────────────────────────────────────────────────────────
// *kneron.KneronDriver 必須真的滿足 InferenceOptionsDriver。
//
// 這是整條鏈路唯一會「靜默壞掉」的接點handler 用 type assert 取得能力,
// 若 driver 的 method 簽章改了(或被誤刪),編譯完全不會報錯 —— endpoint
// 會對所有裝置回 UNSUPPORTED_DEVICE而且只有實機才看得出來。
//
// 刻意 assert 具體型別而非自己寫一個滿足介面的 stubstub 只證明「我寫的
// stub 符合我寫的介面」,對真正的實作零保障。
var _ InferenceOptionsDriver = (*kneron.KneronDriver)(nil)
// KneronDriver 同時必須仍是合法的 driver.DeviceDriver —— 加新能力不能
// 破壞既有契約。
var _ driver.DeviceDriver = (*kneron.KneronDriver)(nil)
// ── helper ───────────────────────────────────────────────────────────
func equalStringSlice(a, b []string) bool {
if len(a) != len(b) {
return false
}
for i := range a {
if a[i] != b[i] {
return false
}
}
return true
}

View File

@ -125,7 +125,9 @@ func (f *fakeDriver) Info() driver.DeviceInfo { return f.in
func (f *fakeDriver) Connect() error { return nil } func (f *fakeDriver) Connect() error { return nil }
func (f *fakeDriver) Disconnect() error { return nil } func (f *fakeDriver) Disconnect() error { return nil }
func (f *fakeDriver) IsConnected() bool { return false } func (f *fakeDriver) IsConnected() bool { return false }
func (f *fakeDriver) Flash(_ string, _ chan<- driver.FlashProgress) error { return nil } func (f *fakeDriver) Flash(_ string, _ driver.FlashOptions, _ chan<- driver.FlashProgress) error {
return nil
}
func (f *fakeDriver) StartInference() error { return nil } func (f *fakeDriver) StartInference() error { return nil }
func (f *fakeDriver) StopInference() error { return nil } func (f *fakeDriver) StopInference() error { return nil }
func (f *fakeDriver) ReadInference() (*driver.InferenceResult, error) { func (f *fakeDriver) ReadInference() (*driver.InferenceResult, error) {

View File

@ -95,6 +95,9 @@ func NewRouter(
api.POST("/devices/:id/flash", deviceHandler.FlashDevice) api.POST("/devices/:id/flash", deviceHandler.FlashDevice)
api.POST("/devices/:id/inference/start", deviceHandler.StartInference) api.POST("/devices/:id/inference/start", deviceHandler.StartInference)
api.POST("/devices/:id/inference/stop", deviceHandler.StopInference) api.POST("/devices/:id/inference/stop", deviceHandler.StopInference)
// M4推論期動態切換解析方式 / label 表(不重燒 model
// 設定不持久化 —— 只作用於當前 bridge session。
api.POST("/devices/:id/inference/options", deviceHandler.SetInferenceOptions)
// Firmware (M9-3、A 階段) // Firmware (M9-3、A 階段)
// upgrade endpoint 走 202 + WebSocket room "firmware:<id>" 推進度。 // upgrade endpoint 走 202 + WebSocket room "firmware:<id>" 推進度。

View File

@ -15,7 +15,9 @@ func (d *testDriver) Info() driver.DeviceInfo { r
func (d *testDriver) Connect() error { d.connected = true; d.info.Status = driver.StatusConnected; return nil } func (d *testDriver) Connect() error { d.connected = true; d.info.Status = driver.StatusConnected; return nil }
func (d *testDriver) Disconnect() error { d.connected = false; d.info.Status = driver.StatusDisconnected; return nil } func (d *testDriver) Disconnect() error { d.connected = false; d.info.Status = driver.StatusDisconnected; return nil }
func (d *testDriver) IsConnected() bool { return d.connected } func (d *testDriver) IsConnected() bool { return d.connected }
func (d *testDriver) Flash(_ string, _ chan<- driver.FlashProgress) error { return nil } func (d *testDriver) Flash(_ string, _ driver.FlashOptions, _ chan<- driver.FlashProgress) error {
return nil
}
func (d *testDriver) StartInference() error { return nil } func (d *testDriver) StartInference() error { return nil }
func (d *testDriver) StopInference() error { return nil } func (d *testDriver) StopInference() error { return nil }
func (d *testDriver) ReadInference() (*driver.InferenceResult, error) { return nil, nil } func (d *testDriver) ReadInference() (*driver.InferenceResult, error) { return nil, nil }

View File

@ -7,7 +7,7 @@ type DeviceDriver interface {
Connect() error Connect() error
Disconnect() error Disconnect() error
IsConnected() bool IsConnected() bool
Flash(modelPath string, progressCh chan<- FlashProgress) error Flash(modelPath string, opts FlashOptions, progressCh chan<- FlashProgress) error
StartInference() error StartInference() error
StopInference() error StopInference() error
ReadInference() (*InferenceResult, error) ReadInference() (*InferenceResult, error)
@ -39,6 +39,53 @@ const (
StatusDisconnected DeviceStatus = "disconnected" StatusDisconnected DeviceStatus = "disconnected"
) )
// FlashOptions 帶入 model metadata供 driver 在 load model 時傳給硬體 bridge。
//
// 為什麼用 struct 而不是多帶兩個參數:載入模型需要的 metadata 之後還會長
// 如前處理色彩格式、top-K用 struct 之後新增欄位不必再改 interface 簽章
// 與所有 test fake。
//
// 兩個欄位都是 optional —— 空值代表「未指定」bridge 端會 fallback 到既有的
// model id / 檔名 heuristics維持既有 detection 行為不變)。
type FlashOptions struct {
// TaskType 為 models.json 宣告的推論類型("classification" /
// "object_detection"。bridge 端有指定就不再用檔名猜測。
TaskType string
// Labels 是 class index → 顯示名稱的對應表,純顯示層用途、非推論必要輸入。
// 沒帶時 classification 輸出原始 enumclass_N、detection 沿用 COCO。
Labels []string
// InputWidth / InputHeight 是 models.json / metadata.json 宣告的模型輸入
// 尺寸。
//
// ⚠️ 這是**最後手段**不是可信來源bridge 端會優先向 SDK 問模型自己
// 宣告的 input tensor shape只有 SDK 沒回報時才用這組值。原因是這裡的
// 數字是人在上傳表單填的,實際案例是使用者填了 640x640 但模型根本不是
// 那個尺寸 —— 尺寸錯了 NPU 不會報錯,只會安靜地給出錯的推論結果。
//
// 零值 = 未宣告bridge 端會忽略並往下 fallback。
InputWidth int
InputHeight int
}
// InferenceOptions 是推論期可即時調整的解析設定。
//
// 與 FlashOptions 的分工FlashOptions 在「把 model 載進裝置」時一次性帶入;
// InferenceOptions 則是在**同一個已載入的 model 上**改變輸出的解讀方式,
// 不需要重燒KL520 重燒要數十秒)。
//
// 兩個欄位的零值語意刻意不同,因為要能表達「不動」與「清空」兩種意圖:
//
// TaskType == "" → 不改變當前解析方式
// Labels == nil → 不改變當前 label 表
// Labels == []string{} → 清空 label 表,回到原始 enumclass_N
//
// ⚠️ 因此 Labels 的判斷必須用 `!= nil` 而非 `len() > 0` —— 用長度判斷會讓
// 「清空」這個合法意圖永遠送不出去。
type InferenceOptions struct {
TaskType string
Labels []string
}
type FlashProgress struct { type FlashProgress struct {
Percent int `json:"percent"` Percent int `json:"percent"`
Stage string `json:"stage"` Stage string `json:"stage"`
@ -68,6 +115,9 @@ type InferenceResult struct {
type ClassResult struct { type ClassResult struct {
Label string `json:"label"` Label string `json:"label"`
Confidence float64 `json:"confidence"` Confidence float64 `json:"confidence"`
// ClassIndex 是模型輸出的原始類別索引,供前端在 label 缺漏時 fallback 顯示。
// 刻意不加 omitempty —— index 0 是合法類別omitempty 會把它吃掉。
ClassIndex int `json:"classIndex"`
} }
type DetectionResult struct { type DetectionResult struct {

View File

@ -508,6 +508,37 @@ func (d *KneronDriver) restartBridge() error {
return nil return nil
} }
// buildLoadModelCommand 組出 load_model 的 JSON-RPC payload。
//
// ⚠️ Flash 有四處 load_model 呼叫點(初次 + 三條 retry 路徑)。四處都必須走這個
// helper —— 如果任一處自己手寫 mapretry 成功後 task_type / labels 會遺失,
// 而且不會報錯bridge 會 fallback 到檔名猜測classification model 被誤判成
// YOLO 只會回空結果)。這種失敗完全靜默,所以刻意集中在單一建構點。
//
// 空值欄位不放進 payloadbridge 端把「缺欄位」與「空值」都當成未指定,
// 但少送欄位可讓 bridge log 的 "(not specified)" 語意精確。
func buildLoadModelCommand(modelPath string, opts driver.FlashOptions) map[string]interface{} {
cmd := map[string]interface{}{
"cmd": "load_model",
"path": modelPath,
}
if opts.TaskType != "" {
cmd["task_type"] = opts.TaskType
}
if len(opts.Labels) > 0 {
cmd["labels"] = opts.Labels
}
// 兩軸都要有值才送:只有一軸的宣告無法描述一個輸入尺寸,送過去只會讓
// bridge 端多做一次驗證再丟掉。
if opts.InputWidth > 0 && opts.InputHeight > 0 {
cmd["input_size"] = map[string]interface{}{
"width": opts.InputWidth,
"height": opts.InputHeight,
}
}
return cmd
}
// Flash loads a model onto the Kneron device. Progress is reported through // Flash loads a model onto the Kneron device. Progress is reported through
// the provided channel. // the provided channel.
// //
@ -516,7 +547,10 @@ func (d *KneronDriver) restartBridge() error {
// a full device reset + bridge restart + firmware reload. // a full device reset + bridge restart + firmware reload.
// - KL720 (flash-based): models can be freely reloaded. Error 40 // - KL720 (flash-based): models can be freely reloaded. Error 40
// should not occur; if it does, a simple retry is attempted first. // should not occur; if it does, a simple retry is attempted first.
func (d *KneronDriver) Flash(modelPath string, progressCh chan<- driver.FlashProgress) error { //
// opts 帶著 models.json 宣告的 model metadatataskType / labels會隨每一次
// load_model 送給 Python bridge —— 包含所有 retry 路徑。
func (d *KneronDriver) Flash(modelPath string, opts driver.FlashOptions, progressCh chan<- driver.FlashProgress) error {
d.mu.Lock() d.mu.Lock()
d.info.Status = driver.StatusFlashing d.info.Status = driver.StatusFlashing
pythonReady := d.pythonReady pythonReady := d.pythonReady
@ -554,10 +588,7 @@ func (d *KneronDriver) Flash(modelPath string, progressCh chan<- driver.FlashPro
} }
d.mu.Lock() d.mu.Lock()
_, err := d.sendCommand(map[string]interface{}{ _, err := d.sendCommand(buildLoadModelCommand(modelPath, opts))
"cmd": "load_model",
"path": modelPath,
})
d.mu.Unlock() d.mu.Unlock()
// Handle retryable errors (error 40, broken pipe). // Handle retryable errors (error 40, broken pipe).
@ -582,10 +613,7 @@ func (d *KneronDriver) Flash(modelPath string, progressCh chan<- driver.FlashPro
} }
d.mu.Lock() d.mu.Lock()
_, err = d.sendCommand(map[string]interface{}{ _, err = d.sendCommand(buildLoadModelCommand(modelPath, opts))
"cmd": "load_model",
"path": modelPath,
})
d.mu.Unlock() d.mu.Unlock()
// If still failing, fall back to bridge restart as last resort. // If still failing, fall back to bridge restart as last resort.
@ -599,10 +627,7 @@ func (d *KneronDriver) Flash(modelPath string, progressCh chan<- driver.FlashPro
} }
d.mu.Lock() d.mu.Lock()
d.info.Status = driver.StatusFlashing d.info.Status = driver.StatusFlashing
_, err = d.sendCommand(map[string]interface{}{ _, err = d.sendCommand(buildLoadModelCommand(modelPath, opts))
"cmd": "load_model",
"path": modelPath,
})
d.mu.Unlock() d.mu.Unlock()
} }
} else { } else {
@ -626,10 +651,7 @@ func (d *KneronDriver) Flash(modelPath string, progressCh chan<- driver.FlashPro
d.driverLog("INFO", "[kneron] bridge restarted, retrying load_model...") d.driverLog("INFO", "[kneron] bridge restarted, retrying load_model...")
d.mu.Lock() d.mu.Lock()
d.info.Status = driver.StatusFlashing d.info.Status = driver.StatusFlashing
_, err = d.sendCommand(map[string]interface{}{ _, err = d.sendCommand(buildLoadModelCommand(modelPath, opts))
"cmd": "load_model",
"path": modelPath,
})
d.mu.Unlock() d.mu.Unlock()
} }
} }
@ -697,6 +719,58 @@ func (d *KneronDriver) Flash(modelPath string, progressCh chan<- driver.FlashPro
return nil return nil
} }
// buildSetInferenceOptionsCommand 組出 set_inference_options 的 JSON-RPC payload。
//
// 與 buildLoadModelCommand 的關鍵差異:這裡用「欄位在不在」表達意圖,所以
// **不能**沿用「空值就不放進 payload」的規則 ——
//
// opts.TaskType == "" → 不帶 task_type 欄位 → bridge 保留當前解析方式
// opts.Labels == nil → 不帶 labels 欄位 → bridge 保留當前 label 表
// opts.Labels == [] → 帶空陣列 → bridge 清掉 label 表
//
// 最後那條是刻意要能表達的狀態(使用者上傳錯 label 想清掉)。若照 load_model
// 的規則用 len()>0 判斷,「清掉」就永遠送不出去、變成靜默無效的操作。
func buildSetInferenceOptionsCommand(opts driver.InferenceOptions) map[string]interface{} {
cmd := map[string]interface{}{
"cmd": "set_inference_options",
}
if opts.TaskType != "" {
cmd["task_type"] = opts.TaskType
}
if opts.Labels != nil {
cmd["labels"] = opts.Labels
}
return cmd
}
// SetInferenceOptions 在**不重新載入模型**的前提下,更新解析方式與 label 表。
//
// KL520 一次只能載一個 model、換 model 要重燒(數十秒);但「怎麼解析輸出」
// 與「index 顯示成什麼名字」都只是 post-process可以即時切換。
//
// 這個 method 刻意不放進 driver.DeviceDriver 介面 —— 它是 Kneron 特有能力,
// 放進去會逼三個既有 test fake 都跟著改。呼叫端改用窄介面 type-assert
// (同 firmware.UpgradeDriver 的做法)。
func (d *KneronDriver) SetInferenceOptions(opts driver.InferenceOptions) error {
d.mu.Lock()
defer d.mu.Unlock()
if !d.pythonReady {
return fmt.Errorf("hardware bridge is not running — device may not be connected")
}
if d.modelLoaded == "" {
return fmt.Errorf("no model loaded on device — flash a model first")
}
if _, err := d.sendCommand(buildSetInferenceOptionsCommand(opts)); err != nil {
return fmt.Errorf("set inference options failed: %w", err)
}
d.driverLog("INFO", "[kneron] inference options updated (taskType=%q, labels=%d)",
opts.TaskType, len(opts.Labels))
return nil
}
// StartInference begins continuous inference mode. // StartInference begins continuous inference mode.
func (d *KneronDriver) StartInference() error { func (d *KneronDriver) StartInference() error {
d.mu.Lock() d.mu.Lock()

View File

@ -0,0 +1,313 @@
package kneron
import (
"encoding/json"
"go/ast"
"go/parser"
"go/token"
"reflect"
"testing"
"visiona-local/server/internal/driver"
)
// TestBuildLoadModelCommand_WithMetadata有帶 metadata 時 payload 含
// task_type + labels欄位名必須與 kneron_bridge.py handle_load_model 的
// params.get("task_type") / params.get("labels") 完全一致。
func TestBuildLoadModelCommand_WithMetadata(t *testing.T) {
cmd := buildLoadModelCommand("/models/rps.nef", driver.FlashOptions{
TaskType: "classification",
Labels: []string{"剪刀", "石頭", "布"},
})
if cmd["cmd"] != "load_model" {
t.Errorf("cmd = %v, want load_model", cmd["cmd"])
}
if cmd["path"] != "/models/rps.nef" {
t.Errorf("path = %v, want /models/rps.nef", cmd["path"])
}
if cmd["task_type"] != "classification" {
t.Errorf("task_type = %v, want classification", cmd["task_type"])
}
labels, ok := cmd["labels"].([]string)
if !ok {
t.Fatalf("labels type = %T, want []string", cmd["labels"])
}
if !reflect.DeepEqual(labels, []string{"剪刀", "石頭", "布"}) {
t.Errorf("labels = %v, want [剪刀 石頭 布]", labels)
}
}
// TestBuildLoadModelCommand_EmptyOptionsOmitsFields沒帶 metadata 時不送
// task_type / labels讓 bridge 走既有 heuristics既有 detection 行為不變)。
func TestBuildLoadModelCommand_EmptyOptionsOmitsFields(t *testing.T) {
cmd := buildLoadModelCommand("/models/fcos.nef", driver.FlashOptions{})
if _, exists := cmd["task_type"]; exists {
t.Errorf("task_type should be omitted when empty, got %v", cmd["task_type"])
}
if _, exists := cmd["labels"]; exists {
t.Errorf("labels should be omitted when empty, got %v", cmd["labels"])
}
if len(cmd) != 2 {
t.Errorf("payload keys = %d (%v), want only cmd+path", len(cmd), cmd)
}
}
// TestBuildLoadModelCommand_EmptyLabelsOmittedlabels 為 non-nil 空 slice 時
// 也要省略 —— 送 [] 會讓 bridge 端 _sanitize_labels 走「空 list 視為未提供」,
// 語意雖同但多送無意義欄位。
func TestBuildLoadModelCommand_EmptyLabelsOmitted(t *testing.T) {
cmd := buildLoadModelCommand("/m.nef", driver.FlashOptions{
TaskType: "object_detection",
Labels: []string{},
})
if _, exists := cmd["labels"]; exists {
t.Errorf("empty labels should be omitted, got %v", cmd["labels"])
}
if cmd["task_type"] != "object_detection" {
t.Errorf("task_type = %v, want object_detection", cmd["task_type"])
}
}
// TestBuildLoadModelCommand_SerializesToBridgeContractpayload 經 JSON 編碼後
// 必須是 bridge 能吃的形狀labels 是 JSON array of string不是物件
// sendCommand 實際就是把 map 丟給 json.Marshal 送進 stdin。
func TestBuildLoadModelCommand_SerializesToBridgeContract(t *testing.T) {
cmd := buildLoadModelCommand("/m.nef", driver.FlashOptions{
TaskType: "classification",
Labels: []string{"a", "b"},
})
data, err := json.Marshal(cmd)
if err != nil {
t.Fatalf("marshal failed: %v", err)
}
var decoded map[string]interface{}
if err := json.Unmarshal(data, &decoded); err != nil {
t.Fatalf("unmarshal failed: %v", err)
}
if decoded["task_type"] != "classification" {
t.Errorf("task_type after roundtrip = %v", decoded["task_type"])
}
rawLabels, ok := decoded["labels"].([]interface{})
if !ok {
t.Fatalf("labels after roundtrip type = %T, want JSON array", decoded["labels"])
}
if len(rawLabels) != 2 || rawLabels[0] != "a" || rawLabels[1] != "b" {
t.Errorf("labels after roundtrip = %v, want [a b]", rawLabels)
}
}
// TestFlashUsesBuilderForEveryLoadModelCall 是本次改動最重要的一條測試。
//
// Flash 有四處 load_model 呼叫點(初次 + KL720 簡單 retry + KL720 restart retry
// + KL520 restart retry。漏改任一處的後果是「retry 成功後 model metadata 靜默
// 遺失」—— 不會報錯、不會 panic只會讓 classification model 被誤判成 YOLO 而
// 回傳空結果極難從現象追回根因plan §7 R-5
//
// 用 AST 掃 Flash 函式本體,斷言:
// 1. 函式內沒有任何自己手寫的 load_model map literal
// 2. 所有 sendCommand 的 load_model 都經過 buildLoadModelCommand
// 3. 呼叫點數量 == 4將來新增 retry 路徑忘了帶 opts 時,這條會亮)
func TestFlashUsesBuilderForEveryLoadModelCall(t *testing.T) {
fset := token.NewFileSet()
file, err := parser.ParseFile(fset, "kl720_driver.go", nil, 0)
if err != nil {
t.Fatalf("parse kl720_driver.go: %v", err)
}
var flashFn *ast.FuncDecl
for _, decl := range file.Decls {
fn, ok := decl.(*ast.FuncDecl)
if !ok || fn.Name.Name != "Flash" || fn.Recv == nil {
continue
}
flashFn = fn
break
}
if flashFn == nil {
t.Fatal("Flash method not found in kl720_driver.go")
}
builderCalls := 0
rawLiterals := 0
ast.Inspect(flashFn, func(n ast.Node) bool {
switch node := n.(type) {
case *ast.CallExpr:
if ident, ok := node.Fun.(*ast.Ident); ok && ident.Name == "buildLoadModelCommand" {
builderCalls++
}
case *ast.CompositeLit:
// 偵測 Flash 內自己手寫的 map其中含 "load_model" 字串。
for _, elt := range node.Elts {
kv, ok := elt.(*ast.KeyValueExpr)
if !ok {
continue
}
lit, ok := kv.Value.(*ast.BasicLit)
if ok && lit.Kind == token.STRING && lit.Value == `"load_model"` {
rawLiterals++
}
}
}
return true
})
if rawLiterals != 0 {
t.Errorf("Flash 內有 %d 個手寫的 load_model map literal"+
"所有呼叫點都必須走 buildLoadModelCommand否則 retry 後 "+
"task_type/labels 會靜默遺失", rawLiterals)
}
const wantCallSites = 4
if builderCalls != wantCallSites {
t.Errorf("buildLoadModelCommand 呼叫點 = %dwant %d "+
"(初次 + KL720 retry + KL720 restart retry + KL520 restart retry)。"+
"若確實新增/移除了 retry 路徑,請確認新路徑有帶 opts 後再更新此數字",
builderCalls, wantCallSites)
}
}
// TestParseInferenceResult_ClassIndexPreservedbridge 回傳的 classIndex 要
// 進得了 Go struct。加 ClassIndex 欄位前,這個值會被 encoding/json 靜默丟棄。
func TestParseInferenceResult_ClassIndexPreserved(t *testing.T) {
resp := map[string]interface{}{
"taskType": "classification",
"timestamp": float64(1721545200000),
"latencyMs": 45.2,
"classifications": []interface{}{
map[string]interface{}{"label": "石頭", "confidence": 0.94, "classIndex": float64(1)},
map[string]interface{}{"label": "class_0", "confidence": 0.04, "classIndex": float64(0)},
},
}
result, err := parseInferenceResult(resp)
if err != nil {
t.Fatalf("parseInferenceResult failed: %v", err)
}
if result.TaskType != "classification" {
t.Errorf("TaskType = %q, want classification", result.TaskType)
}
if len(result.Classifications) != 2 {
t.Fatalf("Classifications = %d, want 2", len(result.Classifications))
}
if result.Classifications[0].ClassIndex != 1 {
t.Errorf("Classifications[0].ClassIndex = %d, want 1", result.Classifications[0].ClassIndex)
}
// index 0 是合法類別 —— 若 struct tag 誤加 omitempty這筆會在序列化時消失。
if result.Classifications[1].ClassIndex != 0 {
t.Errorf("Classifications[1].ClassIndex = %d, want 0", result.Classifications[1].ClassIndex)
}
}
// TestClassResult_ClassIndexZeroNotOmittedindex 0 必須出現在送給前端的 JSON。
// 這是 omitempty 會踩的陷阱 —— 前端拿不到 classIndex 就無法做 fallback 顯示。
func TestClassResult_ClassIndexZeroNotOmitted(t *testing.T) {
data, err := json.Marshal(driver.ClassResult{Label: "class_0", Confidence: 0.9, ClassIndex: 0})
if err != nil {
t.Fatalf("marshal failed: %v", err)
}
var decoded map[string]interface{}
if err := json.Unmarshal(data, &decoded); err != nil {
t.Fatalf("unmarshal failed: %v", err)
}
if _, exists := decoded["classIndex"]; !exists {
t.Errorf("classIndex 不見了(大概是加了 omitempty%s", data)
}
if decoded["classIndex"] != float64(0) {
t.Errorf("classIndex = %v, want 0", decoded["classIndex"])
}
}
// TestBuildLoadModelCommand_InputSizeIncluded宣告的 input size 要送到 bridge
// 欄位名與巢狀結構必須與 kneron_bridge.py 的
// _normalize_declared_input_size(params.get("input_size")) 一致。
func TestBuildLoadModelCommand_InputSizeIncluded(t *testing.T) {
cmd := buildLoadModelCommand("/models/rps.nef", driver.FlashOptions{
TaskType: "classification",
InputWidth: 320,
InputHeight: 256,
})
size, ok := cmd["input_size"].(map[string]interface{})
if !ok {
t.Fatalf("input_size type = %T, want map[string]interface{}", cmd["input_size"])
}
if size["width"] != 320 {
t.Errorf("input_size.width = %v, want 320", size["width"])
}
// 高度不可被壓成寬度 —— 非正方形模型兩軸必須各自送出。
if size["height"] != 256 {
t.Errorf("input_size.height = %v, want 256", size["height"])
}
}
// TestBuildLoadModelCommand_ZeroInputSizeOmittedmodels.json 沒填 inputSize 時
// 兩軸都是 0送 0 過去只會讓 bridge 多驗一次再丟掉,且會讓 log 的
// "(not specified)" 語意失真。
func TestBuildLoadModelCommand_ZeroInputSizeOmitted(t *testing.T) {
cmd := buildLoadModelCommand("/models/fcos.nef", driver.FlashOptions{
TaskType: "object_detection",
})
if _, exists := cmd["input_size"]; exists {
t.Errorf("input_size should be omitted when zero, got %v", cmd["input_size"])
}
}
// TestBuildLoadModelCommand_PartialInputSizeOmitted只有一軸的宣告無法描述
// 一個輸入尺寸。半套送出比不送更危險 —— bridge 端會看到一個看似有效的來源。
func TestBuildLoadModelCommand_PartialInputSizeOmitted(t *testing.T) {
for _, tc := range []struct {
name string
width int
height int
}{
{"only width", 320, 0},
{"only height", 0, 320},
{"negative width", -1, 320},
} {
t.Run(tc.name, func(t *testing.T) {
cmd := buildLoadModelCommand("/models/m.nef", driver.FlashOptions{
InputWidth: tc.width,
InputHeight: tc.height,
})
if _, exists := cmd["input_size"]; exists {
t.Errorf("input_size should be omitted, got %v", cmd["input_size"])
}
})
}
}
// TestBuildLoadModelCommand_InputSizeSerializesToBridgeShapepayload 實際被
// JSON 序列化後的形狀,就是 bridge 端會 parse 到的東西。用序列化後的結果斷言
// 可避免「Go 端看起來對、上 wire 後欄位名或巢狀層級不同」。
func TestBuildLoadModelCommand_InputSizeSerializesToBridgeShape(t *testing.T) {
cmd := buildLoadModelCommand("/models/rps.nef", driver.FlashOptions{
InputWidth: 320,
InputHeight: 320,
})
data, err := json.Marshal(cmd)
if err != nil {
t.Fatalf("marshal failed: %v", err)
}
var decoded struct {
InputSize struct {
Width int `json:"width"`
Height int `json:"height"`
} `json:"input_size"`
}
if err := json.Unmarshal(data, &decoded); err != nil {
t.Fatalf("unmarshal failed: %v", err)
}
if decoded.InputSize.Width != 320 || decoded.InputSize.Height != 320 {
t.Errorf("input_size = %dx%d, want 320x320 (payload: %s)",
decoded.InputSize.Width, decoded.InputSize.Height, data)
}
}

View File

@ -0,0 +1,180 @@
package kneron
// set_inference_options_test.go — M4推論期切換解析方式 / label 表的
// JSON-RPC payload 契約測試。
//
// 這裡的核心不是「欄位名對不對」,而是 **nil / 空陣列 / 缺欄位三種狀態的
// 語意必須各自可表達**
//
// 缺 task_type 欄位 → bridge 保留當前解析方式
// 缺 labels 欄位 → bridge 保留當前 label 表
// labels: [] → bridge 清空 label 表(回到原始 enum
//
// 若照 load_model 的規則用 `len(labels) > 0` 判斷是否放進 payload第三種
// 狀態就永遠送不出去 —— 使用者按「清除標籤」會拿到 200 但什麼都沒發生。
// 這正是這個功能要防的靜默失敗,所以逐條釘死。
import (
"encoding/json"
"strings"
"testing"
"visiona-local/server/internal/driver"
)
func TestBuildSetInferenceOptionsCommand_CommandName(t *testing.T) {
cmd := buildSetInferenceOptionsCommand(driver.InferenceOptions{TaskType: "classification"})
// 必須與 kneron_bridge.py main() dispatch 的字串完全一致。
if cmd["cmd"] != "set_inference_options" {
t.Errorf("cmd = %v, want set_inference_options", cmd["cmd"])
}
}
func TestBuildSetInferenceOptionsCommand_TaskTypeOnly(t *testing.T) {
cmd := buildSetInferenceOptionsCommand(driver.InferenceOptions{TaskType: "classification"})
if cmd["task_type"] != "classification" {
t.Errorf("task_type = %v, want classification", cmd["task_type"])
}
if _, present := cmd["labels"]; present {
t.Error("沒指定 labels 時不可放進 payload —— bridge 會誤以為要改 label 表")
}
}
func TestBuildSetInferenceOptionsCommand_LabelsOnly(t *testing.T) {
cmd := buildSetInferenceOptionsCommand(driver.InferenceOptions{
Labels: []string{"剪刀", "石頭", "布"},
})
if _, present := cmd["task_type"]; present {
t.Error("沒指定 task_type 時不可放進 payload —— bridge 會誤以為要切解析方式")
}
labels, ok := cmd["labels"].([]string)
if !ok {
t.Fatalf("labels type = %T, want []string", cmd["labels"])
}
if len(labels) != 3 || labels[0] != "剪刀" {
t.Errorf("labels = %v", labels)
}
}
// ⭐ 本檔最重要的一條:空陣列必須真的被送出去。
func TestBuildSetInferenceOptionsCommand_EmptyLabelsIsSent(t *testing.T) {
cmd := buildSetInferenceOptionsCommand(driver.InferenceOptions{
Labels: []string{},
})
raw, present := cmd["labels"]
if !present {
t.Fatal("空 labels 陣列沒被送出 —— 「清空 label 表」的意圖丟失了。" +
"(是不是用 len(opts.Labels) > 0 判斷?要用 != nil")
}
labels, ok := raw.([]string)
if !ok {
t.Fatalf("labels type = %T, want []string", raw)
}
if len(labels) != 0 {
t.Errorf("len(labels) = %d, want 0", len(labels))
}
}
// 對照組nil 才代表「不動」。
func TestBuildSetInferenceOptionsCommand_NilLabelsIsOmitted(t *testing.T) {
cmd := buildSetInferenceOptionsCommand(driver.InferenceOptions{
TaskType: "classification",
Labels: nil,
})
if _, present := cmd["labels"]; present {
t.Error("nil labels 不可放進 payload —— nil 代表「保留當前 label 表」")
}
}
// 序列化後空陣列要是 JSON 的 [],不能變成 null。
// bridge 端對 null 與 [] 的處理不同null → 保留(依 handler 邏輯 pending=None
// [] → 清空。變成 null 會讓清空意圖在 wire 上就失真。
func TestBuildSetInferenceOptionsCommand_EmptyLabelsMarshalsToArray(t *testing.T) {
cmd := buildSetInferenceOptionsCommand(driver.InferenceOptions{Labels: []string{}})
data, err := json.Marshal(cmd)
if err != nil {
t.Fatalf("marshal: %v", err)
}
var decoded map[string]interface{}
if err := json.Unmarshal(data, &decoded); err != nil {
t.Fatalf("unmarshal: %v", err)
}
raw, present := decoded["labels"]
if !present {
t.Fatalf("labels 欄位不見了:%s", data)
}
arr, ok := raw.([]interface{})
if !ok {
t.Fatalf("labels 序列化成 %T%swant JSON array —— null 會被 bridge "+
"解讀成「不動」而非「清空」", raw, data)
}
if len(arr) != 0 {
t.Errorf("len = %d, want 0", len(arr))
}
}
func TestBuildSetInferenceOptionsCommand_Both(t *testing.T) {
cmd := buildSetInferenceOptionsCommand(driver.InferenceOptions{
TaskType: "object_detection",
Labels: []string{"a", "", "c"},
})
if cmd["task_type"] != "object_detection" {
t.Errorf("task_type = %v", cmd["task_type"])
}
labels, ok := cmd["labels"].([]string)
if !ok {
t.Fatalf("labels type = %T", cmd["labels"])
}
// 稀疏佔位的空字串要原樣送過去 —— 位置就是 class index
// 壓縮掉會讓所有後面的 index 位移。
if len(labels) != 3 || labels[1] != "" {
t.Errorf("labels = %v, want [a c](稀疏佔位必須保留)", labels)
}
}
// 全空的 options 不該被 builder 擋(那是上層 handler 的責任),但也不該
// 憑空生出欄位 —— 只帶 cmd。這條確保 builder 保持「純翻譯」不做決策。
func TestBuildSetInferenceOptionsCommand_ZeroValueOnlyHasCmd(t *testing.T) {
cmd := buildSetInferenceOptionsCommand(driver.InferenceOptions{})
if len(cmd) != 1 {
t.Errorf("payload = %v, want 只有 cmd 一個鍵", cmd)
}
}
// ── driver 前置條件 ──────────────────────────────────────────────────
func TestSetInferenceOptions_RequiresBridge(t *testing.T) {
d := &KneronDriver{}
err := d.SetInferenceOptions(driver.InferenceOptions{TaskType: "classification"})
if err == nil {
t.Fatal("bridge 沒跑時應該回錯")
}
if !strings.Contains(err.Error(), "bridge is not running") {
t.Errorf("err = %v, 應說明 bridge 未執行", err)
}
}
func TestSetInferenceOptions_RequiresLoadedModel(t *testing.T) {
// bridge 就緒但沒載 model切解析方式沒有意義且 bridge 端也會拒絕。
// 在 driver 層先擋掉,錯誤訊息才能指出「要先燒錄模型」。
d := &KneronDriver{pythonReady: true}
err := d.SetInferenceOptions(driver.InferenceOptions{TaskType: "classification"})
if err == nil {
t.Fatal("沒載 model 時應該回錯")
}
if !strings.Contains(err.Error(), "no model loaded") {
t.Errorf("err = %v, 應說明尚未載入模型", err)
}
}

View File

@ -12,6 +12,28 @@ import (
"visiona-local/server/internal/model" "visiona-local/server/internal/model"
) )
// 可指定的推論種類。與 models.json / 前端同一組值。
//
// models.json 另有 segmentation / pose_estimation但 Python bridge 與前端都還
// 沒有對應的解析路徑,所以只開放這兩種真的能產出結果的種類。
//
// 燒錄時不再讓使用者選推論種類(改由推論期的
// POST /devices/:id/inference/options 即時切換、不必重燒),但這組常數與
// IsValidTaskTypeOverride 仍是「解析方式」的值域來源,由該 endpoint 沿用,
// 讓 wire 上永遠只有一組合法命名。
const (
TaskTypeClassification = "classification"
TaskTypeObjectDetection = "object_detection"
)
// IsValidTaskTypeOverride 回報 taskType 是否為合法的解析方式覆寫值。
//
// 空字串(未指定)不算合法覆寫 —— 呼叫端要自己先判斷「有沒有要覆寫」,
// 這樣「未指定」與「指定了但打錯字」不會被混為一談。
func IsValidTaskTypeOverride(taskType string) bool {
return taskType == TaskTypeClassification || taskType == TaskTypeObjectDetection
}
func isCompatible(modelHardware []string, deviceType string) bool { func isCompatible(modelHardware []string, deviceType string) bool {
dt := strings.ToUpper(deviceType) dt := strings.ToUpper(deviceType)
for _, hw := range modelHardware { for _, hw := range modelHardware {
@ -84,6 +106,11 @@ func (s *Service) CleanupTask(taskID string) {
s.tracker.Remove(taskID) s.tracker.Remove(taskID)
} }
// StartFlash 把 model 載入到裝置。
//
// 推論種類一律用 models.json 宣告的值。使用者若要改解析方式,走推論期的
// POST /devices/:id/inference/options —— 那條路徑不必重燒、可即時切換,
// 功能完全涵蓋燒錄時再選一次的舊做法。
func (s *Service) StartFlash(deviceID, modelID string) (string, <-chan driver.FlashProgress, error) { func (s *Service) StartFlash(deviceID, modelID string) (string, <-chan driver.FlashProgress, error) {
session, err := s.deviceMgr.GetDevice(deviceID) session, err := s.deviceMgr.GetDevice(deviceID)
if err != nil { if err != nil {
@ -137,7 +164,18 @@ func (s *Service) StartFlash(deviceID, modelID string) (string, <-chan driver.Fl
time.Sleep(500 * time.Millisecond) time.Sleep(500 * time.Millisecond)
flashErr := session.Driver.Flash(modelPath, task.ProgressCh) // 把 models.json 宣告的 metadata 一起帶下去 —— bridge 端有 taskType
// 就不再靠檔名猜 model type自訂模型存成 model.nef、檔名沒有關鍵字
// 猜測必定落到 detection 分支。labels 純顯示層、沒有也能跑。
//
// inputSize 是宣告值、**優先序最低**bridge 端會先問 SDK 模型自己
// 宣告的 input shape只有問不到才用這裡的值這欄是人填的可能亂填
flashErr := session.Driver.Flash(modelPath, driver.FlashOptions{
TaskType: m.TaskType,
Labels: m.Labels,
InputWidth: m.InputSize.Width,
InputHeight: m.InputSize.Height,
}, task.ProgressCh)
// Flash 完成或失敗後driver 不會再寫 progressCh安全地寫 error 訊息然後 close。 // Flash 完成或失敗後driver 不會再寫 progressCh安全地寫 error 訊息然後 close。
if flashErr != nil { if flashErr != nil {

View File

@ -0,0 +1,190 @@
package flash
import (
"bytes"
"go/ast"
"go/parser"
"go/printer"
"go/token"
"reflect"
"strings"
"testing"
"visiona-local/server/internal/driver"
"visiona-local/server/internal/model"
)
// recordingDriver 記錄 Flash 收到的 opts用來驗證 model metadata 有被傳下去。
type recordingDriver struct {
gotPath string
gotOpts driver.FlashOptions
called bool
}
func (d *recordingDriver) Info() driver.DeviceInfo { return driver.DeviceInfo{} }
func (d *recordingDriver) Connect() error { return nil }
func (d *recordingDriver) Disconnect() error { return nil }
func (d *recordingDriver) IsConnected() bool { return true }
func (d *recordingDriver) Flash(modelPath string, opts driver.FlashOptions, _ chan<- driver.FlashProgress) error {
d.called = true
d.gotPath = modelPath
d.gotOpts = opts
return nil
}
func (d *recordingDriver) StartInference() error { return nil }
func (d *recordingDriver) StopInference() error { return nil }
func (d *recordingDriver) ReadInference() (*driver.InferenceResult, error) { return nil, nil }
func (d *recordingDriver) RunInference(_ []byte) (*driver.InferenceResult, error) {
return nil, nil
}
func (d *recordingDriver) GetModelInfo() (*driver.ModelInfo, error) { return nil, nil }
// flashOptionsFor 複製 StartFlash 內部組 FlashOptions 的邏輯。
//
// 為什麼不直接跑 StartFlashService 依賴具體的 *device.Manager而 Manager 的
// sessions map 未匯出、只能由真實硬體偵測填入,沒有注入 fake session 的接縫。
// 為了測試而改 production 的依賴結構超出 M2 範圍,所以這裡改為釘住「送進
// driver 的 FlashOptions 必須完整帶著 model 的 TaskType/Labels」這個契約。
func flashOptionsFor(m model.Model) driver.FlashOptions {
return driver.FlashOptions{
TaskType: m.TaskType,
Labels: m.Labels,
InputWidth: m.InputSize.Width,
InputHeight: m.InputSize.Height,
}
}
// TestFlashOptions_CarriesClassificationMetadataclassification model 的
// taskType + labels 要完整傳到 driver不能像改動前一樣只傳 path 就丟棄。
func TestFlashOptions_CarriesClassificationMetadata(t *testing.T) {
m := model.Model{
ID: "custom-rps",
TaskType: "classification",
Labels: []string{"剪刀", "石頭", "布"},
}
d := &recordingDriver{}
opts := flashOptionsFor(m)
if err := d.Flash("/models/custom-rps/model.nef", opts, nil); err != nil {
t.Fatalf("Flash returned error: %v", err)
}
if !d.called {
t.Fatal("Flash was not called")
}
if d.gotOpts.TaskType != "classification" {
t.Errorf("TaskType = %q, want classification", d.gotOpts.TaskType)
}
if !reflect.DeepEqual(d.gotOpts.Labels, []string{"剪刀", "石頭", "布"}) {
t.Errorf("Labels = %v, want [剪刀 石頭 布]", d.gotOpts.Labels)
}
}
// TestFlashOptions_CarriesDetectionMetadata既有 detection model 走同一條路,
// taskType 為 object_detection —— bridge 端收到後仍走 detection 分支,行為不變。
func TestFlashOptions_CarriesDetectionMetadata(t *testing.T) {
m := model.Model{
ID: "kl520-fcos-detection",
TaskType: "object_detection",
Labels: []string{"person", "bicycle", "car"},
}
opts := flashOptionsFor(m)
if opts.TaskType != "object_detection" {
t.Errorf("TaskType = %q, want object_detection", opts.TaskType)
}
if len(opts.Labels) != 3 {
t.Errorf("Labels = %v, want 3 entries", opts.Labels)
}
}
// TestFlashOptions_EmptyMetadataIsZeroValuemodel 沒宣告 taskType/labels 時
// 送出的是零值driver 端會據此省略欄位,讓 bridge fallback 到既有 heuristics。
func TestFlashOptions_EmptyMetadataIsZeroValue(t *testing.T) {
opts := flashOptionsFor(model.Model{ID: "bare"})
if opts.TaskType != "" {
t.Errorf("TaskType = %q, want empty", opts.TaskType)
}
if len(opts.Labels) != 0 {
t.Errorf("Labels = %v, want empty", opts.Labels)
}
}
// TestStartFlashPassesModelMetadata 釘住 StartFlash 原始碼真的有把 m.TaskType /
// m.Labels 傳進 Flash。
//
// 上面的測試只驗「FlashOptions 帶得動 metadata」無法防止有人把 service.go 改回
// 只傳 path —— 那正是改動前的 bug 形態metadata 被靜默丟棄、不會報錯)。
// 這裡直接掃 service.go 的 StartFlash 本體補上這個缺口。
func TestStartFlashPassesModelMetadata(t *testing.T) {
src := readStartFlashSource(t)
for _, want := range []string{
"m.TaskType", "m.Labels", "driver.FlashOptions",
// 宣告的 input size 也要傳下去 —— 沒傳的話 bridge 在 SDK 問不到
// shape 時只能靠檔名猜,那正是「尺寸靜默錯誤」的來源。
"m.InputSize.Width", "m.InputSize.Height",
} {
if !strings.Contains(src, want) {
t.Errorf("StartFlash 原始碼缺少 %q —— model metadata 沒有被傳給 driver.Flash", want)
}
}
}
// TestFlashOptions_CarriesDeclaredInputSizemodels.json 宣告的 inputSize 要
// 完整帶到 driver。改動前這個欄位在推論鏈路上完全沒被讀過 —— 使用者在上傳
// 表單填的寬高毫無作用input size 全由檔名猜測決定。
func TestFlashOptions_CarriesDeclaredInputSize(t *testing.T) {
m := model.Model{
ID: "custom-rps",
TaskType: "classification",
InputSize: model.InputSize{Width: 320, Height: 256},
}
opts := flashOptionsFor(m)
if opts.InputWidth != 320 {
t.Errorf("InputWidth = %d, want 320", opts.InputWidth)
}
// 非正方形模型的高度不可被壓成寬度。
if opts.InputHeight != 256 {
t.Errorf("InputHeight = %d, want 256", opts.InputHeight)
}
}
// TestFlashOptions_MissingInputSizeIsZeromodels.json 沒填 inputSize 時是零值,
// driver 端會省略欄位bridge 據此 fallback既有 detection 模型走這條)。
func TestFlashOptions_MissingInputSizeIsZero(t *testing.T) {
opts := flashOptionsFor(model.Model{ID: "bare"})
if opts.InputWidth != 0 || opts.InputHeight != 0 {
t.Errorf("InputSize = %dx%d, want 0x0",
opts.InputWidth, opts.InputHeight)
}
}
// readStartFlashSource 取出 service.go 中 StartFlash 方法的原始碼文字。
func readStartFlashSource(t *testing.T) string {
t.Helper()
fset := token.NewFileSet()
file, err := parser.ParseFile(fset, "service.go", nil, 0)
if err != nil {
t.Fatalf("parse service.go: %v", err)
}
for _, decl := range file.Decls {
fn, ok := decl.(*ast.FuncDecl)
if !ok || fn.Name.Name != "StartFlash" || fn.Recv == nil {
continue
}
var buf bytes.Buffer
if err := printer.Fprint(&buf, fset, fn); err != nil {
t.Fatalf("print StartFlash: %v", err)
}
return buf.String()
}
t.Fatal("StartFlash method not found in service.go")
return ""
}

View File

@ -0,0 +1,233 @@
// Package labelfile 解析使用者上傳的 label 檔(`<index> <名稱>` 每行一筆),
// 產出可直接注入 Python bridge 的密集 []string。
//
// 為什麼是獨立 package解析規則全是純函式、與 HTTP / driver / model 儲存都
// 無關,抽出來才能用大量 table-driven 測試把容錯規則逐條釘死(規則來源見
// plan-classification-inference.md §3.2,該表已定死、實作不得自行發揮)。
package labelfile
import (
"bufio"
"bytes"
"fmt"
"strconv"
"strings"
"unicode"
"unicode/utf8"
)
const (
// MaxIndex 是允許的最大 class index
//
// 這是本功能最實際的 DoS 面plan §3.5 / §7 R-7解析結果要轉成密集
// []string一行 `999999999 x` 就會要求配置約 8GB 的 slice header 空間。
// 4095 對真實模型綽綽有餘ImageNet 1000 類、COCO 80 類)。
MaxIndex = 4095
// MaxLines 是允許的最大「有效內容行數」(不含空行與註解),與 MaxIndex
// 互為雙保險MaxIndex 擋單一巨大 indexMaxLines 擋「index 都合法但行數
// 爆量」。
//
// 為什麼剛好是 MaxIndex+1 而不是更大的值index 必須唯一且 <= MaxIndex
// 所以合法檔案最多就是 MaxIndex+1 行。設得比這更大會讓這道檢查永遠碰不到
// (重複 index 的檢查會先擋下),變成無效防禦 —— 有一個「看起來有防護但
// 其實跑不到」的常數,比沒有更糟。
MaxLines = MaxIndex + 1
// MaxFileSize 是允許的檔案大小上限bytes
// 3 類 label 約 30 bytes即使 4096 類中文標籤也遠低於 256 KB。
MaxFileSize = 256 * 1024
)
// Result 是一次成功解析的產物。
type Result struct {
// Labels 是密集陣列:位置 = class index稀疏處為空字串。
//
// 為什麼用密集 []string 而非 mapplan §3.2 A1Model.Labels 與
// FlashOptions.Labels 都已經是 []string改成 map 要動 Go struct、TS type、
// models.json 全部既有 model 與 upload handler。Python 端 _resolve_label
// 對空字串已會 fallback 回 class_N稀疏語意天然成立。
Labels []string
// LabelCount 是「實際有名稱的筆數」(不含補洞用的空字串)。
LabelCount int
// MaxIndex 是檔案中出現過的最大 index等於 len(Labels)-1。
MaxIndex int
}
// ParseError 帶行號的解析失敗。整檔拒絕(不做部分接受)—— 靜默略過壞行會讓
// 使用者拿到看似成功但標註錯位的結果,那比直接失敗糟得多。
type ParseError struct {
// Line 是 1-based 行號0 表示錯誤與特定行無關(如空檔、編碼問題)。
Line int
// Reason 是給使用者看的說明。
Reason string
}
func (e *ParseError) Error() string {
if e.Line > 0 {
return fmt.Sprintf("第 %d 行:%s", e.Line, e.Reason)
}
return e.Reason
}
// utf8BOM 是 UTF-8 位元組順序標記。Windows 記事本另存 UTF-8 會加它,
// 不 strip 的話第一行的 index token 會帶著 BOM 位元組而解析失敗。
var utf8BOM = []byte{0xEF, 0xBB, 0xBF}
// Parse 解析 label 檔內容。
//
// 容錯規則完全依照 plan §3.2 的表:
//
// 空行 / 純空白行 → 略過
// `#` 開頭 → 註解、略過
// 行尾 \rCRLF → trim
// 名稱含空白 → 只 split 第一個空白,其餘全算名稱
// index 非整數 / 負數 → 整檔拒絕 + 行號
// index 重複 → 整檔拒絕 + 行號
// index 不連續 / 不從 0 開始 → 接受,缺的位置補空字串
// 空檔 / 全空行 → 拒絕
// 非 UTF-8 → 拒絕BOM 先 strip
// 只有 index 沒名稱 → 拒絕 + 行號
func Parse(data []byte) (*Result, error) {
if len(data) > MaxFileSize {
return nil, &ParseError{
Reason: fmt.Sprintf("檔案過大(%d bytes上限為 %d bytes", len(data), MaxFileSize),
}
}
data = bytes.TrimPrefix(data, utf8BOM)
if !utf8.Valid(data) {
return nil, &ParseError{
Reason: "檔案不是有效的 UTF-8 編碼,請改存成 UTF-8 後再上傳",
}
}
// byIndex 保留原始的稀疏語意,最後才展開成密集陣列 —— 先展開的話,
// 「index 重複」與「index 不連續補洞」兩種情況會分不出來。
byIndex := make(map[int]string)
maxIndex := -1
contentLines := 0
scanner := bufio.NewScanner(bytes.NewReader(data))
// 單行上限放寬到 64KB預設 bufio 上限也是 64KB但預設 buffer 只有 4KB
// 起跳、長行會回 bufio.ErrTooLong 而不是我們自己的錯誤訊息。
scanner.Buffer(make([]byte, 0, 4096), 64*1024)
lineNo := 0
for scanner.Scan() {
lineNo++
line := strings.TrimRight(scanner.Text(), "\r")
trimmed := strings.TrimSpace(line)
if trimmed == "" {
continue
}
if strings.HasPrefix(trimmed, "#") {
continue
}
contentLines++
if contentLines > MaxLines {
return nil, &ParseError{
Line: lineNo,
Reason: fmt.Sprintf("標籤行數超過上限 %d", MaxLines),
}
}
idx, name, err := parseLine(trimmed)
if err != nil {
err.Line = lineNo
return nil, err
}
if _, dup := byIndex[idx]; dup {
return nil, &ParseError{
Line: lineNo,
Reason: fmt.Sprintf("index %d 重複出現", idx),
}
}
byIndex[idx] = name
if idx > maxIndex {
maxIndex = idx
}
}
if err := scanner.Err(); err != nil {
return nil, &ParseError{Reason: fmt.Sprintf("讀取檔案失敗:%v", err)}
}
if len(byIndex) == 0 {
return nil, &ParseError{Reason: "標籤檔沒有任何有效內容"}
}
labels := make([]string, maxIndex+1)
for idx, name := range byIndex {
labels[idx] = name
}
return &Result{
Labels: labels,
LabelCount: len(byIndex),
MaxIndex: maxIndex,
}, nil
}
// parseLine 解析單行 `<index> <名稱>`。回傳的 ParseError 不帶 Line由呼叫端補。
func parseLine(line string) (int, string, *ParseError) {
// 只切第一個空白:`0 traffic light` 必須解析成 {0: "traffic light"}。
// strings.Fields 會把名稱也切碎,所以刻意用 IndexFunc 自己找分界。
sep := strings.IndexFunc(line, unicode.IsSpace)
if sep < 0 {
return 0, "", &ParseError{
Reason: fmt.Sprintf("缺少標籤名稱(只有 %q格式應為 `<index> <名稱>`", line),
}
}
idxToken := line[:sep]
name := strings.TrimSpace(line[sep:])
if name == "" {
return 0, "", &ParseError{
Reason: fmt.Sprintf("index %s 後面缺少標籤名稱", idxToken),
}
}
idx, err := strconv.Atoi(idxToken)
if err != nil {
return 0, "", &ParseError{
Reason: fmt.Sprintf("index 必須為非負整數,收到 %q", idxToken),
}
}
if idx < 0 {
return 0, "", &ParseError{
Reason: fmt.Sprintf("index 必須為非負整數,收到 %q", idxToken),
}
}
if idx > MaxIndex {
return 0, "", &ParseError{
Reason: fmt.Sprintf("index %d 超過上限 %d", idx, MaxIndex),
}
}
if bad, ok := findControlChar(name); ok {
return 0, "", &ParseError{
Reason: fmt.Sprintf("標籤名稱含有不允許的控制字元U+%04X", bad),
}
}
return idx, name, nil
}
// findControlChar 找出名稱中的控制字元。標籤會直接被渲染到前端 DOM 與 canvas
// 控制字元(含 U+202E 這類 bidi override會造成顯示錯亂一律拒絕。
// Tab 已在 TrimSpace / 分隔判斷階段處理掉,這裡不需特別放行。
func findControlChar(name string) (rune, bool) {
for _, r := range name {
if unicode.IsControl(r) || (r >= 0x202A && r <= 0x202E) || (r >= 0x2066 && r <= 0x2069) {
return r, true
}
}
return 0, false
}

View File

@ -0,0 +1,423 @@
package labelfile
import (
"errors"
"fmt"
"strings"
"testing"
)
// asParseError 取出 *ParseError不是的話直接讓測試失敗。
func asParseError(t *testing.T, err error) *ParseError {
t.Helper()
if err == nil {
t.Fatalf("expected error, got nil")
}
var pe *ParseError
if !errors.As(err, &pe) {
t.Fatalf("expected *ParseError, got %T (%v)", err, err)
}
return pe
}
// ── Happy path ───────────────────────────────────────────────────────
func TestParse_RealWorldFile(t *testing.T) {
// 使用者提供的 labels.txt剪刀/石頭/布)逐位元組相同的內容。
got, err := Parse([]byte("0 剪刀\n1 石頭\n2 布\n"))
if err != nil {
t.Fatalf("Parse error: %v", err)
}
want := []string{"剪刀", "石頭", "布"}
if !equalStrings(got.Labels, want) {
t.Errorf("Labels = %v, want %v", got.Labels, want)
}
if got.LabelCount != 3 {
t.Errorf("LabelCount = %d, want 3", got.LabelCount)
}
if got.MaxIndex != 2 {
t.Errorf("MaxIndex = %d, want 2", got.MaxIndex)
}
}
func TestParse_NoTrailingNewline(t *testing.T) {
got, err := Parse([]byte("0 a\n1 b"))
if err != nil {
t.Fatalf("Parse error: %v", err)
}
if !equalStrings(got.Labels, []string{"a", "b"}) {
t.Errorf("Labels = %v", got.Labels)
}
}
// ── plan §3.2 容錯表:逐列 ────────────────────────────────────────────
// 表列 1空行 → 略過
func TestParse_TableRow_BlankLinesSkipped(t *testing.T) {
got, err := Parse([]byte("0 a\n\n\n1 b\n\n"))
if err != nil {
t.Fatalf("Parse error: %v", err)
}
if !equalStrings(got.Labels, []string{"a", "b"}) {
t.Errorf("Labels = %v, want [a b]", got.Labels)
}
}
// 表列 2只有空白的行 → 略過
func TestParse_TableRow_WhitespaceOnlyLinesSkipped(t *testing.T) {
got, err := Parse([]byte("0 a\n \n\t\t\n1 b\n"))
if err != nil {
t.Fatalf("Parse error: %v", err)
}
if !equalStrings(got.Labels, []string{"a", "b"}) {
t.Errorf("Labels = %v, want [a b]", got.Labels)
}
}
// 表列 3以 # 開頭 → 視為註解、略過
func TestParse_TableRow_CommentLinesSkipped(t *testing.T) {
got, err := Parse([]byte("# 這是註解\n0 a\n # 縮排註解也算\n1 b\n"))
if err != nil {
t.Fatalf("Parse error: %v", err)
}
if !equalStrings(got.Labels, []string{"a", "b"}) {
t.Errorf("Labels = %v, want [a b]", got.Labels)
}
}
// 表列 3 反例:`#` 出現在名稱中間不是註解
func TestParse_TableRow_HashInsideNameIsNotComment(t *testing.T) {
got, err := Parse([]byte("0 C#\n"))
if err != nil {
t.Fatalf("Parse error: %v", err)
}
if !equalStrings(got.Labels, []string{"C#"}) {
t.Errorf("Labels = %v, want [C#]", got.Labels)
}
}
// 表列 4行尾 \rCRLF→ trim 掉
func TestParse_TableRow_CRLFTrimmed(t *testing.T) {
got, err := Parse([]byte("0 剪刀\r\n1 石頭\r\n2 布\r\n"))
if err != nil {
t.Fatalf("Parse error: %v", err)
}
want := []string{"剪刀", "石頭", "布"}
if !equalStrings(got.Labels, want) {
t.Errorf("Labels = %v, want %v (CR 未被 trim)", got.Labels, want)
}
// 逐字元確認沒有殘留 \r —— equalStrings 若有 bug 可能漏掉。
for i, l := range got.Labels {
if strings.ContainsRune(l, '\r') {
t.Errorf("Labels[%d] = %q 仍含 CR", i, l)
}
}
}
// 表列 5名稱含空白 → 只 split 第一個空白、其餘全算名稱
func TestParse_TableRow_NameWithSpacesKeptWhole(t *testing.T) {
got, err := Parse([]byte("0 traffic light\n1 stop sign here\n"))
if err != nil {
t.Fatalf("Parse error: %v", err)
}
want := []string{"traffic light", "stop sign here"}
if !equalStrings(got.Labels, want) {
t.Errorf("Labels = %v, want %v", got.Labels, want)
}
}
// 表列 5 變體index 與名稱之間多個空白 / tab
func TestParse_TableRow_MultipleSeparatorWhitespaceCollapsed(t *testing.T) {
got, err := Parse([]byte("0\t\t剪刀\n1 石 頭\n"))
if err != nil {
t.Fatalf("Parse error: %v", err)
}
want := []string{"剪刀", "石 頭"}
if !equalStrings(got.Labels, want) {
t.Errorf("Labels = %v, want %v", got.Labels, want)
}
}
// 表列 6index 不是整數 → 整檔拒絕 + 行號
func TestParse_TableRow_NonIntegerIndexRejected(t *testing.T) {
cases := []struct {
name string
content string
wantLine int
}{
{"字母", "0 a\n1 b\nabc c\n", 3},
{"小數", "0 a\n1.5 b\n", 2},
{"十六進位", "0x1 a\n", 1},
{"含前導加號以外的雜訊", "0 a\n1_ b\n", 2},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
_, err := Parse([]byte(tc.content))
pe := asParseError(t, err)
if pe.Line != tc.wantLine {
t.Errorf("Line = %d, want %d (err=%v)", pe.Line, tc.wantLine, pe)
}
if !strings.Contains(pe.Reason, "非負整數") {
t.Errorf("Reason = %q, 應說明 index 必須為非負整數", pe.Reason)
}
})
}
}
// 表列 7index 為負數 → 整檔拒絕
func TestParse_TableRow_NegativeIndexRejected(t *testing.T) {
_, err := Parse([]byte("0 a\n-1 b\n"))
pe := asParseError(t, err)
if pe.Line != 2 {
t.Errorf("Line = %d, want 2", pe.Line)
}
}
// 表列 8index 重複 → 整檔拒絕 + 指出重複的 index
func TestParse_TableRow_DuplicateIndexRejected(t *testing.T) {
_, err := Parse([]byte("0 a\n1 b\n1 c\n"))
pe := asParseError(t, err)
if pe.Line != 3 {
t.Errorf("Line = %d, want 3", pe.Line)
}
if !strings.Contains(pe.Reason, "1") || !strings.Contains(pe.Reason, "重複") {
t.Errorf("Reason = %q, 應指出重複的 index", pe.Reason)
}
}
// 表列 8 變體:重複且名稱相同也一樣拒絕(不做「反正一樣就放行」的體貼)
func TestParse_TableRow_DuplicateIndexSameNameStillRejected(t *testing.T) {
_, err := Parse([]byte("0 a\n0 a\n"))
asParseError(t, err)
}
// 表列 9index 不連續 → 接受,缺的位置補空字串
func TestParse_TableRow_SparseIndexAccepted(t *testing.T) {
got, err := Parse([]byte("0 a\n1 b\n3 d\n"))
if err != nil {
t.Fatalf("Parse error: %v", err)
}
want := []string{"a", "b", "", "d"}
if !equalStrings(got.Labels, want) {
t.Errorf("Labels = %v, want %v", got.Labels, want)
}
if got.LabelCount != 3 {
t.Errorf("LabelCount = %d, want 3不含補洞的空字串", got.LabelCount)
}
if got.MaxIndex != 3 {
t.Errorf("MaxIndex = %d, want 3", got.MaxIndex)
}
}
// 表列 10index 不從 0 開始 → 接受
func TestParse_TableRow_IndexNotStartingAtZeroAccepted(t *testing.T) {
got, err := Parse([]byte("5 e\n6 f\n"))
if err != nil {
t.Fatalf("Parse error: %v", err)
}
want := []string{"", "", "", "", "", "e", "f"}
if !equalStrings(got.Labels, want) {
t.Errorf("Labels = %v, want %v", got.Labels, want)
}
if got.LabelCount != 2 {
t.Errorf("LabelCount = %d, want 2", got.LabelCount)
}
}
// 表列 10 變體亂序也接受index 決定位置、不是出現順序)
func TestParse_TableRow_OutOfOrderIndexAccepted(t *testing.T) {
got, err := Parse([]byte("2 布\n0 剪刀\n1 石頭\n"))
if err != nil {
t.Fatalf("Parse error: %v", err)
}
want := []string{"剪刀", "石頭", "布"}
if !equalStrings(got.Labels, want) {
t.Errorf("Labels = %v, want %v", got.Labels, want)
}
}
// 表列 11檔案為空 / 全是空行 → 拒絕
func TestParse_TableRow_EmptyFileRejected(t *testing.T) {
cases := map[string]string{
"完全空": "",
"只有換行": "\n\n\n",
"只有空白": " \n\t\n",
"只有註解": "# nothing here\n# still nothing\n",
"只有 BOM": "\xEF\xBB\xBF",
}
for name, content := range cases {
t.Run(name, func(t *testing.T) {
_, err := Parse([]byte(content))
pe := asParseError(t, err)
if pe.Line != 0 {
t.Errorf("Line = %d, want 0與特定行無關", pe.Line)
}
if !strings.Contains(pe.Reason, "有效內容") {
t.Errorf("Reason = %q", pe.Reason)
}
})
}
}
// 表列 12編碼非 UTF-8 → 拒絕BOM 要 strip
func TestParse_TableRow_InvalidUTF8Rejected(t *testing.T) {
// Big5 的「剪刀」= 0xB0 0x45 0xA4 0x4D在 UTF-8 下是非法序列。
content := append([]byte("0 "), 0xB0, 0x45, 0xA4, 0x4D, '\n')
_, err := Parse(content)
pe := asParseError(t, err)
if !strings.Contains(pe.Reason, "UTF-8") {
t.Errorf("Reason = %q, 應提示改存 UTF-8", pe.Reason)
}
}
func TestParse_TableRow_UTF8BOMStripped(t *testing.T) {
content := append([]byte{0xEF, 0xBB, 0xBF}, []byte("0 剪刀\n1 石頭\n")...)
got, err := Parse(content)
if err != nil {
t.Fatalf("Parse error: %vBOM 未被 strip", err)
}
if !equalStrings(got.Labels, []string{"剪刀", "石頭"}) {
t.Errorf("Labels = %v", got.Labels)
}
}
// 表列 13只有 index 沒有名稱 → 拒絕 + 行號
func TestParse_TableRow_IndexWithoutNameRejected(t *testing.T) {
cases := []struct {
name string
content string
wantLine int
}{
{"純數字行", "0 a\n1\n", 2},
{"數字後只有空白", "0 a\n1 \n", 2}, // TrimSpace 後變 "1"、與純數字行同路
{"第一行就缺", "0\n", 1},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
_, err := Parse([]byte(tc.content))
pe := asParseError(t, err)
if pe.Line != tc.wantLine {
t.Errorf("Line = %d, want %d (err=%v)", pe.Line, tc.wantLine, pe)
}
if !strings.Contains(pe.Reason, "名稱") {
t.Errorf("Reason = %q, 應說明缺少名稱", pe.Reason)
}
})
}
}
// ── plan §3.5 安全上限S-2 / R-7───────────────────────────────────
func TestParse_MaxIndexEnforced(t *testing.T) {
t.Run("剛好在上限內", func(t *testing.T) {
got, err := Parse([]byte(fmt.Sprintf("%d ok\n", MaxIndex)))
if err != nil {
t.Fatalf("index %d 應被接受,卻拒絕:%v", MaxIndex, err)
}
if len(got.Labels) != MaxIndex+1 {
t.Errorf("len(Labels) = %d, want %d", len(got.Labels), MaxIndex+1)
}
})
t.Run("超過上限一格就拒絕", func(t *testing.T) {
_, err := Parse([]byte(fmt.Sprintf("%d boom\n", MaxIndex+1)))
pe := asParseError(t, err)
if pe.Line != 1 {
t.Errorf("Line = %d, want 1", pe.Line)
}
if !strings.Contains(pe.Reason, "上限") {
t.Errorf("Reason = %q, 應說明超過上限", pe.Reason)
}
})
t.Run("巨大 index 不會嘗試配置記憶體", func(t *testing.T) {
// 若上限檢查失效,這行會嘗試 make([]string, 1e9),測試會 OOM 而非失敗。
_, err := Parse([]byte("999999999 boom\n"))
asParseError(t, err)
})
}
func TestParse_MaxFileSizeEnforced(t *testing.T) {
big := make([]byte, MaxFileSize+1)
for i := range big {
big[i] = 'a'
}
_, err := Parse(big)
pe := asParseError(t, err)
if !strings.Contains(pe.Reason, "過大") {
t.Errorf("Reason = %q, 應說明檔案過大", pe.Reason)
}
}
func TestParse_MaxLinesEnforced(t *testing.T) {
// index 唯一且 <= MaxIndex 的合法檔案最多 MaxIndex+1 行,所以要觸發行數
// 上限一定得帶重複 index —— 此測試同時釘住「行數檢查排在重複檢查之前」。
// 若哪天有人把行數檢查移到 parseLine / dup 檢查之後,這裡會看到「重複」
// 而非「行數」,測試失敗。
var sb strings.Builder
for i := 0; i <= MaxLines; i++ {
fmt.Fprintf(&sb, "%d l%d\n", i%(MaxIndex+1), i)
}
_, err := Parse([]byte(sb.String()))
pe := asParseError(t, err)
if !strings.Contains(pe.Reason, "行數") {
t.Errorf("Reason = %q, 應說明行數超過上限(行數檢查是否被移到重複檢查之後?)", pe.Reason)
}
}
// MaxLines 必須 <= MaxIndex+1否則行數檢查永遠碰不到重複 index 會先擋)。
// 這條把「無效防禦」的可能性從常數層面就釘死。
func TestParse_MaxLinesIsReachable(t *testing.T) {
if MaxLines > MaxIndex+1 {
t.Fatalf("MaxLines(%d) > MaxIndex+1(%d):行數檢查永遠不可能觸發",
MaxLines, MaxIndex+1)
}
}
func TestParse_ControlCharactersInNameRejected(t *testing.T) {
cases := map[string]string{
"NUL": "0 a\x00b\n",
"ESC": "0 a\x1bb\n",
"BiDi override": "0 ab\n",
"BiDi isolate": "0 ab\n",
}
for name, content := range cases {
t.Run(name, func(t *testing.T) {
_, err := Parse([]byte(content))
pe := asParseError(t, err)
if !strings.Contains(pe.Reason, "控制字元") {
t.Errorf("Reason = %q, 應說明控制字元", pe.Reason)
}
})
}
}
// ── 錯誤訊息品質 ─────────────────────────────────────────────────────
func TestParseError_MessageIncludesLineNumber(t *testing.T) {
pe := &ParseError{Line: 5, Reason: "index 必須為非負整數,收到 \"abc\""}
if !strings.Contains(pe.Error(), "第 5 行") {
t.Errorf("Error() = %q, 應含行號", pe.Error())
}
}
func TestParseError_NoLineOmitsPrefix(t *testing.T) {
pe := &ParseError{Reason: "標籤檔沒有任何有效內容"}
if strings.Contains(pe.Error(), "行") {
t.Errorf("Error() = %q, 無行號時不應帶行號前綴", pe.Error())
}
}
// ── helper ───────────────────────────────────────────────────────────
func equalStrings(a, b []string) bool {
if len(a) != len(b) {
return false
}
for i := range a {
if a[i] != b[i] {
return false
}
}
return true
}

View File

@ -4,6 +4,7 @@ import (
"encoding/json" "encoding/json"
"fmt" "fmt"
"os" "os"
"path/filepath"
"strings" "strings"
"sync" "sync"
) )
@ -13,6 +14,15 @@ type Repository struct {
mu sync.RWMutex mu sync.RWMutex
} }
// NewRepository 載入 models.json 的內建模型目錄。
//
// models.json 會列出所有「產品支援」的模型定義,但安裝包不一定會帶上每個
// 對應的 .nef見 Makefile 的 BUNDLED_NEFS 白名單)。因此載入後會過濾掉
// .nef 檔案實際不存在的 model —— 否則使用者會在 UI 看到選不了的模型,選下去
// 才在 flash 階段拿到 "model file not found" 這種沒頭沒尾的錯誤。
//
// 過濾只作用在 models.json 的內建模型。使用者上傳的自訂模型走 Add()
// 路徑是絕對路徑且必定存在,不受影響。
func NewRepository(dataPath string) *Repository { func NewRepository(dataPath string) *Repository {
r := &Repository{} r := &Repository{}
data, err := os.ReadFile(dataPath) data, err := os.ReadFile(dataPath)
@ -20,11 +30,56 @@ func NewRepository(dataPath string) *Repository {
fmt.Printf("Warning: could not load models from %s: %v\n", dataPath, err) fmt.Printf("Warning: could not load models from %s: %v\n", dataPath, err)
return r return r
} }
if err := json.Unmarshal(data, &r.models); err != nil { var declared []Model
if err := json.Unmarshal(data, &declared); err != nil {
fmt.Printf("Warning: could not parse models JSON: %v\n", err) fmt.Printf("Warning: could not parse models JSON: %v\n", err)
}
return r return r
} }
r.models = filterAvailableModels(declared, filepath.Dir(dataPath))
return r
}
// filterAvailableModels 只保留 .nef 檔案實際存在的 model。
//
// dataDir 是 models.json 所在的目錄(即 bundle 內的 data/models.json 的
// filePath 以它為基準解析。
func filterAvailableModels(models []Model, dataDir string) []Model {
available := make([]Model, 0, len(models))
for _, m := range models {
path := resolveBuiltInModelPath(m.FilePath, dataDir)
// 沒宣告 filePath 的 model 不做檔案檢查(沒有東西可以檢查),保留原行為。
if path == "" {
available = append(available, m)
continue
}
if info, err := os.Stat(path); err != nil || info.IsDir() {
fmt.Printf("[INFO] Skipping model %q (%s): .nef not bundled at %s\n", m.ID, m.Name, path)
continue
}
available = append(available, m)
}
return available
}
// resolveBuiltInModelPath 把 models.json 的 filePath 解析成實際的檔案路徑。
//
// 規則與 flash.Service.StartFlash 一致:
// - 絕對路徑 → 原樣使用
// - "data/nef/..." → 去掉 "data/" 前綴後接在 dataDir 之下
// (因為 dataDir 本身就是那個 data/ 目錄,不去掉會變成 data/data/nef/...
// - 其他相對路徑 → 直接接在 dataDir 之下
func resolveBuiltInModelPath(filePath, dataDir string) string {
if filePath == "" {
return ""
}
if filepath.IsAbs(filePath) {
return filePath
}
if strings.HasPrefix(filePath, "data/") || strings.HasPrefix(filePath, "data\\") {
return filepath.Join(dataDir, filePath[len("data/"):])
}
return filepath.Join(dataDir, filePath)
}
func (r *Repository) List(filter ModelFilter) ([]ModelSummary, int) { func (r *Repository) List(filter ModelFilter) ([]ModelSummary, int) {
r.mu.RLock() r.mu.RLock()

View File

@ -1,6 +1,9 @@
package model package model
import ( import (
"encoding/json"
"os"
"path/filepath"
"testing" "testing"
) )
@ -98,6 +101,193 @@ func TestRepository_Add(t *testing.T) {
} }
} }
// writeModelsJSON 在 dir 底下建立 models.json回傳它的路徑。
func writeModelsJSON(t *testing.T, dir string, models []Model) string {
t.Helper()
data, err := json.MarshalIndent(models, "", " ")
if err != nil {
t.Fatalf("marshal models: %v", err)
}
path := filepath.Join(dir, "models.json")
if err := os.WriteFile(path, data, 0o644); err != nil {
t.Fatalf("write models.json: %v", err)
}
return path
}
// touchNef 在 dir 底下建立一個假的 .nef內容不重要只檢查存在性
func touchNef(t *testing.T, dir, relPath string) {
t.Helper()
full := filepath.Join(dir, relPath)
if err := os.MkdirAll(filepath.Dir(full), 0o755); err != nil {
t.Fatalf("mkdir for %s: %v", relPath, err)
}
if err := os.WriteFile(full, []byte("fake nef"), 0o644); err != nil {
t.Fatalf("write %s: %v", relPath, err)
}
}
func TestResolveBuiltInModelPath(t *testing.T) {
dataDir := filepath.Join("/bundle", "data")
tests := []struct {
name string
filePath string
want string
}{
{
name: "strips data/ prefix so it does not become data/data/",
filePath: "data/nef/kl520/a.nef",
want: filepath.Join("/bundle", "data", "nef", "kl520", "a.nef"),
},
{
name: "relative path without data/ prefix joins directly",
filePath: "nef/kl520/a.nef",
want: filepath.Join("/bundle", "data", "nef", "kl520", "a.nef"),
},
{
name: "absolute path is used as-is",
filePath: filepath.Join("/custom", "models", "x", "model.nef"),
want: filepath.Join("/custom", "models", "x", "model.nef"),
},
{
name: "empty file path stays empty",
filePath: "",
want: "",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
if got := resolveBuiltInModelPath(tt.filePath, dataDir); got != tt.want {
t.Errorf("resolveBuiltInModelPath(%q) = %q, want %q", tt.filePath, got, tt.want)
}
})
}
}
func TestFilterAvailableModels(t *testing.T) {
dataDir := t.TempDir()
touchNef(t, dataDir, "nef/kl520/bundled.nef")
absent := filepath.Join(dataDir, "nef", "kl520", "abs-missing.nef")
present := filepath.Join(dataDir, "nef", "kl520", "abs.nef")
touchNef(t, dataDir, "nef/kl520/abs.nef")
// 目錄而非檔案:不該被當成可用的 model
if err := os.MkdirAll(filepath.Join(dataDir, "nef/kl520/dir.nef"), 0o755); err != nil {
t.Fatalf("mkdir dir.nef: %v", err)
}
models := []Model{
{ID: "bundled", FilePath: "data/nef/kl520/bundled.nef"},
{ID: "not-bundled", FilePath: "data/nef/kl520/nope.nef"},
{ID: "abs-present", FilePath: present},
{ID: "abs-absent", FilePath: absent},
{ID: "no-file-path"},
{ID: "dir-not-file", FilePath: "data/nef/kl520/dir.nef"},
}
got := filterAvailableModels(models, dataDir)
var gotIDs []string
for _, m := range got {
gotIDs = append(gotIDs, m.ID)
}
want := []string{"bundled", "abs-present", "no-file-path"}
if len(gotIDs) != len(want) {
t.Fatalf("filterAvailableModels() = %v, want %v", gotIDs, want)
}
for i := range want {
if gotIDs[i] != want[i] {
t.Errorf("filterAvailableModels()[%d] = %q, want %q", i, gotIDs[i], want[i])
}
}
}
func TestNewRepository_FiltersUnbundledModels(t *testing.T) {
dataDir := t.TempDir()
touchNef(t, dataDir, "nef/kl520/kl520_20004_fcos-drk53s_w512h512.nef")
touchNef(t, dataDir, "nef/kl520/kl520_tiny_yolo_v3.nef")
// 模擬正式情境models.json 宣告 4 個 model但只打包了其中 2 個 .nef
path := writeModelsJSON(t, dataDir, []Model{
{ID: "kl520-fcos-detection", Name: "物件辨識", TaskType: "object_detection",
FilePath: "data/nef/kl520/kl520_20004_fcos-drk53s_w512h512.nef"},
{ID: "kl520-tiny-yolov3", Name: "人型監測", TaskType: "object_detection",
FilePath: "data/nef/kl520/kl520_tiny_yolo_v3.nef"},
{ID: "kl520-yolov5-detection", Name: "YOLOv5", TaskType: "object_detection",
FilePath: "data/nef/kl520/kl520_20005_yolov5-noupsample_w640h640.nef"},
{ID: "kl720-resnet18-classification", Name: "ResNet18", TaskType: "classification",
FilePath: "data/nef/kl720/kl720_20001_resnet18_w224h224.nef"},
})
repo := NewRepository(path)
if repo.Count() != 2 {
t.Fatalf("Count() = %d, want 2 (only bundled .nef should load)", repo.Count())
}
for _, id := range []string{"kl520-fcos-detection", "kl520-tiny-yolov3"} {
if _, err := repo.GetByID(id); err != nil {
t.Errorf("GetByID(%q) failed, expected it to be available: %v", id, err)
}
}
for _, id := range []string{"kl520-yolov5-detection", "kl720-resnet18-classification"} {
if _, err := repo.GetByID(id); err == nil {
t.Errorf("GetByID(%q) succeeded, expected it to be filtered out", id)
}
}
// 過濾後的清單也不該出現在 List()
results, count := repo.List(ModelFilter{})
if count != 2 || len(results) != 2 {
t.Errorf("List() = %d results (count %d), want 2", len(results), count)
}
}
func TestNewRepository_CustomModelsUnaffectedByFilter(t *testing.T) {
dataDir := t.TempDir()
path := writeModelsJSON(t, dataDir, []Model{
{ID: "built-in-missing", FilePath: "data/nef/kl520/missing.nef"},
})
repo := NewRepository(path)
if repo.Count() != 0 {
t.Fatalf("Count() = %d, want 0 after filtering", repo.Count())
}
// 自訂模型走 Add(),不經過過濾
repo.Add(Model{ID: "custom-1", IsCustom: true, FilePath: "/anywhere/model.nef"})
if repo.Count() != 1 {
t.Errorf("Count() = %d after Add(), want 1", repo.Count())
}
if _, err := repo.GetByID("custom-1"); err != nil {
t.Errorf("GetByID(custom-1) failed: %v", err)
}
}
func TestNewRepository_MissingOrInvalidFile(t *testing.T) {
t.Run("missing models.json yields empty repo", func(t *testing.T) {
repo := NewRepository(filepath.Join(t.TempDir(), "nope.json"))
if repo.Count() != 0 {
t.Errorf("Count() = %d, want 0", repo.Count())
}
})
t.Run("invalid JSON yields empty repo", func(t *testing.T) {
dir := t.TempDir()
path := filepath.Join(dir, "models.json")
if err := os.WriteFile(path, []byte("{not json"), 0o644); err != nil {
t.Fatalf("write: %v", err)
}
repo := NewRepository(path)
if repo.Count() != 0 {
t.Errorf("Count() = %d, want 0", repo.Count())
}
})
}
func TestRepository_Remove(t *testing.T) { func TestRepository_Remove(t *testing.T) {
repo := newTestRepo() repo := newTestRepo()

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff