jim800121chen 1231bf0ed2 feat(visionA-backend): Phase 0.8 conversion package — 5 endpoint + 8 個內部模組
Phase 0.8 把 kneron_model_converter 的轉檔功能整合進 visionA Cloud。
visionA backend 當 streaming proxy(upload)+ delegated download token broker(download)+
ownership trust boundary,converter / FAA / MC 三方零修改。

新增 internal/conversion/ 套件(8 個檔,~10,000 行 prod+test,117+ test cases,race -count=3 全綠):

- conversion.go:Service interface 5 method、Job/PromoteResult/InitJobInput types
- errors.go:13+ sentinel errors + ErrorCode/HTTPStatus mapping,對齊 conversion.md §6
- mc_token_client.go:service-to-service token (client_credentials grant) + DCL cache
  (exp - 15s 重取,per-scope cache),IssueDelegatedDownload(MC delegated download token)
  錯誤分 idp_misconfigured (4xx) / idp_unavailable (5xx) / download_token_failed / mc_token_unavailable
- converter_client.go:對 converter scheduler 4 method(InitJob multipart streaming /
  GetJob / Promote / ListInProgressJobs),InitJob 不 retry 5xx(streaming body 無法 replay)
- faa_client.go:對 FAA GET /files/{key} server-to-server pull,Phase A retry(GET 無 body
  可 replay)對齊 §9.1 retry 矩陣,streaming io.ReadCloser 透傳避 OOM
- ownership.go:in-memory job_id → user_id map + per-user mutex 防 thundering herd lazy rebuild
  (不同 user 平行 fetch,同 user 100 caller 收斂成 1 次),visionA 重啟靠 converter
  ListInProgressJobs(user) 重建
- flow.go:Service interface 整合層(5 method 串接 converter/FAA/MC/ownership)
  - InitJob 用 io.Pipe + multipart.Reader/Writer 重組 streaming proxy(黑名單 client user_id
    + 灌入 OIDC sub)
  - DownloadRedirectURL 自動觸發 promote(spec §1 Stage 3b),用 ensurePromoted helper
  - PromoteToModels 冪等(modelStore.FindBySourceJobID 為 source-of-truth)
  - OwnershipMismatch → ErrJobNotFound 不 forbidden(§7.2 防枚舉)
  - storage / modelStore 失敗包 ErrStorageUnavailable / ErrModelStoreUnavailable
    (視為 visionA 自身 500 而非 502 gateway,SRE alarm 才打對 team)

新增 internal/api/conversion.go(5 endpoint handler + main.go wire):
- POST /api/conversion/init(multipart streaming proxy,不呼叫 c.MultipartForm())
- GET  /api/conversion/active(lazy rebuild ownership)
- GET  /api/conversion/{job_id}(poll status)
- POST /api/conversion/{job_id}/promote-to-models(FAA pull → models 三段式)
- GET  /api/conversion/{job_id}/download(server-side HTTP 302 → FAA,token 不過 frontend
  JS,仿 FAA TestSite DownloadFileDirect pattern;Cache-Control: no-store)

5 個 endpoint 全部走 OIDC AuthMiddleware;user_id 從 cookie session 灌(trust boundary),
從不接受 client multipart form / JSON / query 的 user_id。
TestAllAPIEndpointsRequire401WithoutCookie 自動覆蓋新 5 endpoint regression 防呆。

新增 cmd/api-server/conversion_e2e_test.go(4 個 e2e 場景):
- TestConversionE2E_StreamingProxy(10MB body + trust boundary regression)
- TestConversionE2E_LazyRebuildAfterRestart(visionA 重啟仍能 /active)
- TestConversionE2E_Download302Redirect(驗 302 + Location header + token 不在 body)
- TestConversionE2E_ActiveJobConflict(409 + active_job 詳情)

修改 internal/config/{config,load}.go:新增 ConversionConfig 5 欄位
(ConverterBaseURL / FAABaseURL / TenantID / ServiceClientID / ServiceClientSecret)+
Enabled() helper(雙非空判定)。
修改 cmd/api-server/main.go:條件 wire(cfg.Conversion.Enabled() 為 true 才建 client + Service;
否則 Deps.Conversion=nil,handler 自動回 501)。
修改 .env.example:新增 Phase 0.8 區塊註解。
新增 cmd/api-server/conversion_adapters.go:narrow interface adapter(接既有
internal/model.Repository / internal/storage.Store → conversion.ModelStore / Storage,避免 import cycle)。

驗證:go test -race -count=3 ./... 17 packages 全綠 / go vet 0 warning / go build 成功。

對齊文件:
- .autoflow/04-architecture/adr/adr-014-conversion-integration.md
- .autoflow/04-architecture/conversion.md (TDD)
- .autoflow/04-architecture/api/api-conversion.md
- .autoflow/02-prd/features/feature-converter-integration.md
- .autoflow/03-design/wireframes/wireframe-conversion.md
- .autoflow/03-design/flows/flow-conversion.md

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-04 13:56:07 +08:00

152 lines
4.9 KiB
Go
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

package conversion
import (
"context"
"encoding/json"
"io"
"strings"
"testing"
"time"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
// noopService 是一個 compile-time 驗證 — 用來確認 Service interface 的方法集合穩定。
// 真實實作Flow會在 T6 補。這裡只測 interface 簽名沒有打錯(避免 T6 才發現要改 interface
type noopService struct{}
func (noopService) InitJob(ctx context.Context, in InitJobInput) (*Job, error) {
return nil, nil
}
func (noopService) GetJob(ctx context.Context, userID, jobID string) (*Job, error) {
return nil, nil
}
func (noopService) PromoteToModels(ctx context.Context, userID, jobID, name string) (*PromoteResult, error) {
return nil, nil
}
func (noopService) DownloadRedirectURL(ctx context.Context, userID, jobID string) (string, error) {
return "", nil
}
func (noopService) ActiveJob(ctx context.Context, userID string) (*Job, error) {
return nil, nil
}
// File-scope compile-time check — 若 Service interface 改變,
// noopService 就不再實作此 interface編譯失敗。
// 移到 file scopeT1 review M1t.Run 內的 var declaration 只在執行該 test 時驗,
// 而我們希望「package 編譯成功」就保證 interface 穩定。
var _ Service = noopService{}
// TestService_InterfaceSatisfied 在 test 中再 assert 一次,作為文件性說明。
func TestService_InterfaceSatisfied(t *testing.T) {
t.Parallel()
var _ Service = noopService{}
}
// TestJob_JSONShape 驗證 Job struct 的 JSON tag 與 api-conversion.md §1-2 response 對齊。
//
// 這是契約測試frontend 依 api-conversion.md 寫 typebackend 改 json tag 一定要回頭看這個 test。
func TestJob_JSONShape(t *testing.T) {
t.Parallel()
createdAt, _ := time.Parse(time.RFC3339, "2026-04-30T12:00:00Z")
expiresAt := createdAt.Add(7 * 24 * time.Hour)
job := Job{
JobID: "550e8400-e29b-41d4-a716-446655440000",
Status: "running",
Stage: "bie",
Progress: 45,
StageProgress: 60,
CreatedAt: createdAt,
UpdatedAt: createdAt.Add(5 * time.Minute),
ExpiresAt: expiresAt,
SourceFilename: "yolov5s.onnx",
TargetChip: "720",
}
raw, err := json.Marshal(job)
require.NoError(t, err)
// 必要欄位都在
assert.Contains(t, string(raw), `"job_id":"550e8400-e29b-41d4-a716-446655440000"`)
assert.Contains(t, string(raw), `"status":"running"`)
assert.Contains(t, string(raw), `"stage":"bie"`)
assert.Contains(t, string(raw), `"progress":45`)
assert.Contains(t, string(raw), `"stage_progress":60`)
assert.Contains(t, string(raw), `"created_at":"2026-04-30T12:00:00Z"`)
assert.Contains(t, string(raw), `"expires_at":"2026-05-07T12:00:00Z"`)
assert.Contains(t, string(raw), `"source_filename":"yolov5s.onnx"`)
assert.Contains(t, string(raw), `"target_chip":"720"`)
// error 欄位 zero value 時應被 omitempty 隱藏
assert.NotContains(t, string(raw), `"error_code"`)
assert.NotContains(t, string(raw), `"error_message"`)
}
// TestJob_FailedShape 驗證 failed job 的 error 欄位序列化。
func TestJob_FailedShape(t *testing.T) {
t.Parallel()
job := Job{
JobID: "job-failed",
Status: "failed",
ErrorCode: "QUANTIZATION_FAILED",
ErrorMessage: "model has unsupported operator",
}
raw, err := json.Marshal(job)
require.NoError(t, err)
assert.Contains(t, string(raw), `"error_code":"QUANTIZATION_FAILED"`)
assert.Contains(t, string(raw), `"error_message":"model has unsupported operator"`)
}
// TestPromoteResult_JSONShape 對齊 api-conversion.md §3 response。
func TestPromoteResult_JSONShape(t *testing.T) {
t.Parallel()
createdAt, _ := time.Parse(time.RFC3339, "2026-04-30T12:30:00Z")
pr := PromoteResult{
ModelID: "abc-123",
Source: "converted",
SourceJobID: "550e8400-...",
Name: "YOLOv5 Face KL520",
TargetChip: "kl520",
FileSize: 12345678,
Status: "ready",
CreatedAt: createdAt,
}
raw, err := json.Marshal(pr)
require.NoError(t, err)
assert.Contains(t, string(raw), `"model_id":"abc-123"`)
assert.Contains(t, string(raw), `"source":"converted"`)
assert.Contains(t, string(raw), `"source_job_id":"550e8400-..."`)
assert.Contains(t, string(raw), `"file_size":12345678`)
assert.Contains(t, string(raw), `"status":"ready"`)
assert.Contains(t, string(raw), `"target_chip":"kl520"`)
}
// TestInitJobInput_AcceptsReader 驗證 InitJobInput.Body 接受 io.Reader即 streaming 不收 buffer
//
// 關鍵:若有人不小心把欄位改成 []byte這個測試編譯會壞。
func TestInitJobInput_AcceptsReader(t *testing.T) {
t.Parallel()
in := InitJobInput{
UserID: "user-abc",
ContentType: "multipart/form-data; boundary=xyz",
Body: strings.NewReader("--xyz--"),
ContentLength: 7,
}
// 確認 Body 是 io.Readercompile time 透過 type assertion
var _ io.Reader = in.Body
assert.Equal(t, "user-abc", in.UserID)
}