jim800121chen 1231bf0ed2 feat(visionA-backend): Phase 0.8 conversion package — 5 endpoint + 8 個內部模組
Phase 0.8 把 kneron_model_converter 的轉檔功能整合進 visionA Cloud。
visionA backend 當 streaming proxy(upload)+ delegated download token broker(download)+
ownership trust boundary,converter / FAA / MC 三方零修改。

新增 internal/conversion/ 套件(8 個檔,~10,000 行 prod+test,117+ test cases,race -count=3 全綠):

- conversion.go:Service interface 5 method、Job/PromoteResult/InitJobInput types
- errors.go:13+ sentinel errors + ErrorCode/HTTPStatus mapping,對齊 conversion.md §6
- mc_token_client.go:service-to-service token (client_credentials grant) + DCL cache
  (exp - 15s 重取,per-scope cache),IssueDelegatedDownload(MC delegated download token)
  錯誤分 idp_misconfigured (4xx) / idp_unavailable (5xx) / download_token_failed / mc_token_unavailable
- converter_client.go:對 converter scheduler 4 method(InitJob multipart streaming /
  GetJob / Promote / ListInProgressJobs),InitJob 不 retry 5xx(streaming body 無法 replay)
- faa_client.go:對 FAA GET /files/{key} server-to-server pull,Phase A retry(GET 無 body
  可 replay)對齊 §9.1 retry 矩陣,streaming io.ReadCloser 透傳避 OOM
- ownership.go:in-memory job_id → user_id map + per-user mutex 防 thundering herd lazy rebuild
  (不同 user 平行 fetch,同 user 100 caller 收斂成 1 次),visionA 重啟靠 converter
  ListInProgressJobs(user) 重建
- flow.go:Service interface 整合層(5 method 串接 converter/FAA/MC/ownership)
  - InitJob 用 io.Pipe + multipart.Reader/Writer 重組 streaming proxy(黑名單 client user_id
    + 灌入 OIDC sub)
  - DownloadRedirectURL 自動觸發 promote(spec §1 Stage 3b),用 ensurePromoted helper
  - PromoteToModels 冪等(modelStore.FindBySourceJobID 為 source-of-truth)
  - OwnershipMismatch → ErrJobNotFound 不 forbidden(§7.2 防枚舉)
  - storage / modelStore 失敗包 ErrStorageUnavailable / ErrModelStoreUnavailable
    (視為 visionA 自身 500 而非 502 gateway,SRE alarm 才打對 team)

新增 internal/api/conversion.go(5 endpoint handler + main.go wire):
- POST /api/conversion/init(multipart streaming proxy,不呼叫 c.MultipartForm())
- GET  /api/conversion/active(lazy rebuild ownership)
- GET  /api/conversion/{job_id}(poll status)
- POST /api/conversion/{job_id}/promote-to-models(FAA pull → models 三段式)
- GET  /api/conversion/{job_id}/download(server-side HTTP 302 → FAA,token 不過 frontend
  JS,仿 FAA TestSite DownloadFileDirect pattern;Cache-Control: no-store)

5 個 endpoint 全部走 OIDC AuthMiddleware;user_id 從 cookie session 灌(trust boundary),
從不接受 client multipart form / JSON / query 的 user_id。
TestAllAPIEndpointsRequire401WithoutCookie 自動覆蓋新 5 endpoint regression 防呆。

新增 cmd/api-server/conversion_e2e_test.go(4 個 e2e 場景):
- TestConversionE2E_StreamingProxy(10MB body + trust boundary regression)
- TestConversionE2E_LazyRebuildAfterRestart(visionA 重啟仍能 /active)
- TestConversionE2E_Download302Redirect(驗 302 + Location header + token 不在 body)
- TestConversionE2E_ActiveJobConflict(409 + active_job 詳情)

修改 internal/config/{config,load}.go:新增 ConversionConfig 5 欄位
(ConverterBaseURL / FAABaseURL / TenantID / ServiceClientID / ServiceClientSecret)+
Enabled() helper(雙非空判定)。
修改 cmd/api-server/main.go:條件 wire(cfg.Conversion.Enabled() 為 true 才建 client + Service;
否則 Deps.Conversion=nil,handler 自動回 501)。
修改 .env.example:新增 Phase 0.8 區塊註解。
新增 cmd/api-server/conversion_adapters.go:narrow interface adapter(接既有
internal/model.Repository / internal/storage.Store → conversion.ModelStore / Storage,避免 import cycle)。

驗證:go test -race -count=3 ./... 17 packages 全綠 / go vet 0 warning / go build 成功。

對齊文件:
- .autoflow/04-architecture/adr/adr-014-conversion-integration.md
- .autoflow/04-architecture/conversion.md (TDD)
- .autoflow/04-architecture/api/api-conversion.md
- .autoflow/02-prd/features/feature-converter-integration.md
- .autoflow/03-design/wireframes/wireframe-conversion.md
- .autoflow/03-design/flows/flow-conversion.md

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-04 13:56:07 +08:00

470 lines
17 KiB
Go
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

// conversion.go — /api/conversion/* 的 handler 實作Phase 0.8)。
//
// 對齊:
// - .autoflow/04-architecture/api/api-conversion.md5 個 endpoint API spec
// - .autoflow/04-architecture/conversion.md §3 endpoint 表 + §6 錯誤碼 + §10 安全考量
// - internal/conversion/conversion.goService interface
//
// 5 個 endpoint
//
// POST /api/conversion/init — 啟動轉檔multipart streaming
// GET /api/conversion/active — 查當前 active job
// GET /api/conversion/{job_id} — poll 狀態
// POST /api/conversion/{job_id}/promote-to-models — 加到模型庫
// GET /api/conversion/{job_id}/download — server-side 302 redirect → FAA
//
// 安全要點(對齊 conversion.md §7 / §10
// - 全部 5 個 endpoint 都註冊在 apiGroupOIDC AuthMiddleware 之後)
// - userID 一律來自 UserContextFrom(c).UserID從 cookie session 解出 OIDC sub
// - 任何 client 帶來的 user_idmultipart form / JSON / query一律忽略
// - /init 不呼叫 c.MultipartForm() — 會 buffer 全 body 進 RAM / disk破壞 streaming
// - /download 採 HTTP 302 Foundtoken 不出現在任何 JSON response§10.4
//
// Phase 0.8 conversion (見 .autoflow/04-architecture/api/api-conversion.md)
package api
import (
"context"
"encoding/json"
"errors"
"net/http"
"strings"
"github.com/gin-gonic/gin"
"visiona-backend/internal/conversion"
)
// ==========================================================================
// Route 註冊
// ==========================================================================
// registerConversionRoutes 註冊 /api/conversion/* 的 routes。
//
// 由 NewRouter 在 apiGroupOIDC AuthMiddleware 已套)下呼叫;
// 若 deps.Conversion 為 nilPhase 0.8 conversion 未啟用,例如 dev 環境沒設
// CONVERTER_BASE_URL / FAA_BASE_URL→ 5 個 endpoint 一律回 501。
func registerConversionRoutes(g *gin.RouterGroup, deps Deps) {
if deps.Conversion == nil {
// 未啟用 — 註冊 501 stub避免 404讓 frontend 拿到明確 NOT_IMPLEMENTED
notImpl := func(c *gin.Context) {
WriteNotImplemented(c, "conversion service is not configured (set VISIONA_CONVERTER_BASE_URL + VISIONA_FAA_BASE_URL)")
}
conv := g.Group("/conversion")
conv.POST("/init", notImpl)
conv.GET("/active", notImpl)
conv.GET("/:job_id", notImpl)
conv.POST("/:job_id/promote-to-models", notImpl)
conv.GET("/:job_id/download", notImpl)
return
}
conv := g.Group("/conversion")
conv.POST("/init", conversionInitHandler(deps))
conv.GET("/active", conversionActiveHandler(deps))
conv.GET("/:job_id", conversionGetHandler(deps))
conv.POST("/:job_id/promote-to-models", conversionPromoteHandler(deps))
conv.GET("/:job_id/download", conversionDownloadHandler(deps))
}
// ==========================================================================
// 1. POST /api/conversion/init
// ==========================================================================
// conversionInitHandler 處理「啟動轉檔」請求。
//
// 流程:
// 1. UserContextFrom 拿 OIDC subAuthMiddleware 已驗)
// 2. 驗 Content-Type 必須是 multipart/form-data含 boundary
// 3. 直接把 c.Request.Body + Content-Type 傳給 Service.InitJob
// **不**呼叫 c.MultipartForm() — 會破壞 streaming
// 4. 成功 → 201 + Job
// 5. 失敗 → 透過 handleConversionError 對應 sentinel mapping
//
// 對齊 api-conversion.md §1 + conversion.md §4.2 streaming proxy。
func conversionInitHandler(deps Deps) gin.HandlerFunc {
return func(c *gin.Context) {
uc, ok := UserContextFrom(c)
if !ok || uc.UserID == "" {
// AuthMiddleware 已通過卻拿不到 UserContext — 設定錯誤
WriteError(c, http.StatusInternalServerError, ErrCodeInternalError,
"missing user context (auth middleware misconfigured?)", nil)
return
}
ct := c.GetHeader("Content-Type")
if !strings.HasPrefix(strings.ToLower(ct), "multipart/form-data") {
WriteError(c, http.StatusBadRequest, ErrCodeValidationFailed,
"Content-Type must be multipart/form-data with boundary", nil)
return
}
// 把 raw body + Content-Type 傳給 ServiceService 內部處理 multipart streaming
// 重組(注入 user_id、黑名單 client 帶的 user_id。見 conversion.md §4.2。
in := conversion.InitJobInput{
UserID: uc.UserID,
ContentType: ct,
Body: c.Request.Body,
ContentLength: c.Request.ContentLength,
}
job, err := deps.Conversion.InitJob(c.Request.Context(), in)
if err != nil {
handleConversionError(c, err)
return
}
// 成功 — 201 Created對齊 RESTful 慣例POST 建立資源用 201
WriteSuccess(c, http.StatusCreated, jobToResponse(job))
}
}
// ==========================================================================
// 2. GET /api/conversion/active
// ==========================================================================
// conversionActiveHandler 處理「查當前 active job」請求。
//
// 對齊 api-conversion.md §5
// - 有 active → 200 + {has_active: true, job: {...}}
// - 無 active → 200 + {has_active: false, job: null}
//
// 重啟恢復場景由 Service 內部 EnsureRebuilt 處理lazy rebuild from converter
// handler 對 frontend 完全透明。
func conversionActiveHandler(deps Deps) gin.HandlerFunc {
return func(c *gin.Context) {
uc, ok := UserContextFrom(c)
if !ok || uc.UserID == "" {
WriteError(c, http.StatusInternalServerError, ErrCodeInternalError,
"missing user context (auth middleware misconfigured?)", nil)
return
}
job, err := deps.Conversion.ActiveJob(c.Request.Context(), uc.UserID)
if err != nil {
handleConversionError(c, err)
return
}
if job == nil {
WriteSuccess(c, http.StatusOK, gin.H{
"has_active": false,
"job": nil,
})
return
}
WriteSuccess(c, http.StatusOK, gin.H{
"has_active": true,
"job": jobToResponse(job),
})
}
}
// ==========================================================================
// 3. GET /api/conversion/{job_id}
// ==========================================================================
// conversionGetHandler 處理「poll job 狀態」請求。
//
// 對齊 api-conversion.md §2。
// 設計選擇ownership 不符 / job 不存在都對應到 ErrJobNotFound404
// 由 Service 層做安全 mapping見 flow.go GetJob 註解避免「forbidden vs not_found」
// 差異枚舉合法 job_id
func conversionGetHandler(deps Deps) gin.HandlerFunc {
return func(c *gin.Context) {
uc, ok := UserContextFrom(c)
if !ok || uc.UserID == "" {
WriteError(c, http.StatusInternalServerError, ErrCodeInternalError,
"missing user context (auth middleware misconfigured?)", nil)
return
}
jobID := c.Param("job_id")
if jobID == "" {
WriteError(c, http.StatusBadRequest, ErrCodeValidationFailed,
"job_id is required", nil)
return
}
job, err := deps.Conversion.GetJob(c.Request.Context(), uc.UserID, jobID)
if err != nil {
handleConversionError(c, err)
return
}
WriteSuccess(c, http.StatusOK, jobToResponse(job))
}
}
// ==========================================================================
// 4. POST /api/conversion/{job_id}/promote-to-models
// ==========================================================================
// promoteRequest 是 promote-to-models 的 request body對齊 api-conversion.md §3
//
// `name` 是 Phase 0.8 wireframe §7.1 的單一欄位可為空Service 用
// `{source_filename_stem}_{target_chip_lower}` fallback
// `description` 雖在 schema 內但 Phase 0.8 不顯示給使用者backend 接受但忽略。
type promoteRequest struct {
Name string `json:"name"`
Description string `json:"description,omitempty"` // Phase 0.8 ignored, Phase 1 reserved
}
// conversionPromoteHandler 處理「加到模型庫」請求。
//
// 流程:
// 1. 驗 user / job_id
// 2. 解析 bodyname 可空body 整個可空)
// 3. Service.PromoteToModelspromote → FAA pull → models repo finalize
// 4. 成功 → 201 + PromoteResult
// 5. 冪等:同 jobID 重複 promote 由 Service 層處理(回既有 model record也是 201
func conversionPromoteHandler(deps Deps) gin.HandlerFunc {
return func(c *gin.Context) {
uc, ok := UserContextFrom(c)
if !ok || uc.UserID == "" {
WriteError(c, http.StatusInternalServerError, ErrCodeInternalError,
"missing user context (auth middleware misconfigured?)", nil)
return
}
jobID := c.Param("job_id")
if jobID == "" {
WriteError(c, http.StatusBadRequest, ErrCodeValidationFailed,
"job_id is required", nil)
return
}
// body optional — 沒帶或解析失敗都不擋name 可由 Service fallback
var body promoteRequest
if c.Request.Body != nil && c.Request.ContentLength != 0 {
// 寬鬆解析JSON 解失敗只 log不算 hard error因為 name 可選)
if err := json.NewDecoder(c.Request.Body).Decode(&body); err != nil {
// 嚴格一點JSON 格式錯誤回 400避免 silent ignore 讓使用者困惑)
WriteError(c, http.StatusBadRequest, ErrCodeValidationFailed,
"invalid JSON body: "+err.Error(), nil)
return
}
}
result, err := deps.Conversion.PromoteToModels(c.Request.Context(), uc.UserID, jobID, body.Name)
if err != nil {
handleConversionError(c, err)
return
}
WriteSuccess(c, http.StatusCreated, result)
}
}
// ==========================================================================
// 5. GET /api/conversion/{job_id}/download
// ==========================================================================
// conversionDownloadHandler 處理「下載」請求 — server-side HTTP 302 redirect。
//
// 對齊 api-conversion.md §4 + conversion.md §3.1 / §10.4
// - 成功302 Found + Location: <FAA URL with access_token>
// - 失敗:不 redirect依 Accept header 回 JSON / HTML 錯誤
// - Cache-Control: no-store — token 不該被 browser cache即使是 302 Location
//
// 仿 FAA TestSite `DownloadFileDirect` patterntoken 永遠不過 frontend JS。
func conversionDownloadHandler(deps Deps) gin.HandlerFunc {
return func(c *gin.Context) {
uc, ok := UserContextFrom(c)
if !ok || uc.UserID == "" {
WriteError(c, http.StatusInternalServerError, ErrCodeInternalError,
"missing user context (auth middleware misconfigured?)", nil)
return
}
jobID := c.Param("job_id")
if jobID == "" {
WriteError(c, http.StatusBadRequest, ErrCodeValidationFailed,
"job_id is required", nil)
return
}
downloadURL, err := deps.Conversion.DownloadRedirectURL(c.Request.Context(), uc.UserID, jobID)
if err != nil {
// 錯誤情況不 redirect — 依 Accept header 回 JSON / HTMLWriteError 寫 JSON
// 已能滿足主要 caseanchor tag 觸發時 browser 會直接顯示 JSON 也 OK
// Phase 0.8 不額外做 HTML 錯誤頁)
handleConversionError(c, err)
return
}
// 防快取:避免 browser 把 302 + Location 寫入 history / disk cache§10.4
c.Header("Cache-Control", "no-store, no-cache, must-revalidate, max-age=0")
c.Header("Pragma", "no-cache")
// 302 Found不用 301 — 301 可能被某些 browser 永久 cache
c.Redirect(http.StatusFound, downloadURL)
}
}
// ==========================================================================
// 錯誤處理 helper
// ==========================================================================
// handleConversionError 把 conversion package 的 sentinel error 轉成統一 JSON 錯誤回應。
//
// 對齊 conversion.md §6 mapping + api-conversion.md 錯誤碼總覽。
//
// 特殊情況:
// - ActiveJobError附帶 `extra.active_job` 給 frontend 顯示「你已有進行中任務」
// - ConverterValidationError附帶 detailsfields給 frontend 顯示具體欄位錯
// - 其他:用 errorMessageFor 拿 user-friendly 訊息
//
// HTTP status / error code 由 conversion.HTTPStatus / conversion.ErrorCode 決定,
// handler 不做二次 mapping。
func handleConversionError(c *gin.Context, err error) {
if err == nil {
// defensive — caller bug
WriteError(c, http.StatusInternalServerError, ErrCodeInternalError,
"unknown error (handleConversionError called with nil)", nil)
return
}
// ctx cancel / deadline — handler 不主動回client 已斷線gin 收到時通常已 abort
if errors.Is(err, context.Canceled) || errors.Is(err, context.DeadlineExceeded) {
// gin context aborted 時 c.Writer 仍可寫(但 client 看不到),保持簡單寫入
WriteError(c, http.StatusServiceUnavailable, "request_cancelled",
"request cancelled or timed out", nil)
return
}
status := conversion.HTTPStatus(err)
code := conversion.ErrorCode(err)
message := errorMessageFor(code)
// ActiveJobError — 帶 active_job detail前端可顯示「跳到該 job 進度頁」)
var aje *conversion.ActiveJobError
if errors.As(err, &aje) && aje != nil {
var jobJSON any
if aje.Job != nil {
jobJSON = jobToResponse(aje.Job)
}
writeConversionErrorWithExtra(c, status, code, message, nil, gin.H{
"active_job": jobJSON,
})
return
}
// ConverterValidationError — 帶 details.fields
var cve *conversion.ConverterValidationError
if errors.As(err, &cve) && cve != nil {
details := make([]FieldError, 0, len(cve.Fields))
for _, f := range cve.Fields {
details = append(details, FieldError{Field: f.Field, Message: f.Message})
}
WriteError(c, status, code, message, details)
return
}
// 一般 sentinel
WriteError(c, status, code, message, nil)
}
// writeConversionErrorWithExtra 是 WriteError 的擴充版本 — 額外帶 extra 結構化資料。
//
// 用於 ActiveJobError 等需要在 error body 內帶結構化 detail 的場景。
//
// 為什麼不直接複用 errors.go 的 WriteError
// WriteError 簽章是 (status, code, message, details []FieldError) — details 為陣列;
// ActiveJobError 要帶的是 objectactive_job。errors.go 的 ErrorDetail 已預留 Extra
// 欄位給此用途。
func writeConversionErrorWithExtra(c *gin.Context, status int, code, message string,
details []FieldError, extra map[string]any,
) {
c.JSON(status, ErrorBody{
Success: false,
Error: &ErrorDetail{
Code: code,
Message: message,
Details: details,
RequestID: RequestIDFrom(c),
Extra: extra,
},
})
}
// errorMessageFor 把 conversion error code 對應到 zh-TW user-friendly 訊息。
//
// 對齊 api-conversion.md 錯誤碼總覽 i18n 預設訊息。
// 真正的 i18n 切換在 frontend 處理(用 code 當 i18n keybackend 只回預設 zh-TW。
func errorMessageFor(code string) string {
switch code {
case "validation_failed":
return "上傳的內容不符合要求"
case "unauthorized":
return "請先登入"
case "forbidden":
return "你無權存取此任務"
case "not_found":
return "任務不存在"
case "active_job_exists":
return "你目前已有進行中的轉檔任務"
case "job_not_completed":
return "任務尚未完成"
case "payload_too_large":
return "檔案超過大小限制"
case "converter_unavailable":
return "轉檔服務暫時無法使用"
case "faa_unavailable":
return "檔案存取服務暫時無法使用"
case "download_token_failed":
return "無法取得下載授權"
case "mc_token_unavailable":
return "無法取得下載授權,請重試"
case "idp_misconfigured":
return "系統設定錯誤,請聯絡支援"
case "idp_unavailable":
return "認證服務暫時無法使用"
case "service_busy":
return "系統繁忙,請稍後再試"
default:
return "內部錯誤"
}
}
// ==========================================================================
// Response shape helper
// ==========================================================================
// jobToResponse 把 internal *conversion.Job 轉成 api-conversion.md §1-2 規定的 JSON shape。
//
// 直接用 gin.Hmap而非 struct — 為了讓 stage / progress / error_* 等選填欄位
// 在「沒值」時可以直接省略(不出現在 JSON符合 api-conversion.md §2 範例
// error_code: null vs 缺欄位 — 我們選缺欄位frontend 用 nullable 邏輯處理)。
//
// 時間欄位用 RFC3339Go time.Time 預設 marshal 即 RFC3339
func jobToResponse(j *conversion.Job) gin.H {
if j == nil {
return nil
}
out := gin.H{
"job_id": j.JobID,
"status": j.Status,
"created_at": j.CreatedAt,
"updated_at": j.UpdatedAt,
"expires_at": j.ExpiresAt,
"progress": j.Progress,
"stage_progress": j.StageProgress, // T7 review M-2: 對齊 api-conversion.md §1 範例顯式列出 stage_progress即使為 0
}
// 選填欄位 — 有值才寫
if j.Stage != "" {
out["stage"] = j.Stage
}
if j.SourceFilename != "" {
out["source_filename"] = j.SourceFilename
}
if j.TargetChip != "" {
out["target_chip"] = j.TargetChip
}
if j.ErrorCode != "" {
out["error_code"] = j.ErrorCode
}
if j.ErrorMessage != "" {
out["error_message"] = j.ErrorMessage
}
return out
}