Phase 0.8 把 kneron_model_converter 的轉檔功能整合進 visionA Cloud。
visionA backend 當 streaming proxy(upload)+ delegated download token broker(download)+
ownership trust boundary,converter / FAA / MC 三方零修改。
新增 internal/conversion/ 套件(8 個檔,~10,000 行 prod+test,117+ test cases,race -count=3 全綠):
- conversion.go:Service interface 5 method、Job/PromoteResult/InitJobInput types
- errors.go:13+ sentinel errors + ErrorCode/HTTPStatus mapping,對齊 conversion.md §6
- mc_token_client.go:service-to-service token (client_credentials grant) + DCL cache
(exp - 15s 重取,per-scope cache),IssueDelegatedDownload(MC delegated download token)
錯誤分 idp_misconfigured (4xx) / idp_unavailable (5xx) / download_token_failed / mc_token_unavailable
- converter_client.go:對 converter scheduler 4 method(InitJob multipart streaming /
GetJob / Promote / ListInProgressJobs),InitJob 不 retry 5xx(streaming body 無法 replay)
- faa_client.go:對 FAA GET /files/{key} server-to-server pull,Phase A retry(GET 無 body
可 replay)對齊 §9.1 retry 矩陣,streaming io.ReadCloser 透傳避 OOM
- ownership.go:in-memory job_id → user_id map + per-user mutex 防 thundering herd lazy rebuild
(不同 user 平行 fetch,同 user 100 caller 收斂成 1 次),visionA 重啟靠 converter
ListInProgressJobs(user) 重建
- flow.go:Service interface 整合層(5 method 串接 converter/FAA/MC/ownership)
- InitJob 用 io.Pipe + multipart.Reader/Writer 重組 streaming proxy(黑名單 client user_id
+ 灌入 OIDC sub)
- DownloadRedirectURL 自動觸發 promote(spec §1 Stage 3b),用 ensurePromoted helper
- PromoteToModels 冪等(modelStore.FindBySourceJobID 為 source-of-truth)
- OwnershipMismatch → ErrJobNotFound 不 forbidden(§7.2 防枚舉)
- storage / modelStore 失敗包 ErrStorageUnavailable / ErrModelStoreUnavailable
(視為 visionA 自身 500 而非 502 gateway,SRE alarm 才打對 team)
新增 internal/api/conversion.go(5 endpoint handler + main.go wire):
- POST /api/conversion/init(multipart streaming proxy,不呼叫 c.MultipartForm())
- GET /api/conversion/active(lazy rebuild ownership)
- GET /api/conversion/{job_id}(poll status)
- POST /api/conversion/{job_id}/promote-to-models(FAA pull → models 三段式)
- GET /api/conversion/{job_id}/download(server-side HTTP 302 → FAA,token 不過 frontend
JS,仿 FAA TestSite DownloadFileDirect pattern;Cache-Control: no-store)
5 個 endpoint 全部走 OIDC AuthMiddleware;user_id 從 cookie session 灌(trust boundary),
從不接受 client multipart form / JSON / query 的 user_id。
TestAllAPIEndpointsRequire401WithoutCookie 自動覆蓋新 5 endpoint regression 防呆。
新增 cmd/api-server/conversion_e2e_test.go(4 個 e2e 場景):
- TestConversionE2E_StreamingProxy(10MB body + trust boundary regression)
- TestConversionE2E_LazyRebuildAfterRestart(visionA 重啟仍能 /active)
- TestConversionE2E_Download302Redirect(驗 302 + Location header + token 不在 body)
- TestConversionE2E_ActiveJobConflict(409 + active_job 詳情)
修改 internal/config/{config,load}.go:新增 ConversionConfig 5 欄位
(ConverterBaseURL / FAABaseURL / TenantID / ServiceClientID / ServiceClientSecret)+
Enabled() helper(雙非空判定)。
修改 cmd/api-server/main.go:條件 wire(cfg.Conversion.Enabled() 為 true 才建 client + Service;
否則 Deps.Conversion=nil,handler 自動回 501)。
修改 .env.example:新增 Phase 0.8 區塊註解。
新增 cmd/api-server/conversion_adapters.go:narrow interface adapter(接既有
internal/model.Repository / internal/storage.Store → conversion.ModelStore / Storage,避免 import cycle)。
驗證:go test -race -count=3 ./... 17 packages 全綠 / go vet 0 warning / go build 成功。
對齊文件:
- .autoflow/04-architecture/adr/adr-014-conversion-integration.md
- .autoflow/04-architecture/conversion.md (TDD)
- .autoflow/04-architecture/api/api-conversion.md
- .autoflow/02-prd/features/feature-converter-integration.md
- .autoflow/03-design/wireframes/wireframe-conversion.md
- .autoflow/03-design/flows/flow-conversion.md
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
151 lines
5.3 KiB
Go
151 lines
5.3 KiB
Go
// conversion_adapters.go — main.go 的 narrow adapter 實作。
|
||
//
|
||
// internal/conversion 為了避免 import cycle 與保持 interface 純粹(FAANG 慣例:
|
||
// consumer 定義介面),定義了 ModelStore / Storage 兩個 narrow interface。
|
||
// main.go 在 wire 時把 *model.InMemoryRepository / *storage.LocalFSStore 包成 adapter
|
||
// 注入;conversion 完全不知道具體實作。
|
||
//
|
||
// 對齊 .autoflow/04-architecture/conversion.md §2.7(NewService 註解)。
|
||
//
|
||
// Phase 0.8 conversion (見 .autoflow/04-architecture/conversion.md §2.7)
|
||
package main
|
||
|
||
import (
|
||
"context"
|
||
"fmt"
|
||
"io"
|
||
"time"
|
||
|
||
"github.com/google/uuid"
|
||
|
||
"visiona-backend/internal/conversion"
|
||
"visiona-backend/internal/model"
|
||
"visiona-backend/internal/storage"
|
||
)
|
||
|
||
// ==========================================================================
|
||
// ModelStore adapter
|
||
// ==========================================================================
|
||
|
||
// conversionModelStoreAdapter 把 model.Repository 包成 conversion.ModelStore。
|
||
//
|
||
// 額外責任:
|
||
// - ModelRecord ↔ model.Model 雙向轉換
|
||
// - FindBySourceJobID 用 List(filter) + 自行過濾 SourceJobID(既有 model.ListFilter
|
||
// 沒有 SourceJobID 欄位;Phase 1 加 DB 後可改 indexed query)
|
||
// - GenerateID 用 uuid.NewString(與 internal/api/models.go modelsInitUploadHandler 一致)
|
||
type conversionModelStoreAdapter struct {
|
||
repo model.Repository
|
||
}
|
||
|
||
// newConversionModelStoreAdapter 建立 adapter。
|
||
func newConversionModelStoreAdapter(repo model.Repository) conversion.ModelStore {
|
||
return &conversionModelStoreAdapter{repo: repo}
|
||
}
|
||
|
||
// Save 把 conversion.ModelRecord 轉成 model.Model 後 upsert。
|
||
//
|
||
// 設計選擇:UploadedAt 設為 now(promote 完成 = 等同 finalize 後的 ready 狀態),
|
||
// 這樣 GET /api/models/{id} 回的 status 會是 "ready"(對齊 toModelResponse 邏輯)。
|
||
func (a *conversionModelStoreAdapter) Save(ctx context.Context, rec *conversion.ModelRecord) error {
|
||
if rec == nil {
|
||
return fmt.Errorf("conversion adapter: Save requires non-nil record")
|
||
}
|
||
now := time.Now().UTC()
|
||
uploadedAt := now
|
||
if !rec.UpdatedAt.IsZero() {
|
||
uploadedAt = rec.UpdatedAt
|
||
}
|
||
m := &model.Model{
|
||
ID: rec.ID,
|
||
OwnerUserID: rec.OwnerUserID,
|
||
Name: rec.Name,
|
||
Description: rec.Description,
|
||
StorageKey: rec.StorageKey,
|
||
FileSize: rec.FileSize,
|
||
FileChecksum: rec.FileChecksum,
|
||
TargetChip: rec.TargetChip,
|
||
Source: rec.Source, // 應為 "converted"
|
||
SourceJobID: rec.SourceJobID,
|
||
CreatedAt: rec.CreatedAt,
|
||
UpdatedAt: rec.UpdatedAt,
|
||
UploadedAt: &uploadedAt, // promote 完即 ready(對齊 toModelResponse)
|
||
}
|
||
return a.repo.Save(ctx, m)
|
||
}
|
||
|
||
// FindBySourceJobID 找 user 是否已對某 job 建過 model record(冪等檢查用)。
|
||
//
|
||
// Phase 0.8 雛形實作:用 List(filter) + 過濾 SourceJobID(in-memory 慢但對小量 user 足夠)。
|
||
// Phase 1 用 SQL `WHERE owner_user_id = ? AND source_job_id = ?` 加索引。
|
||
//
|
||
// 找不到回 (nil, nil);找到第一個 match 回 (*ModelRecord, nil)。
|
||
func (a *conversionModelStoreAdapter) FindBySourceJobID(ctx context.Context, ownerUserID, sourceJobID string) (*conversion.ModelRecord, error) {
|
||
if ownerUserID == "" || sourceJobID == "" {
|
||
return nil, nil
|
||
}
|
||
models, err := a.repo.List(ctx, model.ListFilter{
|
||
OwnerUserID: ownerUserID,
|
||
Source: model.SourceConverted,
|
||
})
|
||
if err != nil {
|
||
return nil, fmt.Errorf("conversion adapter: list models for FindBySourceJobID: %w", err)
|
||
}
|
||
for _, m := range models {
|
||
if m.SourceJobID == sourceJobID {
|
||
return modelToRecord(m), nil
|
||
}
|
||
}
|
||
return nil, nil
|
||
}
|
||
|
||
// GenerateID 產一個新 model_id(沿用既有 visionA model 命名 — uuid.NewString)。
|
||
func (a *conversionModelStoreAdapter) GenerateID() string {
|
||
return uuid.NewString()
|
||
}
|
||
|
||
// modelToRecord 把 *model.Model 轉成 *conversion.ModelRecord(給 PromoteToModels 冪等回傳用)。
|
||
func modelToRecord(m *model.Model) *conversion.ModelRecord {
|
||
if m == nil {
|
||
return nil
|
||
}
|
||
return &conversion.ModelRecord{
|
||
ID: m.ID,
|
||
OwnerUserID: m.OwnerUserID,
|
||
Name: m.Name,
|
||
Description: m.Description,
|
||
StorageKey: m.StorageKey,
|
||
FileSize: m.FileSize,
|
||
FileChecksum: m.FileChecksum,
|
||
TargetChip: m.TargetChip,
|
||
Source: m.Source,
|
||
SourceJobID: m.SourceJobID,
|
||
CreatedAt: m.CreatedAt,
|
||
UpdatedAt: m.UpdatedAt,
|
||
}
|
||
}
|
||
|
||
// ==========================================================================
|
||
// Storage adapter
|
||
// ==========================================================================
|
||
|
||
// conversionStorageAdapter 把 storage.Store 包成 conversion.Storage。
|
||
//
|
||
// 目前只需要 Put(streaming 寫入),meta 透傳到底層 storage(LocalFS 雛形可能忽略,
|
||
// S3 接上後會寫進 object metadata)。
|
||
type conversionStorageAdapter struct {
|
||
store storage.Store
|
||
}
|
||
|
||
// newConversionStorageAdapter 建立 adapter。
|
||
func newConversionStorageAdapter(store storage.Store) conversion.Storage {
|
||
return &conversionStorageAdapter{store: store}
|
||
}
|
||
|
||
// Put streaming 寫入 storage(meta 透傳給底層 storage 實作)。
|
||
//
|
||
// LocalFS 雛形可能忽略 meta;S3 / R2 等 backend 會寫進 object metadata(給 debug / Tagging)。
|
||
func (a *conversionStorageAdapter) Put(ctx context.Context, key string, r io.Reader, size int64, meta map[string]string) error {
|
||
return a.store.Put(ctx, key, r, size, meta)
|
||
}
|