jim800121chen 1231bf0ed2 feat(visionA-backend): Phase 0.8 conversion package — 5 endpoint + 8 個內部模組
Phase 0.8 把 kneron_model_converter 的轉檔功能整合進 visionA Cloud。
visionA backend 當 streaming proxy(upload)+ delegated download token broker(download)+
ownership trust boundary,converter / FAA / MC 三方零修改。

新增 internal/conversion/ 套件(8 個檔,~10,000 行 prod+test,117+ test cases,race -count=3 全綠):

- conversion.go:Service interface 5 method、Job/PromoteResult/InitJobInput types
- errors.go:13+ sentinel errors + ErrorCode/HTTPStatus mapping,對齊 conversion.md §6
- mc_token_client.go:service-to-service token (client_credentials grant) + DCL cache
  (exp - 15s 重取,per-scope cache),IssueDelegatedDownload(MC delegated download token)
  錯誤分 idp_misconfigured (4xx) / idp_unavailable (5xx) / download_token_failed / mc_token_unavailable
- converter_client.go:對 converter scheduler 4 method(InitJob multipart streaming /
  GetJob / Promote / ListInProgressJobs),InitJob 不 retry 5xx(streaming body 無法 replay)
- faa_client.go:對 FAA GET /files/{key} server-to-server pull,Phase A retry(GET 無 body
  可 replay)對齊 §9.1 retry 矩陣,streaming io.ReadCloser 透傳避 OOM
- ownership.go:in-memory job_id → user_id map + per-user mutex 防 thundering herd lazy rebuild
  (不同 user 平行 fetch,同 user 100 caller 收斂成 1 次),visionA 重啟靠 converter
  ListInProgressJobs(user) 重建
- flow.go:Service interface 整合層(5 method 串接 converter/FAA/MC/ownership)
  - InitJob 用 io.Pipe + multipart.Reader/Writer 重組 streaming proxy(黑名單 client user_id
    + 灌入 OIDC sub)
  - DownloadRedirectURL 自動觸發 promote(spec §1 Stage 3b),用 ensurePromoted helper
  - PromoteToModels 冪等(modelStore.FindBySourceJobID 為 source-of-truth)
  - OwnershipMismatch → ErrJobNotFound 不 forbidden(§7.2 防枚舉)
  - storage / modelStore 失敗包 ErrStorageUnavailable / ErrModelStoreUnavailable
    (視為 visionA 自身 500 而非 502 gateway,SRE alarm 才打對 team)

新增 internal/api/conversion.go(5 endpoint handler + main.go wire):
- POST /api/conversion/init(multipart streaming proxy,不呼叫 c.MultipartForm())
- GET  /api/conversion/active(lazy rebuild ownership)
- GET  /api/conversion/{job_id}(poll status)
- POST /api/conversion/{job_id}/promote-to-models(FAA pull → models 三段式)
- GET  /api/conversion/{job_id}/download(server-side HTTP 302 → FAA,token 不過 frontend
  JS,仿 FAA TestSite DownloadFileDirect pattern;Cache-Control: no-store)

5 個 endpoint 全部走 OIDC AuthMiddleware;user_id 從 cookie session 灌(trust boundary),
從不接受 client multipart form / JSON / query 的 user_id。
TestAllAPIEndpointsRequire401WithoutCookie 自動覆蓋新 5 endpoint regression 防呆。

新增 cmd/api-server/conversion_e2e_test.go(4 個 e2e 場景):
- TestConversionE2E_StreamingProxy(10MB body + trust boundary regression)
- TestConversionE2E_LazyRebuildAfterRestart(visionA 重啟仍能 /active)
- TestConversionE2E_Download302Redirect(驗 302 + Location header + token 不在 body)
- TestConversionE2E_ActiveJobConflict(409 + active_job 詳情)

修改 internal/config/{config,load}.go:新增 ConversionConfig 5 欄位
(ConverterBaseURL / FAABaseURL / TenantID / ServiceClientID / ServiceClientSecret)+
Enabled() helper(雙非空判定)。
修改 cmd/api-server/main.go:條件 wire(cfg.Conversion.Enabled() 為 true 才建 client + Service;
否則 Deps.Conversion=nil,handler 自動回 501)。
修改 .env.example:新增 Phase 0.8 區塊註解。
新增 cmd/api-server/conversion_adapters.go:narrow interface adapter(接既有
internal/model.Repository / internal/storage.Store → conversion.ModelStore / Storage,避免 import cycle)。

驗證:go test -race -count=3 ./... 17 packages 全綠 / go vet 0 warning / go build 成功。

對齊文件:
- .autoflow/04-architecture/adr/adr-014-conversion-integration.md
- .autoflow/04-architecture/conversion.md (TDD)
- .autoflow/04-architecture/api/api-conversion.md
- .autoflow/02-prd/features/feature-converter-integration.md
- .autoflow/03-design/wireframes/wireframe-conversion.md
- .autoflow/03-design/flows/flow-conversion.md

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-04 13:56:07 +08:00

140 lines
5.2 KiB
Go
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

package config
import (
"os"
"strconv"
"strings"
"time"
)
// Load 從環境變數讀取並組出一個 Config。
//
// 所有欄位皆有預設值(雛形便利),因此 Load 不會回傳 error
// 未來加入必填欄位時(例如 Phase 1 的 DB URL應改為回傳 error。
func Load() *Config {
return &Config{
Server: ServerConfig{
Host: getEnvString("VISIONA_HOST", "0.0.0.0"),
Port: getEnvInt("VISIONA_API_PORT", 3721),
TunnelPort: getEnvInt("VISIONA_TUNNEL_PORT", 3800),
InternalPort: getEnvInt("VISIONA_PROXY_INTERNAL_PORT", 3801),
RelayPublicURL: getEnvString("VISIONA_RELAY_PUBLIC_URL", ""),
SeedDemoData: getEnvBool("VISIONA_SEED_DEMO_DATA", false),
},
Session: SessionConfig{
Backend: getEnvString("VISIONA_SESSION_BACKEND", "inmemory"),
ProxyInternalURL: getEnvString("VISIONA_PROXY_INTERNAL_URL", "http://localhost:3801"),
},
Auth: AuthConfig{
// Phase 0.7 security fix C1VISIONA_STATIC_USER_ID 僅供 dev seed / unit test 用,
// stage/prod 留空無影響;不再注入 api.Deps見 internal/api/api.go Deps 註解)。
StaticUserID: getEnvString("VISIONA_STATIC_USER_ID", "demo-user"),
PairingToken: getEnvString("VISIONA_PAIRING_TOKEN", ""),
SigningSecret: getEnvString("VISIONA_STORAGE_SIGNING_SECRET", "dev-signing-secret-do-not-use-in-prod"),
},
OIDC: OIDCConfig{
IssuerURL: getEnvString("VISIONA_OIDC_ISSUER_URL", ""),
ClientID: getEnvString("VISIONA_OIDC_CLIENT_ID", ""),
ClientSecret: getEnvString("VISIONA_OIDC_CLIENT_SECRET", ""),
RedirectURL: getEnvString("VISIONA_OIDC_REDIRECT_URL", ""),
PostLoginURL: getEnvString("VISIONA_FRONTEND_URL", ""),
// A1client_credentials grant 預留欄位,留空表「不啟用 service client」。
ServiceClientID: getEnvString("VISIONA_OIDC_SERVICE_CLIENT_ID", ""),
ServiceClientSecret: getEnvString("VISIONA_OIDC_SERVICE_CLIENT_SECRET", ""),
},
UserSession: UserSessionConfig{
Secret: getEnvString("VISIONA_SESSION_SECRET", ""),
CookieName: getEnvString("VISIONA_SESSION_COOKIE_NAME", "visiona_session"),
CookieDomain: getEnvString("VISIONA_SESSION_COOKIE_DOMAIN", ""),
CookieSecure: getEnvBool("VISIONA_SESSION_COOKIE_SECURE", false),
AbsoluteTTL: getEnvDuration("VISIONA_SESSION_ABSOLUTE_TTL", 168*time.Hour),
IdleTTL: getEnvDuration("VISIONA_SESSION_IDLE_TTL", 24*time.Hour),
},
Storage: StorageConfig{
Backend: getEnvString("VISIONA_STORAGE_BACKEND", "localfs"),
RootDir: getEnvString("VISIONA_STORAGE_LOCALFS_ROOT", "./data/storage"),
BaseURL: getEnvString("VISIONA_STORAGE_LOCALFS_BASE_URL", "http://localhost:3721/storage"),
},
Model: ModelConfig{
MaxSizeMB: getEnvInt("VISIONA_MODEL_MAX_SIZE_MB", 100),
},
Tunnel: TunnelConfig{
HeartbeatInterval: getEnvDuration("VISIONA_TUNNEL_HEARTBEAT_INTERVAL", 10*time.Second),
IdleTimeout: getEnvDuration("VISIONA_TUNNEL_IDLE_TIMEOUT", 30*time.Second),
},
Logger: LoggerConfig{
Level: getEnvString("VISIONA_LOG_LEVEL", "info"),
},
CORS: CORSConfig{
AllowedOrigins: getEnvStringSlice("VISIONA_CORS_ALLOWED_ORIGINS", nil),
},
// Phase 0.8 conversion (見 .autoflow/04-architecture/conversion.md §5.3)
Conversion: ConversionConfig{
ConverterBaseURL: getEnvString("VISIONA_CONVERTER_BASE_URL", ""),
FAABaseURL: getEnvString("VISIONA_FAA_BASE_URL", ""),
TenantID: getEnvString("VISIONA_OIDC_TENANT_ID", ""),
DelegatedTTLSeconds: getEnvInt("VISIONA_FAA_DELEGATED_TTL_SECONDS", 300),
MaxModelSizeMB: getEnvInt("VISIONA_CONVERTER_MAX_MODEL_SIZE_MB", 500),
},
}
}
// getEnvStringSlice 從環境變數取逗號分隔字串,拆成 slice。
// 每段都會 TrimSpace空段會被過濾。若環境變數未設定或為空回傳 fallback。
func getEnvStringSlice(key string, fallback []string) []string {
v, ok := os.LookupEnv(key)
if !ok || v == "" {
return fallback
}
parts := strings.Split(v, ",")
result := make([]string, 0, len(parts))
for _, p := range parts {
if trimmed := strings.TrimSpace(p); trimmed != "" {
result = append(result, trimmed)
}
}
if len(result) == 0 {
return fallback
}
return result
}
// getEnvString 從環境變數取字串,不存在或為空則回傳預設值。
func getEnvString(key, fallback string) string {
if v, ok := os.LookupEnv(key); ok && v != "" {
return v
}
return fallback
}
// getEnvInt 從環境變數取整數,若無法解析則回傳預設值。
func getEnvInt(key string, fallback int) int {
if v, ok := os.LookupEnv(key); ok && v != "" {
if n, err := strconv.Atoi(v); err == nil {
return n
}
}
return fallback
}
// getEnvDuration 從環境變數取 time.Duration支援 "10s"、"1m" 等格式)。
func getEnvDuration(key string, fallback time.Duration) time.Duration {
if v, ok := os.LookupEnv(key); ok && v != "" {
if d, err := time.ParseDuration(v); err == nil {
return d
}
}
return fallback
}
// getEnvBool 從環境變數取布林值(接受 "true"/"false"/"1"/"0",大小寫不敏感)。
// 解析失敗或未設定回傳 fallback。
func getEnvBool(key string, fallback bool) bool {
if v, ok := os.LookupEnv(key); ok && v != "" {
if b, err := strconv.ParseBool(v); err == nil {
return b
}
}
return fallback
}