jim800121chen 1231bf0ed2 feat(visionA-backend): Phase 0.8 conversion package — 5 endpoint + 8 個內部模組
Phase 0.8 把 kneron_model_converter 的轉檔功能整合進 visionA Cloud。
visionA backend 當 streaming proxy(upload)+ delegated download token broker(download)+
ownership trust boundary,converter / FAA / MC 三方零修改。

新增 internal/conversion/ 套件(8 個檔,~10,000 行 prod+test,117+ test cases,race -count=3 全綠):

- conversion.go:Service interface 5 method、Job/PromoteResult/InitJobInput types
- errors.go:13+ sentinel errors + ErrorCode/HTTPStatus mapping,對齊 conversion.md §6
- mc_token_client.go:service-to-service token (client_credentials grant) + DCL cache
  (exp - 15s 重取,per-scope cache),IssueDelegatedDownload(MC delegated download token)
  錯誤分 idp_misconfigured (4xx) / idp_unavailable (5xx) / download_token_failed / mc_token_unavailable
- converter_client.go:對 converter scheduler 4 method(InitJob multipart streaming /
  GetJob / Promote / ListInProgressJobs),InitJob 不 retry 5xx(streaming body 無法 replay)
- faa_client.go:對 FAA GET /files/{key} server-to-server pull,Phase A retry(GET 無 body
  可 replay)對齊 §9.1 retry 矩陣,streaming io.ReadCloser 透傳避 OOM
- ownership.go:in-memory job_id → user_id map + per-user mutex 防 thundering herd lazy rebuild
  (不同 user 平行 fetch,同 user 100 caller 收斂成 1 次),visionA 重啟靠 converter
  ListInProgressJobs(user) 重建
- flow.go:Service interface 整合層(5 method 串接 converter/FAA/MC/ownership)
  - InitJob 用 io.Pipe + multipart.Reader/Writer 重組 streaming proxy(黑名單 client user_id
    + 灌入 OIDC sub)
  - DownloadRedirectURL 自動觸發 promote(spec §1 Stage 3b),用 ensurePromoted helper
  - PromoteToModels 冪等(modelStore.FindBySourceJobID 為 source-of-truth)
  - OwnershipMismatch → ErrJobNotFound 不 forbidden(§7.2 防枚舉)
  - storage / modelStore 失敗包 ErrStorageUnavailable / ErrModelStoreUnavailable
    (視為 visionA 自身 500 而非 502 gateway,SRE alarm 才打對 team)

新增 internal/api/conversion.go(5 endpoint handler + main.go wire):
- POST /api/conversion/init(multipart streaming proxy,不呼叫 c.MultipartForm())
- GET  /api/conversion/active(lazy rebuild ownership)
- GET  /api/conversion/{job_id}(poll status)
- POST /api/conversion/{job_id}/promote-to-models(FAA pull → models 三段式)
- GET  /api/conversion/{job_id}/download(server-side HTTP 302 → FAA,token 不過 frontend
  JS,仿 FAA TestSite DownloadFileDirect pattern;Cache-Control: no-store)

5 個 endpoint 全部走 OIDC AuthMiddleware;user_id 從 cookie session 灌(trust boundary),
從不接受 client multipart form / JSON / query 的 user_id。
TestAllAPIEndpointsRequire401WithoutCookie 自動覆蓋新 5 endpoint regression 防呆。

新增 cmd/api-server/conversion_e2e_test.go(4 個 e2e 場景):
- TestConversionE2E_StreamingProxy(10MB body + trust boundary regression)
- TestConversionE2E_LazyRebuildAfterRestart(visionA 重啟仍能 /active)
- TestConversionE2E_Download302Redirect(驗 302 + Location header + token 不在 body)
- TestConversionE2E_ActiveJobConflict(409 + active_job 詳情)

修改 internal/config/{config,load}.go:新增 ConversionConfig 5 欄位
(ConverterBaseURL / FAABaseURL / TenantID / ServiceClientID / ServiceClientSecret)+
Enabled() helper(雙非空判定)。
修改 cmd/api-server/main.go:條件 wire(cfg.Conversion.Enabled() 為 true 才建 client + Service;
否則 Deps.Conversion=nil,handler 自動回 501)。
修改 .env.example:新增 Phase 0.8 區塊註解。
新增 cmd/api-server/conversion_adapters.go:narrow interface adapter(接既有
internal/model.Repository / internal/storage.Store → conversion.ModelStore / Storage,避免 import cycle)。

驗證:go test -race -count=3 ./... 17 packages 全綠 / go vet 0 warning / go build 成功。

對齊文件:
- .autoflow/04-architecture/adr/adr-014-conversion-integration.md
- .autoflow/04-architecture/conversion.md (TDD)
- .autoflow/04-architecture/api/api-conversion.md
- .autoflow/02-prd/features/feature-converter-integration.md
- .autoflow/03-design/wireframes/wireframe-conversion.md
- .autoflow/03-design/flows/flow-conversion.md

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-04 13:56:07 +08:00

331 lines
13 KiB
Go
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

package config
import (
"testing"
"time"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func TestLoad_Defaults(t *testing.T) {
// Arrange清掉所有相關 envt.Setenv 自動還原)
for _, k := range []string{
"VISIONA_HOST", "VISIONA_API_PORT", "VISIONA_TUNNEL_PORT", "VISIONA_PROXY_INTERNAL_PORT",
"VISIONA_SESSION_BACKEND", "VISIONA_PROXY_INTERNAL_URL",
"VISIONA_AUTH_TYPE", "VISIONA_STATIC_USER_ID", "VISIONA_PAIRING_TOKEN",
"VISIONA_STORAGE_BACKEND", "VISIONA_STORAGE_LOCALFS_ROOT",
"VISIONA_MODEL_MAX_SIZE_MB",
"VISIONA_TUNNEL_HEARTBEAT_INTERVAL", "VISIONA_TUNNEL_IDLE_TIMEOUT",
"VISIONA_LOG_LEVEL",
} {
t.Setenv(k, "")
}
// Act
cfg := Load()
// Assert
assert.Equal(t, "0.0.0.0", cfg.Server.Host)
// Port 預設改為 3721 — 對齊 local-toolB4
assert.Equal(t, 3721, cfg.Server.Port)
assert.Equal(t, 3800, cfg.Server.TunnelPort)
assert.Equal(t, 3801, cfg.Server.InternalPort)
assert.False(t, cfg.Server.SeedDemoData, "預設不 seed demo data")
assert.Equal(t, "inmemory", cfg.Session.Backend)
assert.Equal(t, "http://localhost:3801", cfg.Session.ProxyInternalURL)
assert.Equal(t, "demo-user", cfg.Auth.StaticUserID)
assert.Equal(t, "localfs", cfg.Storage.Backend)
assert.Equal(t, "./data/storage", cfg.Storage.RootDir)
assert.Equal(t, 100, cfg.Model.MaxSizeMB)
assert.Equal(t, 10*time.Second, cfg.Tunnel.HeartbeatInterval)
assert.Equal(t, 30*time.Second, cfg.Tunnel.IdleTimeout)
assert.Equal(t, "info", cfg.Logger.Level)
}
func TestLoad_EnvOverrides(t *testing.T) {
t.Setenv("VISIONA_API_PORT", "8080")
t.Setenv("VISIONA_STATIC_USER_ID", "custom-user")
t.Setenv("VISIONA_MODEL_MAX_SIZE_MB", "500")
t.Setenv("VISIONA_TUNNEL_HEARTBEAT_INTERVAL", "5s")
t.Setenv("VISIONA_LOG_LEVEL", "debug")
cfg := Load()
assert.Equal(t, 8080, cfg.Server.Port)
assert.Equal(t, "custom-user", cfg.Auth.StaticUserID)
assert.Equal(t, 500, cfg.Model.MaxSizeMB)
assert.Equal(t, 5*time.Second, cfg.Tunnel.HeartbeatInterval)
assert.Equal(t, "debug", cfg.Logger.Level)
}
func TestLoad_InvalidIntFallback(t *testing.T) {
t.Setenv("VISIONA_API_PORT", "not-a-number")
cfg := Load()
assert.Equal(t, 3721, cfg.Server.Port, "無法解析時應 fallback 到預設值B4 改為 3721")
}
// TestLoad_SeedDemoData 驗證 VISIONA_SEED_DEMO_DATA env 的解析行為。
func TestLoad_SeedDemoData(t *testing.T) {
t.Setenv("VISIONA_SEED_DEMO_DATA", "true")
cfg := Load()
assert.True(t, cfg.Server.SeedDemoData)
t.Setenv("VISIONA_SEED_DEMO_DATA", "false")
cfg = Load()
assert.False(t, cfg.Server.SeedDemoData)
// 無法解析時 fallback 到預設 false
t.Setenv("VISIONA_SEED_DEMO_DATA", "not-a-bool")
cfg = Load()
assert.False(t, cfg.Server.SeedDemoData, "無法解析時應 fallback 到預設值")
}
// TestLoad_OIDCDefaults 驗證未設定任何 VISIONA_OIDC_* 時OIDC 欄位為空字串。
//
// OB5 起 OIDC.Enabled 已移除OIDC 是唯一認證路徑);空字串就是「未設定」,
// 此時 Validate() 會回 MissingEnvErrormain.go 啟動時 fatal log 退出。
func TestLoad_OIDCDefaults(t *testing.T) {
for _, k := range []string{
"VISIONA_OIDC_ISSUER_URL", "VISIONA_OIDC_CLIENT_ID",
"VISIONA_OIDC_CLIENT_SECRET", "VISIONA_OIDC_REDIRECT_URL", "VISIONA_FRONTEND_URL",
"VISIONA_OIDC_SERVICE_CLIENT_ID", "VISIONA_OIDC_SERVICE_CLIENT_SECRET",
"VISIONA_SESSION_SECRET", "VISIONA_SESSION_COOKIE_NAME", "VISIONA_SESSION_COOKIE_DOMAIN",
"VISIONA_SESSION_COOKIE_SECURE", "VISIONA_SESSION_ABSOLUTE_TTL", "VISIONA_SESSION_IDLE_TTL",
} {
t.Setenv(k, "")
}
cfg := Load()
assert.Empty(t, cfg.OIDC.IssuerURL)
assert.Empty(t, cfg.OIDC.ClientID)
assert.Empty(t, cfg.OIDC.ClientSecret)
assert.Empty(t, cfg.OIDC.RedirectURL)
assert.Empty(t, cfg.OIDC.PostLoginURL)
assert.Empty(t, cfg.OIDC.ServiceClientID, "ServiceClientID 預設留空A1未啟用")
assert.Empty(t, cfg.OIDC.ServiceClientSecret, "ServiceClientSecret 預設留空A1未啟用")
assert.Empty(t, cfg.UserSession.Secret, "雛形 dev 預設不附 secret由 caller 注入或啟動失敗")
assert.Equal(t, "visiona_session", cfg.UserSession.CookieName)
assert.Empty(t, cfg.UserSession.CookieDomain)
assert.False(t, cfg.UserSession.CookieSecure)
assert.Equal(t, 168*time.Hour, cfg.UserSession.AbsoluteTTL)
assert.Equal(t, 24*time.Hour, cfg.UserSession.IdleTTL)
}
// TestLoad_OIDC_ClientSecretOptionalA12026-05-01— 缺 ClientSecret 不再回 MissingEnvError。
//
// 模擬 Stage 用的 public PKCE-only clientMC 給的 b8093fea... 沒有 client_secret
func TestLoad_OIDC_ClientSecretOptional(t *testing.T) {
t.Setenv("VISIONA_OIDC_ISSUER_URL", "https://stage-9527.innovedus.com:7850/")
t.Setenv("VISIONA_OIDC_CLIENT_ID", "b8093fea1a504a5d8f0e04bee9f78f2e")
t.Setenv("VISIONA_OIDC_CLIENT_SECRET", "") // 故意留空 — public client
t.Setenv("VISIONA_OIDC_REDIRECT_URL", "https://stage-9527.innovedus.com:9527/api/auth/callback")
t.Setenv("VISIONA_FRONTEND_URL", "https://stage-9527.innovedus.com:9527")
t.Setenv("VISIONA_SESSION_SECRET", "32-byte-or-longer-random-secret-aaaa")
cfg := Load()
assert.Empty(t, cfg.OIDC.ClientSecret, "public client modeClientSecret 應為空字串")
assert.NoError(t, cfg.Validate(), "ClientSecret 為空不應觸發 MissingEnvError")
}
// TestLoad_OIDC_ServiceClientFieldsA1 預留 client_credentials grant 兩個欄位能正確讀取。
// 測試固定值故意用顯而易見的 fake — 不要貼任何環境的真實 client_id / secret 進測試。
func TestLoad_OIDC_ServiceClientFields(t *testing.T) {
const fakeServiceID = "fake-service-client-id-for-test"
const fakeServiceSecret = "fake-service-client-secret-for-test"
t.Setenv("VISIONA_OIDC_SERVICE_CLIENT_ID", fakeServiceID)
t.Setenv("VISIONA_OIDC_SERVICE_CLIENT_SECRET", fakeServiceSecret)
cfg := Load()
assert.Equal(t, fakeServiceID, cfg.OIDC.ServiceClientID)
assert.Equal(t, fakeServiceSecret, cfg.OIDC.ServiceClientSecret)
}
// TestLoad_OIDCAllSet 驗證 OIDC env vars 設定後能正確讀取。
func TestLoad_OIDCAllSet(t *testing.T) {
t.Setenv("VISIONA_OIDC_ISSUER_URL", "http://localhost:5050")
t.Setenv("VISIONA_OIDC_CLIENT_ID", "visionA")
t.Setenv("VISIONA_OIDC_CLIENT_SECRET", "secret")
t.Setenv("VISIONA_OIDC_REDIRECT_URL", "http://localhost:3721/api/auth/callback")
t.Setenv("VISIONA_FRONTEND_URL", "http://localhost:3000")
t.Setenv("VISIONA_SESSION_SECRET", "32-byte-or-longer-random-secret-aaaa")
t.Setenv("VISIONA_SESSION_COOKIE_SECURE", "true")
t.Setenv("VISIONA_SESSION_ABSOLUTE_TTL", "72h")
t.Setenv("VISIONA_SESSION_IDLE_TTL", "12h")
cfg := Load()
assert.Equal(t, "http://localhost:5050", cfg.OIDC.IssuerURL)
assert.Equal(t, "visionA", cfg.OIDC.ClientID)
assert.Equal(t, "secret", cfg.OIDC.ClientSecret)
assert.Equal(t, "http://localhost:3721/api/auth/callback", cfg.OIDC.RedirectURL)
assert.Equal(t, "http://localhost:3000", cfg.OIDC.PostLoginURL)
assert.Equal(t, "32-byte-or-longer-random-secret-aaaa", cfg.UserSession.Secret)
assert.True(t, cfg.UserSession.CookieSecure)
assert.Equal(t, 72*time.Hour, cfg.UserSession.AbsoluteTTL)
assert.Equal(t, 12*time.Hour, cfg.UserSession.IdleTTL)
}
// TestConfig_Validate_MissingFields 驗證 OIDC 必填欄位缺失時回 MissingEnvError。
//
// A12026-05-01ClientSecret 改為選填,已從必填清單移除;剩 5 項必填。
func TestConfig_Validate_MissingFields(t *testing.T) {
cfg := &Config{} // 全部欄位 zero value
err := cfg.Validate()
require.Error(t, err)
var missErr *MissingEnvError
require.ErrorAs(t, err, &missErr, "錯誤型別應可被 errors.As 解出")
// 應列出 5 個必填欄位(不含 ClientSecret
assert.ElementsMatch(t, []string{
"VISIONA_OIDC_ISSUER_URL",
"VISIONA_OIDC_CLIENT_ID",
"VISIONA_OIDC_REDIRECT_URL",
"VISIONA_FRONTEND_URL",
"VISIONA_SESSION_SECRET",
}, missErr.Vars)
assert.NotContains(t, missErr.Vars, "VISIONA_OIDC_CLIENT_SECRET",
"A1ClientSecret 為選填,不應出現在必填缺失清單")
}
// TestValidate_ConfidentialClient完整 confidential client含 ClientSecret能通過 Validate。
func TestValidate_ConfidentialClient(t *testing.T) {
cfg := &Config{
OIDC: OIDCConfig{
IssuerURL: "http://localhost:5050",
ClientID: "visionA",
ClientSecret: "secret", // 有值 → confidential mode
RedirectURL: "http://localhost:3721/api/auth/callback",
PostLoginURL: "http://localhost:3000",
},
UserSession: UserSessionConfig{Secret: "session-secret-32-bytes-aaaaaaaaaaaa"},
}
assert.NoError(t, cfg.Validate())
}
// TestValidate_PKCEOnlyPublicClientA1 — 只給 ClientID 沒給 Secret 也能通過 Validate。
//
// 對應 Stage 部署的真實情境MC 配給 visionA 的 client `b8093fea1a504a5d8f0e04bee9f78f2e`
// 是 public client沒有 client_secret靠 PKCE 防 code interception。
func TestValidate_PKCEOnlyPublicClient(t *testing.T) {
cfg := &Config{
OIDC: OIDCConfig{
IssuerURL: "https://stage-9527.innovedus.com:7850/",
ClientID: "b8093fea1a504a5d8f0e04bee9f78f2e",
// ClientSecret 留空 — public PKCE-only client
RedirectURL: "https://stage-9527.innovedus.com:9527/api/auth/callback",
PostLoginURL: "https://stage-9527.innovedus.com:9527",
},
UserSession: UserSessionConfig{Secret: "session-secret-32-bytes-aaaaaaaaaaaa"},
}
assert.NoError(t, cfg.Validate(),
"A1public PKCE-only clientClientSecret 留空)應通過 Validate")
}
// TestValidate_ServiceClientFieldsNotCheckedA1 — ServiceClientID/Secret 留空不影響 Validate。
//
// 兩個欄位是 client_credentials grant 預留鉤子A1 階段不啟用、不檢查。
func TestValidate_ServiceClientFieldsNotChecked(t *testing.T) {
cfg := &Config{
OIDC: OIDCConfig{
IssuerURL: "http://localhost:5050",
ClientID: "visionA",
RedirectURL: "http://localhost:3721/api/auth/callback",
PostLoginURL: "http://localhost:3000",
// 兩個 Service* 都留空 — 預期通過
},
UserSession: UserSessionConfig{Secret: "session-secret-32-bytes-aaaaaaaaaaaa"},
}
assert.NoError(t, cfg.Validate())
}
// TestLoad_CORSAllowedOrigins 驗證 VISIONA_CORS_ALLOWED_ORIGINS 的逗號分隔解析。
// 空字串 / 純分隔字元 → fallback 到 nil交由 api.Deps.validate 塞預設)。
func TestLoad_CORSAllowedOrigins(t *testing.T) {
// 未設 → nil
t.Setenv("VISIONA_CORS_ALLOWED_ORIGINS", "")
cfg := Load()
assert.Nil(t, cfg.CORS.AllowedOrigins)
// 單一 origin
t.Setenv("VISIONA_CORS_ALLOWED_ORIGINS", "http://localhost:3000")
cfg = Load()
assert.Equal(t, []string{"http://localhost:3000"}, cfg.CORS.AllowedOrigins)
// 多個 origin + trim space
t.Setenv("VISIONA_CORS_ALLOWED_ORIGINS", "http://a.com, http://b.com ,http://c.com")
cfg = Load()
assert.Equal(t, []string{"http://a.com", "http://b.com", "http://c.com"}, cfg.CORS.AllowedOrigins)
// 只有分隔字元 → fallback過濾後 len == 0
t.Setenv("VISIONA_CORS_ALLOWED_ORIGINS", " , ,")
cfg = Load()
assert.Nil(t, cfg.CORS.AllowedOrigins)
}
// TestLoad_ConversionDefaults 驗證 Phase 0.8 conversion 欄位的預設行為。
//
// 對齊 .autoflow/04-architecture/conversion.md §5.3:留空時 Enabled() 為 false
// 5 個 endpoint 不會 wiremain.go 在 wire 階段會跳過)。
func TestLoad_ConversionDefaults(t *testing.T) {
for _, k := range []string{
"VISIONA_CONVERTER_BASE_URL", "VISIONA_FAA_BASE_URL", "VISIONA_OIDC_TENANT_ID",
"VISIONA_FAA_DELEGATED_TTL_SECONDS", "VISIONA_CONVERTER_MAX_MODEL_SIZE_MB",
} {
t.Setenv(k, "")
}
cfg := Load()
assert.Empty(t, cfg.Conversion.ConverterBaseURL)
assert.Empty(t, cfg.Conversion.FAABaseURL)
assert.Empty(t, cfg.Conversion.TenantID)
assert.Equal(t, 300, cfg.Conversion.DelegatedTTLSeconds, "預設 5 分鐘 TTL")
assert.Equal(t, 500, cfg.Conversion.MaxModelSizeMB, "預設 500 MB與 converter 對齊)")
assert.False(t, cfg.Conversion.Enabled(), "URL 全空 → 不啟用")
}
// TestLoad_ConversionEnabled 驗證 Conversion.Enabled() 的判定邏輯。
func TestLoad_ConversionEnabled(t *testing.T) {
cases := []struct {
name string
converter string
faa string
wantEnabled bool
}{
{"both_set_enables", "http://converter:9501", "http://faa:5081", true},
{"only_converter_disabled", "http://converter:9501", "", false},
{"only_faa_disabled", "", "http://faa:5081", false},
{"both_empty_disabled", "", "", false},
}
for _, tc := range cases {
tc := tc
t.Run(tc.name, func(t *testing.T) {
t.Setenv("VISIONA_CONVERTER_BASE_URL", tc.converter)
t.Setenv("VISIONA_FAA_BASE_URL", tc.faa)
cfg := Load()
assert.Equal(t, tc.wantEnabled, cfg.Conversion.Enabled())
})
}
}
// TestLoad_ConversionAllSet 驗證所有欄位設定後正確讀取。
func TestLoad_ConversionAllSet(t *testing.T) {
t.Setenv("VISIONA_CONVERTER_BASE_URL", "http://192.168.0.130:9501")
t.Setenv("VISIONA_FAA_BASE_URL", "http://192.168.0.130:5081")
t.Setenv("VISIONA_OIDC_TENANT_ID", "fake-tenant-id-for-test")
t.Setenv("VISIONA_FAA_DELEGATED_TTL_SECONDS", "600")
t.Setenv("VISIONA_CONVERTER_MAX_MODEL_SIZE_MB", "300")
cfg := Load()
assert.Equal(t, "http://192.168.0.130:9501", cfg.Conversion.ConverterBaseURL)
assert.Equal(t, "http://192.168.0.130:5081", cfg.Conversion.FAABaseURL)
assert.Equal(t, "fake-tenant-id-for-test", cfg.Conversion.TenantID)
assert.Equal(t, 600, cfg.Conversion.DelegatedTTLSeconds)
assert.Equal(t, 300, cfg.Conversion.MaxModelSizeMB)
assert.True(t, cfg.Conversion.Enabled())
}