kneron_model_converter

warrenchen/kneron_model_converter

Fork 0

Commit Graph

Author	SHA1	Message	Date
jim800121chen	aeaecb8c06	fix(compose): Phase 0.8b deploy blocker — env 透傳 + 命名規格 d8a9517 commit 漏改 docker-compose.yml：scheduler service environment block 沒透傳 Phase 0.8b 新 env、即使 stage .env 設了 container 也讀不到、 deploy 後 CONVERTER_API_KEY undefined 會啟動 503 reject all requests。 docker-compose.yml： - 新增 10 個 Phase 0.8b env 透傳（CONVERTER_API_KEY 無 default fail-secure、其他用 ${VAR:-default} fail-soft） - 砍 9 個已廢 OAuth resource-server env（MEMBER_CENTER_ISSUER / JWKS_URL / AUDIENCE / CONVERTER_TENANT_ID / SCOPE_* / JWKS_* / JWT_） - 保留 8 個 promote → FAA 用 env（MEMBER_CENTER_TOKEN_URL / KNERON_CONVERTER_CLIENT_ID/SECRET / FILE_ACCESS_AGENT_ / OAUTH_TOKEN_* / PROMOTE_TIMEOUT_MS） docs/autoflow/04-architecture/api/api-result.md §16： - 新增 Env Naming Reference Table（30 個 canonical env names） - 拍板 source code 為 single source of truth、env.example 對齊 - 確認 /result 8 個 env + 其他 22 個的命名規格 - 留歷史記錄：Orchestrator 之前用過想像中縮寫名（_MAX / _HOURLY_QUOTA / RESULT_CONCURRENT_STREAM_MAX）造成命名混亂、§16 為未來 prompt 引用標準 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-18 01:01:59 +08:00
jim800121chen	4d381c0b50	feat(task-scheduler): Phase 1 — modularize server + add OAuth/JWKS + /api/v1/* routes Refactor server.js (647 → 99 lines) into 30+ modules under src/: - auth/: JWKS validation, JWT middleware, OAuth client_credentials - routes/v1/: jobs (POST/GET/:id) + promote with input validation - routes/legacy.js: existing /jobs multipart path (backward compatible) - services/: jobService, healthService, sseService, statusMapper, doneListener - middleware/: requestId, errorHandler, perClientRateLimit, uploadConcurrency, upload (multer + storage) - redis/: Lua scripts for atomic claim/release_active_job - storage/: local + minio adapters; fileAccessAgent/: PUT promote client - config.js: env var validation with fail-fast Phase 1 features (T1–T11): - T1 Auth middleware + JWKS (Member Center OAuth2 resource server) - T2 OAuth client (Member Center client_credentials, Basic auth) - T3 /api/v1/* router skeleton - T4 server.js refactor (legacy endpoints fully preserved, real-Redis regression verified — existing worker consumer group untouched) - T5 POST /api/v1/jobs (multipart, OWASP-audited, 2 Critical / 6 Major fixed; Risk-A/B documented as accepted) - T6 GET /api/v1/jobs + GET /:id (cursor pagination, ETag, IDOR-safe) - T7 POST /jobs/:id/promote (FAA PUT with own service token, 300s timeout, fail-fast on missing FAA URL) - T8 /health upgrade (healthy/degraded/unhealthy + 30s background cache) - T9 stage_timings (release_active_job in terminal states) - T10 env + Docker integration (MULTIPART_* + concurrency limiter) - T11 README (498 lines) + OpenAPI 3.0 spec (1588 lines) Tests: 630 pass across 29 suites. Updated Dockerfile + .dockerignore + docker-compose.yml env passthrough (no hardcoded secrets, fail-fast on missing required vars). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-01 10:55:05 +08:00
jim800121chen	efa67d59a4	Add web frontend, MinIO storage, monitoring, and docker-compose deployment - Frontend: rewrite Home.vue to match backend POST /jobs API (remove single-stage options) - Frontend: add Monitor page (/monitor) for queue and job monitoring - Frontend: add job history with localStorage tracking (per-browser) - Frontend: fix Nginx proxy rewrite (/api -> /) and add 500MB upload limit - Backend: add MinIO storage support (STORAGE_BACKEND=minio) alongside local mode - Backend: add GET /queues/stats API for queue monitoring - Backend: fix download handler for MinIO (buffer mode for Node 18 compat) - Workers: add S3/MinIO download/upload in consumer.py with isolated temp dirs - Workers: add s3_storage.py helper with lifecycle rule (7-day TTL) - Docker: add docker-compose.yml with all services (web, scheduler, redis, workers) - Docker: ports mapped to 9500 (web) and 9501 (scheduler) - Config: add .env to .gitignore to protect secrets Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>	2026-04-01 15:04:09 +08:00

Author

SHA1

Message

Date

jim800121chen

aeaecb8c06

fix(compose): Phase 0.8b deploy blocker — env 透傳 + 命名規格

d8a9517 commit 漏改 docker-compose.yml：scheduler service environment block
沒透傳 Phase 0.8b 新 env、即使 stage .env 設了 container 也讀不到、
deploy 後 CONVERTER_API_KEY undefined 會啟動 503 reject all requests。

docker-compose.yml：
- 新增 10 個 Phase 0.8b env 透傳（CONVERTER_API_KEY 無 default fail-secure、
  其他用 ${VAR:-default} fail-soft）
- 砍 9 個已廢 OAuth resource-server env（MEMBER_CENTER_ISSUER / JWKS_URL /
  AUDIENCE / CONVERTER_TENANT_ID / SCOPE_* / JWKS_* / JWT_*）
- 保留 8 個 promote → FAA 用 env（MEMBER_CENTER_TOKEN_URL /
  KNERON_CONVERTER_CLIENT_ID/SECRET / FILE_ACCESS_AGENT_* /
  OAUTH_TOKEN_* / PROMOTE_TIMEOUT_MS）

docs/autoflow/04-architecture/api/api-result.md §16：
- 新增 Env Naming Reference Table（30 個 canonical env names）
- 拍板 source code 為 single source of truth、env.example 對齊
- 確認 /result 8 個 env + 其他 22 個的命名規格
- 留歷史記錄：Orchestrator 之前用過想像中縮寫名（_MAX / _HOURLY_QUOTA /
  RESULT_CONCURRENT_STREAM_MAX）造成命名混亂、§16 為未來 prompt 引用標準

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

2026-05-18 01:01:59 +08:00

jim800121chen

4d381c0b50

feat(task-scheduler): Phase 1 — modularize server + add OAuth/JWKS + /api/v1/* routes

Refactor server.js (647 → 99 lines) into 30+ modules under src/:
- auth/: JWKS validation, JWT middleware, OAuth client_credentials
- routes/v1/: jobs (POST/GET/:id) + promote with input validation
- routes/legacy.js: existing /jobs multipart path (backward compatible)
- services/: jobService, healthService, sseService, statusMapper,
  doneListener
- middleware/: requestId, errorHandler, perClientRateLimit,
  uploadConcurrency, upload (multer + storage)
- redis/: Lua scripts for atomic claim/release_active_job
- storage/: local + minio adapters; fileAccessAgent/: PUT promote client
- config.js: env var validation with fail-fast

Phase 1 features (T1–T11):
- T1 Auth middleware + JWKS (Member Center OAuth2 resource server)
- T2 OAuth client (Member Center client_credentials, Basic auth)
- T3 /api/v1/* router skeleton
- T4 server.js refactor (legacy endpoints fully preserved, real-Redis
  regression verified — existing worker consumer group untouched)
- T5 POST /api/v1/jobs (multipart, OWASP-audited, 2 Critical / 6 Major
  fixed; Risk-A/B documented as accepted)
- T6 GET /api/v1/jobs + GET /:id (cursor pagination, ETag, IDOR-safe)
- T7 POST /jobs/:id/promote (FAA PUT with own service token, 300s
  timeout, fail-fast on missing FAA URL)
- T8 /health upgrade (healthy/degraded/unhealthy + 30s background cache)
- T9 stage_timings (release_active_job in terminal states)
- T10 env + Docker integration (MULTIPART_* + concurrency limiter)
- T11 README (498 lines) + OpenAPI 3.0 spec (1588 lines)

Tests: 630 pass across 29 suites. Updated Dockerfile + .dockerignore +
docker-compose.yml env passthrough (no hardcoded secrets, fail-fast on
missing required vars).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

2026-05-01 10:55:05 +08:00

jim800121chen

efa67d59a4

Add web frontend, MinIO storage, monitoring, and docker-compose deployment

- Frontend: rewrite Home.vue to match backend POST /jobs API (remove single-stage options)
- Frontend: add Monitor page (/monitor) for queue and job monitoring
- Frontend: add job history with localStorage tracking (per-browser)
- Frontend: fix Nginx proxy rewrite (/api -> /) and add 500MB upload limit
- Backend: add MinIO storage support (STORAGE_BACKEND=minio) alongside local mode
- Backend: add GET /queues/stats API for queue monitoring
- Backend: fix download handler for MinIO (buffer mode for Node 18 compat)
- Workers: add S3/MinIO download/upload in consumer.py with isolated temp dirs
- Workers: add s3_storage.py helper with lifecycle rule (7-day TTL)
- Docker: add docker-compose.yml with all services (web, scheduler, redis, workers)
- Docker: ports mapped to 9500 (web) and 9501 (scheduler)
- Config: add .env to .gitignore to protect secrets

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

2026-04-01 15:04:09 +08:00

3 Commits