jim800121chen d8a9517c9d feat(task-scheduler): Phase 0.8b — API key auth + /result endpoint
Auth pillar 從 OAuth 2.0 resource server 改成 pre-shared API key
(visionA ↔ converter 1:1 internal trust)。新增 GET /api/v1/jobs/:id/result
streaming endpoint 給 visionA backend 中轉 NEF 下載。

Phase A(auth 切換):
- 新增 apiKeyMiddleware(constant-time compare、tokenFingerprint、4 audit events)
- 砍 OAuth middleware + JWKS(保留 oauthClient 供 promote → FAA 使用)
- 4 個 endpoint 換掛 requireApiKey
- 加 TRUST_PROXY env + Express trust proxy 設定(forensic source_ip)

Phase B(/result endpoint):
- streaming NEF download with 5min timeout + concurrent cap 10
- Two-tier rate limit(burst 5/10s + sustained 20/min)
- Bandwidth quota(1 GB/hr + 6 GB/24hr)by token_fingerprint
- Range header silently ignored + Accept-Ranges: none
- filename quote-escape + RFC 5987 fallback + sanitize
- 8 個 /result audit events(forensic 完整)

設計演進記錄:docs/TODO-visionA-integration-v2.md(5/2 OAuth → 5/16 API key
→ 5/16 download via converter;對應 visionA repo ADR-015/016)

Tests: 597 → 666 (+69)、29 suites all pass
Security: APPROVE WITH CONDITIONS(單 instance 部署、6 新 env、24hr 監控)
npm audit: 3 vuln → 0(transitive AWS SDK xml chain)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-17 22:47:28 +08:00

296 lines
8.6 KiB
Markdown
Raw Permalink Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

# API: `/api/v1/jobs`POST / GET / GET :id
> **狀態**Phase 1 完工OAuth→ Phase 0.8b 換 authAPI key其他流程不變。
>
> **配套**`auth.md`、`api/api-result.md`、`api/api-promote.md`、`database.md`。
---
## 1. 通用約定
- **Base URL**`https://<converter-host>/api/v1`
- **Content-Type**
- `POST /api/v1/jobs``multipart/form-data`
- 其他 GETresponse 為 `application/json; charset=utf-8`
- **時間格式**ISO 8601 UTC
- **ID 格式**`job_id` UUIDv4
- **認證**`Authorization: Bearer <CONVERTER_API_KEY>`(除 `/health` 外全部必要)
- **Request ID**:若 client 傳 `X-Request-Id`,回應帶同一值;未傳則 server 產 UUIDv4
- **速率限制**per `client_id`API key 模式下固定 `visionA-service`300 req / 5min
---
## 2. 統一錯誤格式
```json
{
"error": {
"code": "string_code",
"message": "human readable message (zh-TW)",
"details": { /* 可選 */ },
"request_id": "uuid-v4"
}
}
```
---
## 3. `GET /health`(不需 auth
**Response 200**
```json
{
"service": "kneron-converter-api",
"status": "healthy",
"version": "1.0.0",
"timestamp": "2026-05-16T12:00:00Z",
"dependencies": {
"redis": "connected",
"file_access_agent": "reachable"
}
}
```
**Phase 0.8b 變動**:移除 `member_center` dependency不再驗 MC token保留 `file_access_agent`promote 用)+ `redis`
**Response 503**:任一 critical dependency 失敗。
---
## 4. `POST /api/v1/jobs`
### 4.1 Request
```http
POST /api/v1/jobs
Authorization: Bearer <CONVERTER_API_KEY>
Content-Type: multipart/form-data; boundary=----...
X-Request-Id: <uuid> (optional)
------...
Content-Disposition: form-data; name="model"; filename="model.onnx"
Content-Type: application/octet-stream
<binary model file>
------...
Content-Disposition: form-data; name="ref_images[]"; filename="img_0.jpg"
Content-Type: image/jpeg
<binary image>
------...
Content-Disposition: form-data; name="user_id"
visionA-user-12345
------...
Content-Disposition: form-data; name="model_id"
1001
------...
Content-Disposition: form-data; name="version"
0001
------...
Content-Disposition: form-data; name="platform"
520
------...--
```
### 4.2 Multer 設定
- `multer.memoryStorage()`
- `limits.fileSize`: 500MB`MULTIPART_MODEL_MAX_BYTES` env 可覆寫)
- `fields`: `model`1 個 file`ref_images[]``maxCount: 100`
### 4.3 欄位定義
| 欄位 | 類型 | 位置 | 必填 | 驗證 |
|------|------|------|------|------|
| `model` | file | multipart file | ✅ | 副檔名 ∈ {`.onnx`, `.pt`, `.pth`, `.tflite`, `.h5`, `.pb`};大小 ≤ 500MB |
| `ref_images[]` | file[] | multipart file | ❌ | `image/*`;最多 100 張;單張 ≤ 10MB |
| `user_id` | string | multipart field | ✅ | 1-128 字元,嚴格白名單 `^[A-Za-z0-9._-]+$`,不含 `..` |
| `model_id` | string → int | multipart field | ✅ | 轉 int 後 1 ≤ x ≤ 65535 |
| `version` | string | multipart field | ✅ | 1-32 字元,嚴格白名單 `^[A-Za-z0-9._-]+$` |
| `platform` | string | multipart field | ✅ | enum: `520`, `720`, `530`, `630`, `730` |
| `enable_evaluate` | string `'true'`/`'false'` | multipart field | ❌ | 預設 `'false'` |
| `enable_sim_fp` | string `'true'`/`'false'` | multipart field | ❌ | 預設 `'false'` |
| `enable_sim_fixed` | string `'true'`/`'false'` | multipart field | ❌ | 預設 `'false'` |
| `enable_sim_hw` | string `'true'`/`'false'` | multipart field | ❌ | 預設 `'false'` |
| `metadata` | stringJSON| multipart field | ❌ | 合法 JSON 物件字串 |
### 4.4 Middleware 順序(**勿改**
```
requireApiKey()
perClientLimiterper client_id rate limit
uploadConcurrencySemaphoreper-process MAX_CONCURRENT_UPLOADS
uploader.fields([{ name: 'model', maxCount: 1 }, { name: 'ref_images[]', maxCount: 100 }])
multerErrorAdapter捕 multer LIMIT_FILE_SIZE → 413
createJobHandler
```
**理由**API key middleware 必須在 multer 之前(避免未驗證就 parse 500MB 大檔rate limiter 第二(超 quota 不該吃 multipartupload concurrency semaphore 第三(防 OOMmulter 最後auth + quota + concurrency 三重通過後才 parse
### 4.5 Response 201 Created
```json
{
"job_id": "550e8400-e29b-41d4-a716-446655440000",
"status": "created",
"stage": "onnx",
"progress": 0,
"created_at": "2026-05-16T12:00:00Z",
"expires_at": "2026-05-23T12:00:00Z",
"user_id": "visionA-user-12345"
}
```
### 4.6 Error responses
| HTTP | error.code | 情境 |
|------|-----------|------|
| 400 | `validation_error` | 欄位缺漏或格式錯誤 |
| 400 | `invalid_multipart` | multipart parse 失敗、缺必要 file、副檔名不符 |
| 401 | `invalid_token` | API key 缺 / 不符 |
| 409 | `user_has_active_job` | user_id 已有進行中 job |
| 413 | `file_too_large` | model 檔超過 500MB |
| 500 | `misconfiguration` | `STORAGE_BACKEND !== 'minio'` |
| 502 | `storage_unavailable` | MinIO 寫入失敗 |
| 503 | `service_unavailable` | upload concurrency semaphore 滿、API key 未配置 |
| 503 | `service_unavailable` | upload semaphore 滿(含 `Retry-After` header |
---
## 5. `GET /api/v1/jobs/:id`
### 5.1 Request
```http
GET /api/v1/jobs/550e8400-...
Authorization: Bearer <CONVERTER_API_KEY>
If-None-Match: "etag-value" (optional)
```
### 5.2 Response 200
```json
{
"job_id": "550e8400-...",
"user_id": "visionA-user-12345",
"status": "running",
"stage": "bie",
"progress": 45,
"stage_progress": 60,
"created_at": "...",
"updated_at": "...",
"expires_at": "...",
"stage_timings": {
"onnx": { "started_at": "...", "completed_at": "..." },
"bie": { "started_at": "...", "completed_at": null },
"nef": null
},
"input": {
"filename": "model.onnx",
"object_key": "jobs/.../input/model.onnx",
"size_bytes": 204800000,
"ref_images_count": 0
},
"result_object_keys": null,
"error": null,
"parameters": { /* model_id, version, platform, enable_* */ },
"metadata": {},
"estimated_completion_at": null
}
```
### 5.3 狀態機(對外 `status` 欄位)
- `created` — 剛建立,等第一階段開工
- `running` — 正在某個 stage`stage` 欄位有值)
- `completed` — 全部完成(`result_object_keys` 有值,`stage=null`
- `failed` — 失敗(`error` 有值)
**內部 → 對外映射**statusMapper
| 內部 status | 對外 `status` + `stage` |
|------------|----------------------|
| `ONNX` + `stage_timings.onnx.started_at == null` | `created` + stage=onnx |
| `ONNX` + `started_at != null` | `running` + stage=onnx |
| `BIE` | `running` + stage=bie |
| `NEF` | `running` + stage=nef |
| `COMPLETED` | `completed` + stage=null |
| `FAILED` | `failed` + stage=<最後階段> |
### 5.4 ETag 支援
ETag = hash(`job.updated_at`)。If-None-Match 吻合 → 304 Not Modified省 body
### 5.5 Error responses
| HTTP | error.code | 情境 |
|------|-----------|------|
| 401 | `invalid_token` | API key 缺 / 不符 |
| 404 | `job_not_found` | job 不存在 / 不屬於 client避免資訊洩露|
---
## 6. `GET /api/v1/jobs`(列表 / Recovery
### 6.1 Query 參數
| 參數 | 類型 | 必填 | 說明 |
|------|------|------|------|
| `user_id` | string | **✅** | 過濾 user_id強制必填避免全掃|
| `status` | string | ❌ | `in_progress` (= `created` `running`) / `completed` / `failed` / `all`(預設 `all`|
| `limit` | int | ❌ | 預設 20上限 100 |
| `offset` | int | ❌ | 預設 0 |
| `created_after` | ISO 8601 | ❌ | 過濾 `created_at >= created_after` |
### 6.2 Response 200
```json
{
"total": 2,
"limit": 20,
"offset": 0,
"items": [
{ /* GET /jobs/:id 格式,精簡版 */ }
]
}
```
### 6.3 實作
`user:{user_id}:jobs` Set 為索引,避免全掃 `KEYS job:*`
---
## 7. Phase 2 預留端點Phase 1 + 0.8b 回 501
| 方法 | 路徑 | 說明 |
|------|------|------|
| POST | `/api/v1/jobs/:id/download-tokens` | Phase 2 預留(未來 browser 直連 download 用)|
| DELETE | `/api/v1/jobs/:id` | 取消 jobPhase 2/3|
兩個都回 501 `not_implemented`
---
## 8. 端點清單總表
| 方法 | 路徑 | 說明 | Auth |
|------|------|------|------|
| GET | `/health` | 健康檢查 | — |
| POST | `/api/v1/jobs` | 建立轉檔 job | API key |
| GET | `/api/v1/jobs` | 列出 jobuser_id 必填)| API key |
| GET | `/api/v1/jobs/:id` | 單一 job 狀態 | API key |
| POST | `/api/v1/jobs/:id/promote` | 搬檔到 FAA | API key |
| GET | `/api/v1/jobs/:id/result` | **NEW** stream NEF | API key |
| POST | `/api/v1/jobs/:id/download-tokens` | Phase 2回 501 | API key |
| DELETE | `/api/v1/jobs/:id` | Phase 2回 501 | API key |