Compare commits

...

2 Commits

Author SHA1 Message Date
warrenchen
7647a8cb3b feat: Add initial database migration for SendEngine with new tables and relationships
- Created migration file for rebaseline of the database schema.
- Added tables: auth_clients, tenants, auth_client_keys, webhook_nonces, events_inbox, lists, campaigns, subscriptions, send_jobs, delivery_summary, and send_batches.
- Defined relationships and constraints between tables.
- Updated DbContext and model snapshot to reflect new entities and their configurations.
- Removed deprecated ListMember entity and its references.
- Introduced Dockerfile for building and running the SendEngine application.
- Enhanced installer program to support tenant creation and webhook client management with Member Center integration.
2026-02-19 17:21:06 +09:00
warrenchen
a7752c8ce0 feat: Enhance SES webhook handling and add support for soft bounce thresholds 2026-02-18 12:56:54 +09:00
21 changed files with 1432 additions and 408 deletions

View File

@ -10,3 +10,14 @@ Webhook__Secrets__member_center=change_me_webhook_secret
Webhook__TimestampSkewSeconds=300 Webhook__TimestampSkewSeconds=300
Webhook__AllowNullTenantClient=false Webhook__AllowNullTenantClient=false
Ses__SkipSignatureValidation=true Ses__SkipSignatureValidation=true
Bounce__SoftBounceThreshold=5
MemberCenter__BaseUrl=
MemberCenter__DisableSubscriptionPath=/subscriptions/disable
MemberCenter__TokenPath=/oauth/token
MemberCenter__DisableSubscriptionUrl=
MemberCenter__TokenUrl=
MemberCenter__ClientId=
MemberCenter__ClientSecret=
MemberCenter__Scope=newsletter:events.write
MemberCenter__ApiToken=
TestFriendly__Enabled=false

View File

@ -6,3 +6,5 @@
- If the command fails or hangs due to sandbox limits (for example restore/build stalls), rerun with `sandbox_permissions: "require_escalated"`. - If the command fails or hangs due to sandbox limits (for example restore/build stalls), rerun with `sandbox_permissions: "require_escalated"`.
- The escalation request must include a short justification explaining that sandbox restrictions are blocking normal .NET execution. - The escalation request must include a short justification explaining that sandbox restrictions are blocking normal .NET execution.
- Do not change project paths or command intent when escalating; rerun the same command with elevated permissions. - Do not change project paths or command intent when escalating; rerun the same command with elevated permissions.
- If a sandbox command appears hung, run it with `tty=true` and explicitly terminate the sandbox session first (send `Ctrl+C`) before escalating.
- Never leave a hung sandbox process running in the background while starting the escalated rerun.

View File

@ -20,6 +20,7 @@
- Campaign / Send Job 基本流程 - Campaign / Send Job 基本流程
- Sender Adapter至少一種 ESP - Sender Adapter至少一種 ESP
- 投遞與退信記錄 - 投遞與退信記錄
- SES 回流建議採 `Configuration Set -> SNS -> SQS -> Worker`
## 非目標(暫不處理) ## 非目標(暫不處理)
- 自建 SMTP server - 自建 SMTP server

View File

@ -11,6 +11,44 @@ ESP 介接暫定為 Amazon SES。
- Sender Adapter - Sender Adapter
- Delivery & Bounce Handling - Delivery & Bounce Handling
## Sending Proxy 規格整合
### 目標與邊界
- 接收內容網站或會員平台送來的發送工作
- 呼叫 Amazon SES API優先 SES v2
- 必帶 Configuration Set + Message Tags
- 消費 SES 回流事件Bounce / Complaint / Delivery
- 必要時回寫 Member Center
不負責:
- List-Unsubscribe one-click endpoint 本身的服務實作
- 會員最終名單權威資料庫
### 狀態機(規劃)
Job 狀態:
- queued
- sending
- sent
- partially_failed
- failed
- completed
Recipient 狀態:
- pending
- sent
- delivered
- soft_bounced
- hard_bounced
- complained
- suppressed
### SES 事件回流架構(建議)
- `SES Configuration Set -> SNS Topic -> SQS Queue -> ECS Worker`
- Worker 職責:
- Poll SQS
- 解析 SNS envelope 與 SES payload
- 更新 DB 狀態
- 必要時呼叫 Member Center 停用/註記 API
## 信任邊界與 Auth 模型 ## 信任邊界與 Auth 模型
### 外部角色 ### 外部角色
- Member Center事件來源與名單權威來源authority - Member Center事件來源與名單權威來源authority

View File

@ -48,14 +48,19 @@ Member Center 為多租戶架構,信件內容由各租戶網站產生後送入
- 選填排程時間、發送窗口、追蹤設定open/click - 選填排程時間、發送窗口、追蹤設定open/click
2. Send Engine 驗證 tenant scope 與內容完整性,建立 Send Job。 2. Send Engine 驗證 tenant scope 與內容完整性,建立 Send Job。
- tenant_id 以 token 為準body 的 tenant_id 僅作一致性檢查 - tenant_id 以 token 為準body 的 tenant_id 僅作一致性檢查
- list_id 必須屬於 tenant scope - tenant 必須預先存在(建議由 Installer 建立)
- list_id 若不存在Send Engine 會在該 tenant 下自動建立 listplaceholder
3. Scheduler 在排程時間點啟動 Send Job 3. Scheduler 在排程時間點啟動 Send Job
- 讀取 List Store 快照 - 讀取 List Store 快照
- 依規則過濾已退訂、bounced、黑名單 - 依規則過濾已退訂、bounced、黑名單
4. 切分成可控批次batch寫入 Outbox。 4. 切分成可控批次batch寫入 Outbox。
5. Sender Worker 取出 batch轉成 SES API 請求。 5. Sender Worker 取出 batch轉成 SES API 請求。
6. SES 回應 message_id → 記錄 delivery log。 6. 發送時必帶:
7. 更新 Send Job 進度(成功/失敗/重試)。 - SES Configuration Set
- Message Tags至少含 campaign_id / site_id / list_id
- Newsletter 類型需帶 `List-Unsubscribe``List-Unsubscribe-Post` headers
7. SES 回應 message_id → 記錄 delivery log。
8. 更新 Send Job 進度(成功/失敗/重試)。
控速策略(範例): 控速策略(範例):
- 全域 TPS 上限 + tenant TPS 上限 - 全域 TPS 上限 + tenant TPS 上限
@ -70,19 +75,36 @@ Member Center 為多租戶架構,信件內容由各租戶網站產生後送入
目的:處理 ESP 回報的 bounce/complaint並回寫本地名單狀態。 目的:處理 ESP 回報的 bounce/complaint並回寫本地名單狀態。
流程: 流程:
1. SES 透過 SNS/Webhook 回報事件bounce/complaint/delivery/open/click 1. SES 事件由 Configuration Set 發送至 SNS再落到 SQS
2. Webhook 驗證簽章與來源SES/SNS 驗證) 2. ECS Worker 輪詢 SQS解析 SNS envelope 與 SES payload
3. 將事件寫入 Inboxappend-only 3. 將事件寫入 Inboxappend-only
4. Consumer 解析事件: 4. Consumer 解析事件:
- hard bounce → 標記 bounced + 停用 - hard bounce → 立即標記 blacklisted同義於 `suppressed`
- soft bounce → 記錄次數,超過門檻停用 - soft bounce → 累計次數,達門檻(預設 5才標記 blacklisted`suppressed`
- complaint → 立即停用並列入黑名單 - complaint → 立即取消訂閱並標記 blacklisted`suppressed`
- suppression 事件 → 直接對應為 `suppressed`(即黑名單)
5. 更新 List Store 快照與投遞記錄。 5. 更新 List Store 快照與投遞記錄。
6. 回寫 Member Center 以停用訂閱(例如 hard bounce / complaint 6. 回寫 Member Center僅在以下條件
- hard bounce已設黑名單
- soft bounce達門檻後設黑名單
- complaint取消訂閱
- suppression設黑名單
補充:
- Unknown event 不應使 worker crash應記錄後送入 DLQ
- Throttle/暫時性網路錯誤使用指數退避重試
回寫規則: 回寫規則:
- Send Engine 僅回寫「停用原因」與必要欄位 - Send Engine 僅回寫「停用原因」與必要欄位
- Member Center 需提供可標註來源的欄位(例如 `disabled_by=send_engine` - Member Center 需提供可標註來源的欄位(例如 `disabled_by=send_engine`
- 回寫原因碼固定為:
- `hard_bounce`
- `soft_bounce_threshold`
- `complaint`
- `suppression`
名詞定義:
- `blacklisted``suppressed` 同義,表示此收件者不可再發送
資料一致性: 資料一致性:
- 任何狀態改變需保留歷史append-only events + current snapshot - 任何狀態改變需保留歷史append-only events + current snapshot

View File

@ -7,10 +7,33 @@
- 需要關閉時請設定 `Db__AutoMigrate=false` - 需要關閉時請設定 `Db__AutoMigrate=false`
- 手動執行可用 `dotnet run --project src/SendEngine.Installer -- migrate` - 手動執行可用 `dotnet run --project src/SendEngine.Installer -- migrate`
- Webhook Auth 初始化(不使用 SQL 檔,改用 Installer - Webhook Auth 初始化(不使用 SQL 檔,改用 Installer
- 若僅需先建立 tenant 基本資料:
- `dotnet run --project src/SendEngine.Installer -- ensure-tenant --tenant-id <tenant_uuid> [--tenant-name <name>]`
- 使用 Installer 建立 webhook client`id` 自動隨機產生): - 使用 Installer 建立 webhook client`id` 自動隨機產生):
- `dotnet run --project src/SendEngine.Installer -- add-webhook-client --tenant-id <tenant_uuid> --client-id <client_id> --name <display_name> --scopes <scope1,scope2>` - `dotnet run --project src/SendEngine.Installer -- add-webhook-client --tenant-id <tenant_uuid> [--tenant-name <name>] --client-id <client_id> --name <display_name> --scopes <scope1,scope2>`
- 例如:`dotnet run --project src/SendEngine.Installer -- add-webhook-client --tenant-id 11111111-1111-1111-1111-111111111111 --client-id member-center-webhook --name "Member Center Webhook" --scopes newsletter:events.write` - 例如:`dotnet run --project src/SendEngine.Installer -- add-webhook-client --tenant-id 11111111-1111-1111-1111-111111111111 --tenant-name "Tenant A" --client-id member-center-webhook --name "Member Center Webhook" --scopes newsletter:events.write`
- 若 tenant 不存在Installer 會先自動建立 `tenants` 基本資料,避免 webhook 出現 `tenant_not_found`
- 建立成功後Member Center webhook header `X-Client-Id` 請帶回傳的 `id` - 建立成功後Member Center webhook header `X-Client-Id` 請帶回傳的 `id`
- 若要自動同步到 Member Center `POST /integrations/send-engine/webhook-clients/upsert`(保留原手動流程):
- `dotnet run --project src/SendEngine.Installer -- add-webhook-client --tenant-id <tenant_uuid> --client-id <client_id> --name <display_name> --scopes <scope1,scope2> --upsert-member-center --mc-base-url <member_center_base_url> --mc-client-id <oauth_client_id> --mc-client-secret <oauth_client_secret> --mc-scope newsletter:events.write.global`
- 可選參數:
- `--mc-token-path`(預設 `/oauth/token`
- `--mc-upsert-path`(預設 `/integrations/send-engine/webhook-clients/upsert`
- `--mc-token-url` / `--mc-upsert-url`(使用完整 URL 時可覆蓋 path 組合)
- Webhook 驗證規則為 tenant 綁定:`auth_clients.tenant_id` 必須等於 payload `tenant_id` - Webhook 驗證規則為 tenant 綁定:`auth_clients.tenant_id` 必須等於 payload `tenant_id`
- 不支援 `X-Client-Id` fallback - 不支援 `X-Client-Id` fallback
- 預設拒絕 `tenant_id = NULL` 的通用 client`Webhook__AllowNullTenantClient=false` - 預設拒絕 `tenant_id = NULL` 的通用 client`Webhook__AllowNullTenantClient=false`
- Member Center 回寫授權(建議):
- `MemberCenter__BaseUrl`(建議)
- `MemberCenter__DisableSubscriptionPath`(預設 `/subscriptions/disable`
- `MemberCenter__TokenPath`(預設 `/oauth/token`
- `MemberCenter__ClientId`
- `MemberCenter__ClientSecret`
- `MemberCenter__Scope=newsletter:events.write`
- `MemberCenter__DisableSubscriptionUrl``MemberCenter__TokenUrl` 可用完整 URL 覆蓋fallback
- `MemberCenter__ApiToken` 僅作暫時 fallback非首選
- 本機測試輔助(臨時):
- `TestFriendly__Enabled=true` 時:
- webhook 收到未知 tenant 會自動建立 tenant
- `/webhooks/ses` 不做任何 DB 存取(僅用於測試流程打通)
- 正式環境建議維持 `false`

View File

@ -40,6 +40,10 @@ scope 最小化:
- `newsletter:events.write`(停用回寫) - `newsletter:events.write`(停用回寫)
- `newsletter:list.read`(若未來仍需查詢) - `newsletter:list.read`(若未來仍需查詢)
實作約定:
- 優先走 token endpointclient credentials
- `ApiToken` 僅作暫時 fallback建議逐步移除
## 通用欄位 ## 通用欄位
### Timestamp ### Timestamp
- 欄位:`occurred_at` - 欄位:`occurred_at`
@ -157,6 +161,8 @@ Request Body
- `subject` 必填,最小長度 1 - `subject` 必填,最小長度 1
- `body_html` / `body_text` / `template` 至少擇一 - `body_html` / `body_text` / `template` 至少擇一
- `window_start` 必須小於 `window_end`(若有提供) - `window_start` 必須小於 `window_end`(若有提供)
- `tenant_id` 必須已存在(不存在回 `422 tenant_not_found`
- `list_id` 若不存在,會在該 tenant 下自動建立 placeholder list 後建立 send job
Response Response
```json ```json
@ -166,6 +172,35 @@ Response
} }
``` ```
### C-1. Sending Proxy Submit Job整合規格
用途:對齊內容網站/會員平台呼叫發信代理的標準接口。
Endpoint
- `POST /v1/send-jobs`
Request Body欄位
- `message_type``newsletter` | `transactional`
- `from`:發件人
- `to`:收件人陣列
- `subject`:主旨
- `html`HTML 內容
- `text`:純文字內容
- `headers`:自定義 header白名單
- `list_unsubscribe.url`:退訂 URL
- `list_unsubscribe.mailto`:可選
- `tags.campaign_id` / `tags.site_id` / `tags.list_id` / `tags.segment`
- `idempotency_key`:冪等鍵
Response
- `job_id`
- `status=queued`
規則:
- 必須帶 Configuration Set + Message Tags 後才能呼叫 SES
- `newsletter` 類型需帶:
- `List-Unsubscribe`
- `List-Unsubscribe-Post: List-Unsubscribe=One-Click`
### D. 查詢 Send Job ### D. 查詢 Send Job
Endpoint Endpoint
- `GET /api/send-jobs/{id}` - `GET /api/send-jobs/{id}`
@ -204,9 +239,13 @@ Response
## WebhookSES → Send Engine ## WebhookSES → Send Engine
### F. SES 事件回報 ### F. SES 事件回報
用途:接收 bounce/complaint/delivery/open/click 等事件。 用途:接收 bounce/hard_bounced/soft_bounced/complaint/suppression/delivery/open/click 等事件。
Endpoint 推薦架構(正式):
- `SES Configuration Set -> SNS -> SQS -> ECS Worker`
- 由 Worker 消費事件,不要求對外公開 webhook
相容模式(可選):
- `POST /webhooks/ses` - `POST /webhooks/ses`
驗證: 驗證:
@ -227,6 +266,24 @@ Request Body示意
Response Response
- `200 OK` - `200 OK`
事件對應規則(固定):
- `hard_bounced`:立即設為黑名單(`suppressed`
- `soft_bounced`:累計達門檻後設為黑名單(`suppressed`
- `complaint`:取消訂閱並回寫 Member Center
- `suppression`:設為黑名單(`suppressed`
回寫 Member Center 條件:
- `hard_bounced`:設黑名單後回寫
- `soft_bounced`:達門檻設黑名單後回寫
- `complaint`:立即回寫
- `suppression`:設黑名單後回寫
回寫原因碼(固定):
- `hard_bounce`
- `soft_bounce_threshold`
- `complaint`
- `suppression`
## 外部 APISend Engine → Member Center ## 外部 APISend Engine → Member Center
以下為 Member Center 端提供的 API非 Send Engine 的 OpenAPI 規格範圍。 以下為 Member Center 端提供的 API非 Send Engine 的 OpenAPI 規格範圍。
@ -234,7 +291,7 @@ Response
用途:因 hard bounce / complaint 停用訂閱,並在 Member Center 註記來源。 用途:因 hard bounce / complaint 停用訂閱,並在 Member Center 註記來源。
EndpointMember Center 提供): EndpointMember Center 提供):
- `POST /api/subscriptions/disable` - `POST /subscriptions/disable`
Scope Scope
- `newsletter:events.write` - `newsletter:events.write`
@ -257,3 +314,12 @@ Request Body示意
- `409`重放或事件重複nonce / event_id - `409`重放或事件重複nonce / event_id
- `422`:資料格式錯誤 - `422`:資料格式錯誤
- `500`:伺服器內部錯誤 - `500`:伺服器內部錯誤
## Retry 策略(整合規格)
- Throttle指數退避重試
- Temporary network error重試
- Hard failure不重試
- Retry 上限可設定(例如 5 次)
## 相關環境參數
- `Bounce__SoftBounceThreshold`soft bounce 轉黑名單門檻(預設 `5`

View File

@ -25,49 +25,25 @@ ALTER TABLE lists
ADD CONSTRAINT fk_lists_tenant ADD CONSTRAINT fk_lists_tenant
FOREIGN KEY (tenant_id) REFERENCES tenants(id); FOREIGN KEY (tenant_id) REFERENCES tenants(id);
-- Subscribers (per tenant) -- List subscriptions (per list, keyed by email)
CREATE TABLE IF NOT EXISTS subscribers ( CREATE TABLE IF NOT EXISTS subscriptions (
id UUID PRIMARY KEY, id UUID PRIMARY KEY,
tenant_id UUID NOT NULL, list_id UUID NOT NULL,
email CITEXT NOT NULL, email CITEXT NOT NULL,
external_subscriber_id UUID, -- Member Center subscriber_id
status TEXT NOT NULL, -- active/unsubscribed/bounced/complaint status TEXT NOT NULL, -- active/unsubscribed/bounced/complaint
preferences JSONB, preferences JSONB,
created_at TIMESTAMPTZ NOT NULL DEFAULT now(), created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT now(), updated_at TIMESTAMPTZ NOT NULL DEFAULT now(),
UNIQUE (tenant_id, email) UNIQUE (list_id, email)
); );
CREATE INDEX IF NOT EXISTS idx_subscribers_tenant ON subscribers(tenant_id); CREATE INDEX IF NOT EXISTS idx_subscriptions_list ON subscriptions(list_id);
CREATE INDEX IF NOT EXISTS idx_subscriptions_external_subscriber ON subscriptions(external_subscriber_id);
ALTER TABLE subscribers ALTER TABLE subscriptions
ADD CONSTRAINT fk_subscribers_tenant ADD CONSTRAINT fk_subscriptions_list
FOREIGN KEY (tenant_id) REFERENCES tenants(id);
-- List membership snapshot
CREATE TABLE IF NOT EXISTS list_members (
id UUID PRIMARY KEY,
tenant_id UUID NOT NULL,
list_id UUID NOT NULL,
subscriber_id UUID NOT NULL,
status TEXT NOT NULL, -- active/unsubscribed/bounced/complaint
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT now(),
UNIQUE (tenant_id, list_id, subscriber_id)
);
CREATE INDEX IF NOT EXISTS idx_list_members_tenant ON list_members(tenant_id);
CREATE INDEX IF NOT EXISTS idx_list_members_list ON list_members(list_id);
ALTER TABLE list_members
ADD CONSTRAINT fk_list_members_tenant
FOREIGN KEY (tenant_id) REFERENCES tenants(id);
ALTER TABLE list_members
ADD CONSTRAINT fk_list_members_list
FOREIGN KEY (list_id) REFERENCES lists(id); FOREIGN KEY (list_id) REFERENCES lists(id);
ALTER TABLE list_members
ADD CONSTRAINT fk_list_members_subscriber
FOREIGN KEY (subscriber_id) REFERENCES subscribers(id);
-- Event inbox (append-only) -- Event inbox (append-only)
CREATE TABLE IF NOT EXISTS events_inbox ( CREATE TABLE IF NOT EXISTS events_inbox (
id UUID PRIMARY KEY, id UUID PRIMARY KEY,

View File

@ -11,9 +11,58 @@ security:
- bearerAuth: [] - bearerAuth: []
paths: paths:
/v1/send-jobs:
post:
summary: Submit send job (sending proxy)
security:
- bearerAuth: []
requestBody:
required: true
content:
application/json:
schema:
$ref: '#/components/schemas/SubmitSendJobRequest'
responses:
'200':
description: Queued
content:
application/json:
schema:
$ref: '#/components/schemas/SubmitSendJobResponse'
'401':
description: Unauthorized
content:
application/json:
schema:
$ref: '#/components/schemas/ErrorResponse'
'403':
description: Forbidden
content:
application/json:
schema:
$ref: '#/components/schemas/ErrorResponse'
'422':
description: Unprocessable Entity (e.g. tenant not found)
content:
application/json:
schema:
$ref: '#/components/schemas/ErrorResponse'
'409':
description: Idempotency conflict
content:
application/json:
schema:
$ref: '#/components/schemas/ErrorResponse'
'422':
description: Validation error
content:
application/json:
schema:
$ref: '#/components/schemas/ErrorResponse'
/api/send-jobs: /api/send-jobs:
post: post:
summary: Create send job summary: Create send job (legacy/internal)
security: security:
- bearerAuth: [] - bearerAuth: []
requestBody: requestBody:
@ -356,6 +405,76 @@ components:
type: string type: string
enum: [pending, running, completed, failed, cancelled] enum: [pending, running, completed, failed, cancelled]
SubmitSendJobRequest:
type: object
required: [message_type, from, to, subject, html, text, tags, idempotency_key]
properties:
message_type:
type: string
enum: [newsletter, transactional]
from:
type: string
format: email
to:
type: array
items:
type: string
format: email
minItems: 1
subject:
type: string
minLength: 1
html:
type: string
text:
type: string
headers:
type: object
additionalProperties:
type: string
list_unsubscribe:
$ref: '#/components/schemas/ListUnsubscribe'
tags:
$ref: '#/components/schemas/MessageTags'
idempotency_key:
type: string
minLength: 1
SubmitSendJobResponse:
type: object
required: [job_id, status]
properties:
job_id:
type: string
format: uuid
status:
type: string
enum: [queued]
ListUnsubscribe:
type: object
required: [url]
properties:
url:
type: string
format: uri
mailto:
type: string
format: email
MessageTags:
type: object
required: [campaign_id, site_id, list_id, segment]
properties:
campaign_id:
type: string
site_id:
type: string
list_id:
type: string
segment:
type: string
TrackingOptions: TrackingOptions:
type: object type: object
properties: properties:
@ -398,7 +517,7 @@ components:
format: email format: email
status: status:
type: string type: string
enum: [active, unsubscribed, bounced, complaint] enum: [active, unsubscribed, bounced, complaint, suppressed]
preferences: preferences:
type: object type: object
additionalProperties: true additionalProperties: true
@ -436,7 +555,7 @@ components:
properties: properties:
event_type: event_type:
type: string type: string
enum: [bounce, complaint, delivery, open, click] enum: [bounce, hard_bounced, soft_bounced, complaint, suppression, delivery, open, click]
message_id: message_id:
type: string type: string
tenant_id: tenant_id:

View File

@ -1,40 +1,87 @@
using System.Text.Json.Serialization;
namespace SendEngine.Api.Models; namespace SendEngine.Api.Models;
public sealed class SubscriptionEventRequest public sealed class SubscriptionEventRequest
{ {
[JsonPropertyName("event_id")]
public Guid EventId { get; set; } public Guid EventId { get; set; }
[JsonPropertyName("event_type")]
public string EventType { get; set; } = string.Empty; public string EventType { get; set; } = string.Empty;
[JsonPropertyName("tenant_id")]
public Guid TenantId { get; set; } public Guid TenantId { get; set; }
[JsonPropertyName("list_id")]
public Guid ListId { get; set; } public Guid ListId { get; set; }
[JsonPropertyName("subscriber")]
public SubscriberPayload Subscriber { get; set; } = new(); public SubscriberPayload Subscriber { get; set; } = new();
[JsonPropertyName("occurred_at")]
public DateTimeOffset OccurredAt { get; set; } public DateTimeOffset OccurredAt { get; set; }
} }
public sealed class SubscriberPayload public sealed class SubscriberPayload
{ {
[JsonPropertyName("id")]
public Guid Id { get; set; } public Guid Id { get; set; }
[JsonPropertyName("email")]
public string Email { get; set; } = string.Empty; public string Email { get; set; } = string.Empty;
[JsonPropertyName("status")]
public string Status { get; set; } = string.Empty; public string Status { get; set; } = string.Empty;
[JsonPropertyName("preferences")]
public Dictionary<string, object>? Preferences { get; set; } public Dictionary<string, object>? Preferences { get; set; }
} }
public sealed class FullSyncBatchRequest public sealed class FullSyncBatchRequest
{ {
[JsonPropertyName("sync_id")]
public Guid SyncId { get; set; } public Guid SyncId { get; set; }
[JsonPropertyName("batch_no")]
public int BatchNo { get; set; } public int BatchNo { get; set; }
[JsonPropertyName("batch_total")]
public int BatchTotal { get; set; } public int BatchTotal { get; set; }
[JsonPropertyName("tenant_id")]
public Guid TenantId { get; set; } public Guid TenantId { get; set; }
[JsonPropertyName("list_id")]
public Guid ListId { get; set; } public Guid ListId { get; set; }
[JsonPropertyName("subscribers")]
public List<SubscriberPayload> Subscribers { get; set; } = new(); public List<SubscriberPayload> Subscribers { get; set; } = new();
[JsonPropertyName("occurred_at")]
public DateTimeOffset OccurredAt { get; set; } public DateTimeOffset OccurredAt { get; set; }
} }
public sealed class SesEventRequest public sealed class SesEventRequest
{ {
[JsonPropertyName("event_type")]
public string EventType { get; set; } = string.Empty; public string EventType { get; set; } = string.Empty;
[JsonPropertyName("message_id")]
public string MessageId { get; set; } = string.Empty; public string MessageId { get; set; } = string.Empty;
[JsonPropertyName("tenant_id")]
public Guid TenantId { get; set; } public Guid TenantId { get; set; }
[JsonPropertyName("email")]
public string Email { get; set; } = string.Empty; public string Email { get; set; } = string.Empty;
[JsonPropertyName("bounce_type")]
public string? BounceType { get; set; } public string? BounceType { get; set; }
[JsonPropertyName("occurred_at")]
public DateTimeOffset OccurredAt { get; set; } public DateTimeOffset OccurredAt { get; set; }
[JsonPropertyName("tags")]
public Dictionary<string, string>? Tags { get; set; }
} }

View File

@ -1,6 +1,7 @@
using System.Security.Claims; using System.Security.Claims;
using System.Text; using System.Text;
using System.Text.Json; using System.Text.Json;
using System.Net.Http.Json;
using Microsoft.AspNetCore.Authentication.JwtBearer; using Microsoft.AspNetCore.Authentication.JwtBearer;
using Microsoft.EntityFrameworkCore; using Microsoft.EntityFrameworkCore;
using Microsoft.IdentityModel.Tokens; using Microsoft.IdentityModel.Tokens;
@ -57,13 +58,32 @@ if (app.Environment.IsDevelopment())
app.UseHttpsRedirection(); app.UseHttpsRedirection();
app.UseAuthentication(); app.UseAuthentication();
app.UseAuthorization(); app.UseAuthorization();
app.Use(async (context, next) =>
{
if (context.Request.Path.StartsWithSegments("/webhooks/subscriptions") ||
context.Request.Path.StartsWithSegments("/webhooks/lists/full-sync"))
{
context.Request.EnableBuffering();
using var reader = new StreamReader(context.Request.Body, Encoding.UTF8, leaveOpen: true);
var rawBody = await reader.ReadToEndAsync();
context.Request.Body.Position = 0;
context.Items[WebhookValidator.RawBodyItemKey] = rawBody;
}
await next();
});
app.MapGet("/health", () => Results.Ok(new { status = "ok" })) app.MapGet("/health", () => Results.Ok(new { status = "ok" }))
.WithName("Health") .WithName("Health")
.WithOpenApi(); .WithOpenApi();
app.MapPost("/api/send-jobs", async (HttpContext httpContext, CreateSendJobRequest request, SendEngineDbContext db) => app.MapPost("/api/send-jobs", async (
HttpContext httpContext,
CreateSendJobRequest request,
SendEngineDbContext db,
ILoggerFactory loggerFactory) =>
{ {
var logger = loggerFactory.CreateLogger("SendEngine.Api.SendJobs");
var tenantId = GetTenantId(httpContext.User); var tenantId = GetTenantId(httpContext.User);
if (tenantId is null) if (tenantId is null)
{ {
@ -104,6 +124,29 @@ app.MapPost("/api/send-jobs", async (HttpContext httpContext, CreateSendJobReque
return Results.UnprocessableEntity(new { error = "window_invalid" }); return Results.UnprocessableEntity(new { error = "window_invalid" });
} }
var tenantExists = await db.Tenants.AsNoTracking().AnyAsync(x => x.Id == request.TenantId);
if (!tenantExists)
{
return Results.UnprocessableEntity(new { error = "tenant_not_found" });
}
var listExists = await db.Lists.AsNoTracking()
.AnyAsync(x => x.Id == request.ListId && x.TenantId == request.TenantId);
if (!listExists)
{
logger.LogWarning(
"CreateSendJob auto-created list because list was missing. tenant_id={TenantId} list_id={ListId}",
request.TenantId,
request.ListId);
db.Lists.Add(new MailingList
{
Id = request.ListId,
TenantId = request.TenantId,
Name = $"list-{request.ListId:N}",
CreatedAt = DateTimeOffset.UtcNow
});
}
var campaign = new Campaign var campaign = new Campaign
{ {
Id = Guid.NewGuid(), Id = Guid.NewGuid(),
@ -191,15 +234,23 @@ app.MapPost("/api/send-jobs/{id:guid}/cancel", async (HttpContext httpContext, G
return Results.Ok(new SendJobStatusResponse { Id = sendJob.Id, Status = sendJob.Status }); return Results.Ok(new SendJobStatusResponse { Id = sendJob.Id, Status = sendJob.Status });
}).RequireAuthorization().WithName("CancelSendJob").WithOpenApi(); }).RequireAuthorization().WithName("CancelSendJob").WithOpenApi();
app.MapPost("/webhooks/subscriptions", async (HttpContext httpContext, SubscriptionEventRequest request, SendEngineDbContext db) => app.MapPost("/webhooks/subscriptions", async (
HttpContext httpContext,
SubscriptionEventRequest request,
SendEngineDbContext db,
ILoggerFactory loggerFactory) =>
{ {
var logger = loggerFactory.CreateLogger("SendEngine.Webhooks.Subscriptions");
if (request.TenantId == Guid.Empty || request.ListId == Guid.Empty || request.Subscriber.Id == Guid.Empty) if (request.TenantId == Guid.Empty || request.ListId == Guid.Empty || request.Subscriber.Id == Guid.Empty)
{ {
logger.LogWarning("Subscription webhook rejected: tenant_id/list_id/subscriber.id required.");
return Results.UnprocessableEntity(new { error = "tenant_id_list_id_subscriber_id_required" }); return Results.UnprocessableEntity(new { error = "tenant_id_list_id_subscriber_id_required" });
} }
if (!IsSupportedSubscriptionEvent(request.EventType)) if (!IsSupportedSubscriptionEvent(request.EventType))
{ {
logger.LogWarning("Subscription webhook rejected: unsupported_event_type={EventType}", request.EventType);
return Results.UnprocessableEntity(new { error = "unsupported_event_type" }); return Results.UnprocessableEntity(new { error = "unsupported_event_type" });
} }
@ -212,12 +263,45 @@ app.MapPost("/webhooks/subscriptions", async (HttpContext httpContext, Subscript
secret, secret,
skewSeconds, skewSeconds,
request.TenantId, request.TenantId,
allowNullTenantClient); allowNullTenantClient,
"newsletter:events.write");
if (validation is not null) if (validation is not null)
{ {
return validation; var reason = httpContext.Items.TryGetValue(WebhookValidator.FailureReasonItemKey, out var value)
? value?.ToString()
: "unknown";
var statusCode = httpContext.Items.TryGetValue(WebhookValidator.FailureStatusCodeItemKey, out var statusValue) &&
statusValue is int code
? code
: StatusCodes.Status401Unauthorized;
logger.LogWarning(
"Subscription webhook auth validation failed. tenant_id={TenantId} event_type={EventType} reason={Reason}",
request.TenantId,
request.EventType,
reason);
return Results.Json(
new
{
error = reason == "replay_detected" ? "replay_detected" : "webhook_auth_failed",
reason
},
statusCode: statusCode);
} }
var testFriendlyEnabled = builder.Configuration.GetValue("TestFriendly:Enabled", false);
if (!await EnsureTenantForWebhookAsync(db, request.TenantId, testFriendlyEnabled))
{
logger.LogWarning("Subscription webhook rejected: tenant_not_found. tenant_id={TenantId}", request.TenantId);
return Results.UnprocessableEntity(new { error = "tenant_not_found" });
}
logger.LogInformation(
"Subscription webhook processing started. tenant_id={TenantId} list_id={ListId} subscriber_id={SubscriberId} event_type={EventType}",
request.TenantId,
request.ListId,
request.Subscriber.Id,
request.EventType);
var payload = JsonSerializer.Serialize(request); var payload = JsonSerializer.Serialize(request);
var inbox = new EventInbox var inbox = new EventInbox
@ -244,16 +328,36 @@ app.MapPost("/webhooks/subscriptions", async (HttpContext httpContext, Subscript
} }
catch (DbUpdateException) catch (DbUpdateException)
{ {
logger.LogWarning(
"Subscription webhook conflict. tenant_id={TenantId} list_id={ListId} subscriber_id={SubscriberId} event_type={EventType}",
request.TenantId,
request.ListId,
request.Subscriber.Id,
request.EventType);
return Results.Conflict(new { error = "duplicate_event_or_unique_violation" }); return Results.Conflict(new { error = "duplicate_event_or_unique_violation" });
} }
logger.LogInformation(
"Subscription webhook processed. tenant_id={TenantId} list_id={ListId} subscriber_id={SubscriberId} event_type={EventType}",
request.TenantId,
request.ListId,
request.Subscriber.Id,
request.EventType);
return Results.Ok(); return Results.Ok();
}).WithName("SubscriptionWebhook").WithOpenApi(); }).WithName("SubscriptionWebhook").WithOpenApi();
app.MapPost("/webhooks/lists/full-sync", async (HttpContext httpContext, FullSyncBatchRequest request, SendEngineDbContext db) => app.MapPost("/webhooks/lists/full-sync", async (
HttpContext httpContext,
FullSyncBatchRequest request,
SendEngineDbContext db,
ILoggerFactory loggerFactory) =>
{ {
var logger = loggerFactory.CreateLogger("SendEngine.Webhooks.FullSync");
if (request.TenantId == Guid.Empty || request.ListId == Guid.Empty) if (request.TenantId == Guid.Empty || request.ListId == Guid.Empty)
{ {
logger.LogWarning("Full-sync webhook rejected: tenant_id/list_id required.");
return Results.UnprocessableEntity(new { error = "tenant_id_list_id_required" }); return Results.UnprocessableEntity(new { error = "tenant_id_list_id_required" });
} }
@ -266,12 +370,48 @@ app.MapPost("/webhooks/lists/full-sync", async (HttpContext httpContext, FullSyn
secret, secret,
skewSeconds, skewSeconds,
request.TenantId, request.TenantId,
allowNullTenantClient); allowNullTenantClient,
"newsletter:events.write");
if (validation is not null) if (validation is not null)
{ {
return validation; var reason = httpContext.Items.TryGetValue(WebhookValidator.FailureReasonItemKey, out var value)
? value?.ToString()
: "unknown";
var statusCode = httpContext.Items.TryGetValue(WebhookValidator.FailureStatusCodeItemKey, out var statusValue) &&
statusValue is int code
? code
: StatusCodes.Status401Unauthorized;
logger.LogWarning(
"Full-sync webhook auth validation failed. tenant_id={TenantId} list_id={ListId} sync_id={SyncId} reason={Reason}",
request.TenantId,
request.ListId,
request.SyncId,
reason);
return Results.Json(
new
{
error = reason == "replay_detected" ? "replay_detected" : "webhook_auth_failed",
reason
},
statusCode: statusCode);
} }
var testFriendlyEnabled = builder.Configuration.GetValue("TestFriendly:Enabled", false);
if (!await EnsureTenantForWebhookAsync(db, request.TenantId, testFriendlyEnabled))
{
logger.LogWarning("Full-sync webhook rejected: tenant_not_found. tenant_id={TenantId}", request.TenantId);
return Results.UnprocessableEntity(new { error = "tenant_not_found" });
}
logger.LogInformation(
"Full-sync webhook processing started. tenant_id={TenantId} list_id={ListId} sync_id={SyncId} batch={BatchNo}/{BatchTotal} subscriber_count={SubscriberCount}",
request.TenantId,
request.ListId,
request.SyncId,
request.BatchNo,
request.BatchTotal,
request.Subscribers.Count);
var payload = JsonSerializer.Serialize(request); var payload = JsonSerializer.Serialize(request);
var inbox = new EventInbox var inbox = new EventInbox
@ -295,9 +435,8 @@ app.MapPost("/webhooks/lists/full-sync", async (HttpContext httpContext, FullSyn
continue; continue;
} }
await UpsertSubscriberAndListMemberAsync( await UpsertSubscriptionAsync(
db, db,
request.TenantId,
request.ListId, request.ListId,
subscriber.Id, subscriber.Id,
subscriber.Email, subscriber.Email,
@ -309,25 +448,114 @@ app.MapPost("/webhooks/lists/full-sync", async (HttpContext httpContext, FullSyn
inbox.ProcessedAt = DateTimeOffset.UtcNow; inbox.ProcessedAt = DateTimeOffset.UtcNow;
await db.SaveChangesAsync(); await db.SaveChangesAsync();
logger.LogInformation(
"Full-sync webhook processed. tenant_id={TenantId} list_id={ListId} sync_id={SyncId} batch={BatchNo}/{BatchTotal}",
request.TenantId,
request.ListId,
request.SyncId,
request.BatchNo,
request.BatchTotal);
return Results.Ok(); return Results.Ok();
}).WithName("FullSyncWebhook").WithOpenApi(); }).WithName("FullSyncWebhook").WithOpenApi();
app.MapPost("/webhooks/ses", async (HttpContext httpContext, SesEventRequest request, SendEngineDbContext db) => app.MapPost("/webhooks/ses", async (
HttpContext httpContext,
SesEventRequest request,
SendEngineDbContext db,
ILoggerFactory loggerFactory) =>
{ {
var logger = loggerFactory.CreateLogger("SendEngine.Webhooks.Ses");
if (request.TenantId == Guid.Empty || string.IsNullOrWhiteSpace(request.Email))
{
logger.LogWarning("SES webhook rejected: tenant_id or email missing.");
return Results.UnprocessableEntity(new { error = "tenant_id_email_required" });
}
var skipValidation = builder.Configuration.GetValue("Ses:SkipSignatureValidation", true); var skipValidation = builder.Configuration.GetValue("Ses:SkipSignatureValidation", true);
logger.LogInformation(
"SES webhook received. Ses__SkipSignatureValidation={SkipValidation}",
skipValidation);
var sesSignature = httpContext.Request.Headers["X-Amz-Sns-Signature"].ToString(); var sesSignature = httpContext.Request.Headers["X-Amz-Sns-Signature"].ToString();
if (!skipValidation && string.IsNullOrWhiteSpace(sesSignature)) if (!skipValidation && string.IsNullOrWhiteSpace(sesSignature))
{ {
logger.LogWarning("SES webhook rejected: missing X-Amz-Sns-Signature while signature validation is enabled.");
return Results.Unauthorized(); return Results.Unauthorized();
} }
var normalizedEventType = NormalizeSesEventType(request.EventType, request.BounceType);
request.Email = request.Email.Trim().ToLowerInvariant();
request.EventType = normalizedEventType;
// TEST-FRIENDLY TEMPORARY LOGIC:
// In local integration testing, skip DB operations but keep callback flow to Member Center.
// TODO(remove-test-friendly): Remove this branch once end-to-end DB flow is stable.
var testFriendlyEnabled = builder.Configuration.GetValue("TestFriendly:Enabled", false);
if (testFriendlyEnabled)
{
Console.WriteLine(
"[TEST-FRIENDLY][SES] received event_type={0} tenant_id={1} email={2} skip_signature_validation={3}",
normalizedEventType,
request.TenantId,
request.Email,
skipValidation);
logger.LogWarning("TEST-FRIENDLY enabled: skip DB operations for /webhooks/ses, keep callback flow.");
if (TryMapDisableReason(normalizedEventType, out var reason))
{
if (!TryExtractGuidFromTags(request.Tags, "subscriber_id", out var subscriberId) ||
!TryExtractGuidFromTags(request.Tags, "list_id", out var listId))
{
return Results.UnprocessableEntity(new
{
error = "test_friendly_tags_required",
message = "tags.subscriber_id and tags.list_id must be UUID strings when TestFriendly is enabled."
});
}
var shouldNotify = true;
if (normalizedEventType == "soft_bounced")
{
var threshold = builder.Configuration.GetValue("Bounce:SoftBounceThreshold", 5);
shouldNotify = IsSoftBounceThresholdReachedFromTags(request.Tags, threshold);
}
if (shouldNotify)
{
await NotifyMemberCenterDisableAsync(
builder.Configuration,
logger,
request.TenantId,
new[] { (SubscriberId: subscriberId, ListId: listId) },
reason,
request.OccurredAt);
}
}
return Results.Ok(new { status = "ok", mode = "test-friendly-no-db" });
}
if (!await EnsureTenantForWebhookAsync(db, request.TenantId, testFriendlyEnabled))
{
logger.LogWarning("SES webhook rejected: tenant_not_found. tenant_id={TenantId}", request.TenantId);
return Results.UnprocessableEntity(new { error = "tenant_not_found" });
}
logger.LogInformation(
"SES webhook processing started. mode=normal event_type={EventType} tenant_id={TenantId} email={Email}",
normalizedEventType,
request.TenantId,
request.Email);
var payload = JsonSerializer.Serialize(request); var payload = JsonSerializer.Serialize(request);
var inbox = new EventInbox var inbox = new EventInbox
{ {
Id = Guid.NewGuid(), Id = Guid.NewGuid(),
TenantId = request.TenantId, TenantId = request.TenantId,
EventType = $"ses.{request.EventType}", EventType = $"ses.{normalizedEventType}",
Source = "ses", Source = "ses",
Payload = payload, Payload = payload,
ReceivedAt = DateTimeOffset.UtcNow, ReceivedAt = DateTimeOffset.UtcNow,
@ -337,6 +565,17 @@ app.MapPost("/webhooks/ses", async (HttpContext httpContext, SesEventRequest req
db.EventsInbox.Add(inbox); db.EventsInbox.Add(inbox);
await db.SaveChangesAsync(); await db.SaveChangesAsync();
await ApplySesEventAsync(db, builder.Configuration, logger, request, normalizedEventType);
inbox.Status = "processed";
inbox.ProcessedAt = DateTimeOffset.UtcNow;
await db.SaveChangesAsync();
logger.LogInformation(
"SES webhook processed. event_type={EventType} tenant_id={TenantId} email={Email}",
normalizedEventType,
request.TenantId,
request.Email);
return Results.Ok(); return Results.Ok();
}).WithName("SesWebhook").WithOpenApi(); }).WithName("SesWebhook").WithOpenApi();
@ -353,6 +592,33 @@ static bool IsSupportedSubscriptionEvent(string eventType)
return eventType is "subscription.activated" or "subscription.unsubscribed" or "preferences.updated"; return eventType is "subscription.activated" or "subscription.unsubscribed" or "preferences.updated";
} }
// TEST-FRIENDLY TEMPORARY LOGIC:
// When enabled, webhook ingestion can auto-create missing tenants to simplify local end-to-end testing.
// TODO(remove-test-friendly): Remove this helper and always require pre-provisioned tenant records.
static async Task<bool> EnsureTenantForWebhookAsync(SendEngineDbContext db, Guid tenantId, bool autoCreateTenant)
{
var tenantExists = await db.Tenants.AsNoTracking().AnyAsync(x => x.Id == tenantId);
if (tenantExists)
{
return true;
}
if (!autoCreateTenant)
{
return false;
}
db.Tenants.Add(new Tenant
{
Id = tenantId,
Name = $"tenant-{tenantId:N}",
CreatedAt = DateTimeOffset.UtcNow
});
await db.SaveChangesAsync();
return true;
}
static async Task EnsureListExistsAsync(SendEngineDbContext db, Guid tenantId, Guid listId) static async Task EnsureListExistsAsync(SendEngineDbContext db, Guid tenantId, Guid listId)
{ {
var listExists = await db.Lists.AsNoTracking() var listExists = await db.Lists.AsNoTracking()
@ -381,9 +647,8 @@ static async Task ApplySubscriptionSnapshotAsync(SendEngineDbContext db, Subscri
_ => "active" _ => "active"
}; };
await UpsertSubscriberAndListMemberAsync( await UpsertSubscriptionAsync(
db, db,
request.TenantId,
request.ListId, request.ListId,
request.Subscriber.Id, request.Subscriber.Id,
request.Subscriber.Email, request.Subscriber.Email,
@ -391,74 +656,41 @@ static async Task ApplySubscriptionSnapshotAsync(SendEngineDbContext db, Subscri
request.Subscriber.Preferences); request.Subscriber.Preferences);
} }
static async Task UpsertSubscriberAndListMemberAsync( static async Task UpsertSubscriptionAsync(
SendEngineDbContext db, SendEngineDbContext db,
Guid tenantId,
Guid listId, Guid listId,
Guid subscriberId, Guid externalSubscriberId,
string email, string email,
string status, string status,
Dictionary<string, object>? preferences) Dictionary<string, object>? preferences)
{ {
var now = DateTimeOffset.UtcNow; var now = DateTimeOffset.UtcNow;
var normalizedEmail = email.Trim().ToLowerInvariant(); var normalizedEmail = email.Trim().ToLowerInvariant();
var effectiveSubscriberId = subscriberId; var subscription = await db.Subscriptions
.FirstOrDefaultAsync(x => x.ListId == listId && x.Email == normalizedEmail);
var subscriber = await db.Subscribers
.FirstOrDefaultAsync(x => x.Id == subscriberId && x.TenantId == tenantId);
if (subscriber is null)
{
subscriber = await db.Subscribers
.FirstOrDefaultAsync(x => x.TenantId == tenantId && x.Email == normalizedEmail);
if (subscriber is not null)
{
effectiveSubscriberId = subscriber.Id;
}
}
var preferenceJson = preferences is null ? null : JsonSerializer.Serialize(preferences); var preferenceJson = preferences is null ? null : JsonSerializer.Serialize(preferences);
if (subscriber is null) if (subscription is null)
{ {
subscriber = new Subscriber subscription = new Subscription
{ {
Id = subscriberId, Id = Guid.NewGuid(),
TenantId = tenantId, ListId = listId,
Email = normalizedEmail, Email = normalizedEmail,
ExternalSubscriberId = externalSubscriberId == Guid.Empty ? null : externalSubscriberId,
Status = status, Status = status,
Preferences = preferenceJson, Preferences = preferenceJson,
CreatedAt = now, CreatedAt = now,
UpdatedAt = now UpdatedAt = now
}; };
db.Subscribers.Add(subscriber); db.Subscriptions.Add(subscription);
} }
else else
{ {
subscriber.Email = normalizedEmail; subscription.Email = normalizedEmail;
subscriber.Status = status; subscription.ExternalSubscriberId = externalSubscriberId == Guid.Empty ? subscription.ExternalSubscriberId : externalSubscriberId;
subscriber.Preferences = preferenceJson; subscription.Status = status;
subscriber.UpdatedAt = now; subscription.Preferences = preferenceJson;
} subscription.UpdatedAt = now;
var listMember = await db.ListMembers
.FirstOrDefaultAsync(x => x.TenantId == tenantId && x.ListId == listId && x.SubscriberId == effectiveSubscriberId);
if (listMember is null)
{
listMember = new ListMember
{
Id = Guid.NewGuid(),
TenantId = tenantId,
ListId = listId,
SubscriberId = effectiveSubscriberId,
Status = status,
CreatedAt = now,
UpdatedAt = now
};
db.ListMembers.Add(listMember);
}
else
{
listMember.Status = status;
listMember.UpdatedAt = now;
} }
} }
@ -476,6 +708,428 @@ static string NormalizeStatus(string? status, string fallback)
"unsubscribed" => "unsubscribed", "unsubscribed" => "unsubscribed",
"bounced" => "bounced", "bounced" => "bounced",
"complaint" => "complaint", "complaint" => "complaint",
"suppressed" => "suppressed",
_ => fallback _ => fallback
}; };
} }
static string NormalizeSesEventType(string? eventType, string? bounceType)
{
var normalized = eventType?.Trim().ToLowerInvariant() ?? string.Empty;
if (normalized == "bounce")
{
var bounce = bounceType?.Trim().ToLowerInvariant() ?? string.Empty;
return bounce switch
{
"hard" => "hard_bounced",
"soft" => "soft_bounced",
_ => "bounce"
};
}
return normalized;
}
static bool TryMapDisableReason(string normalizedEventType, out string reason)
{
switch (normalizedEventType)
{
case "hard_bounced":
reason = "hard_bounce";
return true;
case "soft_bounced":
reason = "soft_bounce_threshold";
return true;
case "complaint":
reason = "complaint";
return true;
case "suppression":
reason = "suppression";
return true;
default:
reason = string.Empty;
return false;
}
}
static bool TryExtractGuidFromTags(Dictionary<string, string>? tags, string key, out Guid value)
{
value = Guid.Empty;
if (tags is null)
{
return false;
}
var keyCandidates = new[]
{
key,
key.ToLowerInvariant(),
key.ToUpperInvariant(),
key.Replace("_", string.Empty),
ToCamelCase(key)
};
foreach (var candidate in keyCandidates)
{
if (tags.TryGetValue(candidate, out var raw) && Guid.TryParse(raw, out value))
{
return true;
}
}
return false;
}
static string ToCamelCase(string value)
{
var parts = value.Split('_', StringSplitOptions.RemoveEmptyEntries | StringSplitOptions.TrimEntries);
if (parts.Length == 0)
{
return value;
}
return parts[0] + string.Concat(parts.Skip(1).Select(p => char.ToUpperInvariant(p[0]) + p[1..]));
}
static bool IsSoftBounceThresholdReachedFromTags(Dictionary<string, string>? tags, int threshold)
{
if (threshold < 1)
{
threshold = 1;
}
if (tags is null)
{
return true;
}
if (tags.TryGetValue("soft_bounce_count", out var raw) && int.TryParse(raw, out var count))
{
return count >= threshold;
}
return true;
}
static async Task ApplySesEventAsync(
SendEngineDbContext db,
IConfiguration configuration,
ILogger logger,
SesEventRequest request,
string normalizedEventType)
{
var subscriptions = await db.Subscriptions
.Join(
db.Lists.AsNoTracking(),
s => s.ListId,
l => l.Id,
(s, l) => new { Subscription = s, ListTenantId = l.TenantId })
.Where(x => x.ListTenantId == request.TenantId && x.Subscription.Email == request.Email)
.Select(x => x.Subscription)
.ToListAsync();
if (subscriptions.Count == 0)
{
logger.LogWarning(
"SES event ignored: subscription not found. tenant_id={TenantId} email={Email} event_type={EventType}",
request.TenantId,
request.Email,
normalizedEventType);
return;
}
logger.LogInformation(
"SES event matched subscriptions. tenant_id={TenantId} email={Email} matched_count={MatchedCount} event_type={EventType}",
request.TenantId,
request.Email,
subscriptions.Count,
normalizedEventType);
switch (normalizedEventType)
{
case "hard_bounced":
await SuppressAndNotifyAsync(db, configuration, logger, request.TenantId, subscriptions, "hard_bounce", request.OccurredAt);
return;
case "soft_bounced":
var threshold = configuration.GetValue("Bounce:SoftBounceThreshold", 5);
var reached = await IsSoftBounceThresholdReachedAsync(db, request.TenantId, request.Email, threshold);
if (reached)
{
await SuppressAndNotifyAsync(db, configuration, logger, request.TenantId, subscriptions, "soft_bounce_threshold", request.OccurredAt);
}
else
{
logger.LogInformation(
"Soft bounce threshold not reached yet. tenant_id={TenantId} email={Email} threshold={Threshold}",
request.TenantId,
request.Email,
threshold);
}
return;
case "complaint":
await SuppressAndNotifyAsync(db, configuration, logger, request.TenantId, subscriptions, "complaint", request.OccurredAt);
return;
case "suppression":
await SuppressAndNotifyAsync(db, configuration, logger, request.TenantId, subscriptions, "suppression", request.OccurredAt);
return;
default:
return;
}
}
static async Task<bool> IsSoftBounceThresholdReachedAsync(
SendEngineDbContext db,
Guid tenantId,
string normalizedEmail,
int threshold)
{
if (threshold < 1)
{
threshold = 1;
}
var count = await db.Database.SqlQuery<int>($"""
SELECT count(*)::int AS "Value"
FROM events_inbox
WHERE tenant_id = {tenantId}
AND source = 'ses'
AND event_type = 'ses.soft_bounced'
AND payload->>'email' = {normalizedEmail}
""").SingleAsync();
return count >= threshold;
}
static async Task SuppressAndNotifyAsync(
SendEngineDbContext db,
IConfiguration configuration,
ILogger logger,
Guid tenantId,
IReadOnlyCollection<Subscription> subscriptions,
string reason,
DateTimeOffset occurredAt)
{
if (subscriptions.Count == 0)
{
return;
}
var now = DateTimeOffset.UtcNow;
foreach (var subscription in subscriptions)
{
subscription.Status = "suppressed";
subscription.UpdatedAt = now;
}
await db.SaveChangesAsync();
var notifyTargets = subscriptions
.Where(x => x.ExternalSubscriberId.HasValue)
.Select(x => (SubscriberId: x.ExternalSubscriberId!.Value, ListId: x.ListId))
.Distinct()
.ToArray();
logger.LogInformation(
"Subscriptions suppressed. tenant_id={TenantId} reason={Reason} matched_count={MatchedCount} notify_target_count={NotifyCount}",
tenantId,
reason,
subscriptions.Count,
notifyTargets.Length);
await NotifyMemberCenterDisableAsync(configuration, logger, tenantId, notifyTargets, reason, occurredAt);
}
static async Task NotifyMemberCenterDisableAsync(
IConfiguration configuration,
ILogger logger,
Guid tenantId,
IReadOnlyCollection<(Guid SubscriberId, Guid ListId)> targets,
string reason,
DateTimeOffset occurredAt)
{
if (targets.Count == 0)
{
logger.LogWarning("MemberCenter callback skipped: no disable targets.");
return;
}
var url = ResolveMemberCenterUrl(
configuration,
"MemberCenter:DisableSubscriptionUrl",
"MemberCenter:BaseUrl",
"MemberCenter:DisableSubscriptionPath",
"/subscriptions/disable");
if (string.IsNullOrWhiteSpace(url))
{
logger.LogWarning("MemberCenter callback skipped: URL is empty.");
return;
}
using var client = new HttpClient();
var token = await ResolveMemberCenterAccessTokenAsync(configuration, client, logger);
if (string.IsNullOrWhiteSpace(token))
{
logger.LogWarning("MemberCenter callback skipped: access token is empty.");
return;
}
client.DefaultRequestHeaders.Authorization = new("Bearer", token);
logger.LogInformation(
"MemberCenter callback prepared. url={Url} auth_header={AuthHeader}",
url,
BuildMaskedAuthHeader(token));
foreach (var target in targets)
{
var payload = new
{
tenant_id = tenantId,
subscriber_id = target.SubscriberId,
list_id = target.ListId,
reason,
disabled_by = "send_engine",
occurred_at = occurredAt
};
try
{
var payloadJson = JsonSerializer.Serialize(payload);
logger.LogInformation(
"MemberCenter callback request. method=POST url={Url} headers={Headers} body={Body}",
url,
$"Authorization:{BuildMaskedAuthHeader(token)}; Content-Type:application/json",
payloadJson);
using var response = await client.PostAsJsonAsync(url, payload);
var responseBody = await response.Content.ReadAsStringAsync();
logger.LogInformation(
"MemberCenter callback response. status={StatusCode} body={Body}",
(int)response.StatusCode,
Truncate(responseBody, 1000));
}
catch (Exception ex)
{
logger.LogError(ex, "MemberCenter callback failed. url={Url} list_id={ListId}", url, target.ListId);
}
}
}
static async Task<string?> ResolveMemberCenterAccessTokenAsync(IConfiguration configuration, HttpClient client, ILogger logger)
{
var tokenUrl = ResolveMemberCenterUrl(
configuration,
"MemberCenter:TokenUrl",
"MemberCenter:BaseUrl",
"MemberCenter:TokenPath",
"/oauth/token");
var clientId = configuration["MemberCenter:ClientId"];
var clientSecret = configuration["MemberCenter:ClientSecret"];
var scope = configuration["MemberCenter:Scope"];
if (!string.IsNullOrWhiteSpace(tokenUrl) &&
!string.IsNullOrWhiteSpace(clientId) &&
!string.IsNullOrWhiteSpace(clientSecret))
{
var form = new Dictionary<string, string>
{
["grant_type"] = "client_credentials",
["client_id"] = clientId,
["client_secret"] = clientSecret
};
if (!string.IsNullOrWhiteSpace(scope))
{
form["scope"] = scope;
}
try
{
logger.LogInformation(
"MemberCenter token request. url={TokenUrl} client_id={ClientId} scope={Scope}",
tokenUrl,
clientId,
scope ?? string.Empty);
using var response = await client.PostAsync(tokenUrl, new FormUrlEncodedContent(form));
if (response.IsSuccessStatusCode)
{
await using var stream = await response.Content.ReadAsStreamAsync();
using var json = await JsonDocument.ParseAsync(stream);
if (json.RootElement.TryGetProperty("access_token", out var tokenElement))
{
var accessToken = tokenElement.GetString();
if (!string.IsNullOrWhiteSpace(accessToken))
{
logger.LogInformation("MemberCenter token request succeeded.");
return accessToken;
}
}
}
else
{
var body = await response.Content.ReadAsStringAsync();
logger.LogWarning(
"MemberCenter token request failed. status={StatusCode} body={Body}",
(int)response.StatusCode,
Truncate(body, 1000));
}
}
catch (Exception ex)
{
logger.LogWarning(ex, "MemberCenter token request error, fallback to static token.");
}
}
var fallbackToken = configuration["MemberCenter:ApiToken"];
if (!string.IsNullOrWhiteSpace(fallbackToken))
{
logger.LogWarning("Using MemberCenter__ApiToken fallback.");
}
return fallbackToken;
}
static string? ResolveMemberCenterUrl(
IConfiguration configuration,
string fullUrlKey,
string baseUrlKey,
string pathKey,
string defaultPath)
{
var fullUrl = configuration[fullUrlKey];
if (!string.IsNullOrWhiteSpace(fullUrl))
{
return fullUrl;
}
var baseUrl = configuration[baseUrlKey];
if (string.IsNullOrWhiteSpace(baseUrl))
{
return null;
}
var path = configuration[pathKey];
if (string.IsNullOrWhiteSpace(path))
{
path = defaultPath;
}
return $"{baseUrl.TrimEnd('/')}/{path.TrimStart('/')}";
}
static string BuildMaskedAuthHeader(string token)
{
if (string.IsNullOrWhiteSpace(token))
{
return "Bearer <empty>";
}
var visible = token.Length <= 8 ? token : token[^8..];
return $"Bearer ***{visible}";
}
static string Truncate(string? input, int maxLen)
{
if (string.IsNullOrEmpty(input))
{
return string.Empty;
}
return input.Length <= maxLen ? input : $"{input[..maxLen]}...(truncated)";
}

View File

@ -8,16 +8,23 @@ namespace SendEngine.Api.Security;
public static class WebhookValidator public static class WebhookValidator
{ {
public const string FailureReasonItemKey = "webhook_auth_failure_reason";
public const string FailureStatusCodeItemKey = "webhook_auth_failure_status_code";
public const string RawBodyItemKey = "webhook_raw_body";
public static async Task<IResult?> ValidateAsync( public static async Task<IResult?> ValidateAsync(
HttpContext context, HttpContext context,
SendEngineDbContext db, SendEngineDbContext db,
string secret, string secret,
int maxSkewSeconds, int maxSkewSeconds,
Guid payloadTenantId, Guid payloadTenantId,
bool allowNullTenantClient) bool allowNullTenantClient,
params string[] requiredAnyScopes)
{ {
if (string.IsNullOrWhiteSpace(secret)) if (string.IsNullOrWhiteSpace(secret))
{ {
context.Items[FailureReasonItemKey] = "server_secret_missing";
context.Items[FailureStatusCodeItemKey] = StatusCodes.Status500InternalServerError;
return Results.StatusCode(StatusCodes.Status500InternalServerError); return Results.StatusCode(StatusCodes.Status500InternalServerError);
} }
var signature = context.Request.Headers["X-Signature"].ToString(); var signature = context.Request.Headers["X-Signature"].ToString();
@ -28,60 +35,96 @@ public static class WebhookValidator
if (string.IsNullOrWhiteSpace(signature) || string.IsNullOrWhiteSpace(timestampHeader) if (string.IsNullOrWhiteSpace(signature) || string.IsNullOrWhiteSpace(timestampHeader)
|| string.IsNullOrWhiteSpace(nonce) || string.IsNullOrWhiteSpace(clientIdHeader)) || string.IsNullOrWhiteSpace(nonce) || string.IsNullOrWhiteSpace(clientIdHeader))
{ {
context.Items[FailureReasonItemKey] = "missing_required_headers";
context.Items[FailureStatusCodeItemKey] = StatusCodes.Status401Unauthorized;
return Results.Unauthorized(); return Results.Unauthorized();
} }
if (!long.TryParse(timestampHeader, out var timestampSeconds)) if (!long.TryParse(timestampHeader, out var timestampSeconds))
{ {
context.Items[FailureReasonItemKey] = "invalid_timestamp";
context.Items[FailureStatusCodeItemKey] = StatusCodes.Status401Unauthorized;
return Results.Unauthorized(); return Results.Unauthorized();
} }
var nowSeconds = DateTimeOffset.UtcNow.ToUnixTimeSeconds(); var nowSeconds = DateTimeOffset.UtcNow.ToUnixTimeSeconds();
if (Math.Abs(nowSeconds - timestampSeconds) > maxSkewSeconds) if (Math.Abs(nowSeconds - timestampSeconds) > maxSkewSeconds)
{ {
context.Items[FailureReasonItemKey] = "timestamp_out_of_skew";
context.Items[FailureStatusCodeItemKey] = StatusCodes.Status401Unauthorized;
return Results.StatusCode(StatusCodes.Status401Unauthorized); return Results.StatusCode(StatusCodes.Status401Unauthorized);
} }
if (!Guid.TryParse(clientIdHeader, out var clientId)) if (!Guid.TryParse(clientIdHeader, out var clientId))
{ {
context.Items[FailureReasonItemKey] = "invalid_client_id_header";
context.Items[FailureStatusCodeItemKey] = StatusCodes.Status401Unauthorized;
return Results.Unauthorized(); return Results.Unauthorized();
} }
var authClient = await db.AuthClients.AsNoTracking().FirstOrDefaultAsync(x => x.Id == clientId); var authClient = await db.AuthClients.AsNoTracking().FirstOrDefaultAsync(x => x.Id == clientId);
if (authClient is null) if (authClient is null)
{ {
context.Items[FailureReasonItemKey] = "auth_client_not_found";
context.Items[FailureStatusCodeItemKey] = StatusCodes.Status403Forbidden;
return Results.StatusCode(StatusCodes.Status403Forbidden); return Results.StatusCode(StatusCodes.Status403Forbidden);
} }
if (!string.Equals(authClient.Status, "active", StringComparison.OrdinalIgnoreCase)) if (!string.Equals(authClient.Status, "active", StringComparison.OrdinalIgnoreCase))
{ {
context.Items[FailureReasonItemKey] = "auth_client_inactive";
context.Items[FailureStatusCodeItemKey] = StatusCodes.Status403Forbidden;
return Results.StatusCode(StatusCodes.Status403Forbidden); return Results.StatusCode(StatusCodes.Status403Forbidden);
} }
if (authClient.TenantId is null && !allowNullTenantClient) if (authClient.TenantId is null && !allowNullTenantClient)
{ {
context.Items[FailureReasonItemKey] = "null_tenant_client_forbidden";
context.Items[FailureStatusCodeItemKey] = StatusCodes.Status403Forbidden;
return Results.StatusCode(StatusCodes.Status403Forbidden); return Results.StatusCode(StatusCodes.Status403Forbidden);
} }
if (authClient.TenantId is not null && authClient.TenantId.Value != payloadTenantId) if (authClient.TenantId is not null && authClient.TenantId.Value != payloadTenantId)
{ {
context.Items[FailureReasonItemKey] = "tenant_mismatch";
context.Items[FailureStatusCodeItemKey] = StatusCodes.Status403Forbidden;
return Results.StatusCode(StatusCodes.Status403Forbidden); return Results.StatusCode(StatusCodes.Status403Forbidden);
} }
context.Request.EnableBuffering(); if (requiredAnyScopes.Length > 0)
using var reader = new StreamReader(context.Request.Body, Encoding.UTF8, leaveOpen: true); {
var body = await reader.ReadToEndAsync(); var granted = authClient.Scopes ?? Array.Empty<string>();
context.Request.Body.Position = 0; var matched = granted
.Select(x => x.Trim())
.Where(x => x.Length > 0)
.Intersect(requiredAnyScopes, StringComparer.OrdinalIgnoreCase)
.Any();
if (!matched)
{
context.Items[FailureReasonItemKey] = "scope_missing";
context.Items[FailureStatusCodeItemKey] = StatusCodes.Status403Forbidden;
return Results.StatusCode(StatusCodes.Status403Forbidden);
}
}
var body = context.Items.TryGetValue(RawBodyItemKey, out var rawBodyValue) &&
rawBodyValue is string rawBody
? rawBody
: await ReadBodyAsync(context);
var expected = ComputeHmacHex(secret, body); var expected = ComputeHmacHex(secret, body);
if (!FixedTimeEquals(expected, signature)) if (!FixedTimeEquals(expected, signature))
{ {
context.Items[FailureReasonItemKey] = "invalid_signature";
context.Items[FailureStatusCodeItemKey] = StatusCodes.Status401Unauthorized;
return Results.Unauthorized(); return Results.Unauthorized();
} }
var hasNonce = await db.WebhookNonces.AsNoTracking().AnyAsync(x => x.ClientId == clientId && x.Nonce == nonce); var hasNonce = await db.WebhookNonces.AsNoTracking().AnyAsync(x => x.ClientId == clientId && x.Nonce == nonce);
if (hasNonce) if (hasNonce)
{ {
context.Items[FailureReasonItemKey] = "replay_detected";
context.Items[FailureStatusCodeItemKey] = StatusCodes.Status409Conflict;
return Results.Conflict(new { error = "replay_detected" }); return Results.Conflict(new { error = "replay_detected" });
} }
@ -95,6 +138,8 @@ public static class WebhookValidator
db.WebhookNonces.Add(nonceEntry); db.WebhookNonces.Add(nonceEntry);
await db.SaveChangesAsync(); await db.SaveChangesAsync();
context.Items[FailureReasonItemKey] = "ok";
context.Items[FailureStatusCodeItemKey] = StatusCodes.Status200OK;
return null; return null;
} }
@ -113,4 +158,14 @@ public static class WebhookValidator
var bBytes = Encoding.UTF8.GetBytes(b); var bBytes = Encoding.UTF8.GetBytes(b);
return aBytes.Length == bBytes.Length && CryptographicOperations.FixedTimeEquals(aBytes, bBytes); return aBytes.Length == bBytes.Length && CryptographicOperations.FixedTimeEquals(aBytes, bBytes);
} }
private static async Task<string> ReadBodyAsync(HttpContext context)
{
context.Request.EnableBuffering();
using var reader = new StreamReader(context.Request.Body, Encoding.UTF8, leaveOpen: true);
var body = await reader.ReadToEndAsync();
context.Request.Body.Position = 0;
return body;
}
} }

View File

@ -9,6 +9,10 @@
<ItemGroup> <ItemGroup>
<PackageReference Include="Microsoft.AspNetCore.Authentication.JwtBearer" Version="8.0.0" /> <PackageReference Include="Microsoft.AspNetCore.Authentication.JwtBearer" Version="8.0.0" />
<PackageReference Include="Microsoft.AspNetCore.OpenApi" Version="8.0.23" /> <PackageReference Include="Microsoft.AspNetCore.OpenApi" Version="8.0.23" />
<PackageReference Include="Microsoft.EntityFrameworkCore.Design" Version="8.0.0">
<IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
<PrivateAssets>all</PrivateAssets>
</PackageReference>
<PackageReference Include="Swashbuckle.AspNetCore" Version="6.6.2" /> <PackageReference Include="Swashbuckle.AspNetCore" Version="6.6.2" />
</ItemGroup> </ItemGroup>

View File

@ -1,12 +0,0 @@
namespace SendEngine.Domain.Entities;
public sealed class ListMember
{
public Guid Id { get; set; }
public Guid TenantId { get; set; }
public Guid ListId { get; set; }
public Guid SubscriberId { get; set; }
public string Status { get; set; } = string.Empty;
public DateTimeOffset CreatedAt { get; set; }
public DateTimeOffset UpdatedAt { get; set; }
}

View File

@ -1,10 +1,11 @@
namespace SendEngine.Domain.Entities; namespace SendEngine.Domain.Entities;
public sealed class Subscriber public sealed class Subscription
{ {
public Guid Id { get; set; } public Guid Id { get; set; }
public Guid TenantId { get; set; } public Guid ListId { get; set; }
public string Email { get; set; } = string.Empty; public string Email { get; set; } = string.Empty;
public Guid? ExternalSubscriberId { get; set; }
public string Status { get; set; } = string.Empty; public string Status { get; set; } = string.Empty;
public string? Preferences { get; set; } public string? Preferences { get; set; }
public DateTimeOffset CreatedAt { get; set; } public DateTimeOffset CreatedAt { get; set; }

View File

@ -12,8 +12,8 @@ using SendEngine.Infrastructure.Data;
namespace SendEngine.Infrastructure.Data.Migrations namespace SendEngine.Infrastructure.Data.Migrations
{ {
[DbContext(typeof(SendEngineDbContext))] [DbContext(typeof(SendEngineDbContext))]
[Migration("20260210083240_Initial")] [Migration("20260219074637_20260219_Rebaseline")]
partial class Initial partial class _20260219_Rebaseline
{ {
/// <inheritdoc /> /// <inheritdoc />
protected override void BuildTargetModel(ModelBuilder modelBuilder) protected override void BuildTargetModel(ModelBuilder modelBuilder)
@ -257,54 +257,6 @@ namespace SendEngine.Infrastructure.Data.Migrations
b.ToTable("events_inbox", (string)null); b.ToTable("events_inbox", (string)null);
}); });
modelBuilder.Entity("SendEngine.Domain.Entities.ListMember", b =>
{
b.Property<Guid>("Id")
.ValueGeneratedOnAdd()
.HasColumnType("uuid")
.HasColumnName("id");
b.Property<DateTimeOffset>("CreatedAt")
.HasColumnType("timestamp with time zone")
.HasColumnName("created_at");
b.Property<Guid>("ListId")
.HasColumnType("uuid")
.HasColumnName("list_id");
b.Property<string>("Status")
.IsRequired()
.HasColumnType("text")
.HasColumnName("status");
b.Property<Guid>("SubscriberId")
.HasColumnType("uuid")
.HasColumnName("subscriber_id");
b.Property<Guid>("TenantId")
.HasColumnType("uuid")
.HasColumnName("tenant_id");
b.Property<DateTimeOffset>("UpdatedAt")
.HasColumnType("timestamp with time zone")
.HasColumnName("updated_at");
b.HasKey("Id");
b.HasIndex("ListId")
.HasDatabaseName("idx_list_members_list");
b.HasIndex("SubscriberId");
b.HasIndex("TenantId")
.HasDatabaseName("idx_list_members_tenant");
b.HasIndex("TenantId", "ListId", "SubscriberId")
.IsUnique();
b.ToTable("list_members", (string)null);
});
modelBuilder.Entity("SendEngine.Domain.Entities.MailingList", b => modelBuilder.Entity("SendEngine.Domain.Entities.MailingList", b =>
{ {
b.Property<Guid>("Id") b.Property<Guid>("Id")
@ -437,7 +389,7 @@ namespace SendEngine.Infrastructure.Data.Migrations
b.ToTable("send_jobs", (string)null); b.ToTable("send_jobs", (string)null);
}); });
modelBuilder.Entity("SendEngine.Domain.Entities.Subscriber", b => modelBuilder.Entity("SendEngine.Domain.Entities.Subscription", b =>
{ {
b.Property<Guid>("Id") b.Property<Guid>("Id")
.ValueGeneratedOnAdd() .ValueGeneratedOnAdd()
@ -453,6 +405,14 @@ namespace SendEngine.Infrastructure.Data.Migrations
.HasColumnType("citext") .HasColumnType("citext")
.HasColumnName("email"); .HasColumnName("email");
b.Property<Guid?>("ExternalSubscriberId")
.HasColumnType("uuid")
.HasColumnName("external_subscriber_id");
b.Property<Guid>("ListId")
.HasColumnType("uuid")
.HasColumnName("list_id");
b.Property<string>("Preferences") b.Property<string>("Preferences")
.HasColumnType("jsonb") .HasColumnType("jsonb")
.HasColumnName("preferences"); .HasColumnName("preferences");
@ -462,23 +422,22 @@ namespace SendEngine.Infrastructure.Data.Migrations
.HasColumnType("text") .HasColumnType("text")
.HasColumnName("status"); .HasColumnName("status");
b.Property<Guid>("TenantId")
.HasColumnType("uuid")
.HasColumnName("tenant_id");
b.Property<DateTimeOffset>("UpdatedAt") b.Property<DateTimeOffset>("UpdatedAt")
.HasColumnType("timestamp with time zone") .HasColumnType("timestamp with time zone")
.HasColumnName("updated_at"); .HasColumnName("updated_at");
b.HasKey("Id"); b.HasKey("Id");
b.HasIndex("TenantId") b.HasIndex("ExternalSubscriberId")
.HasDatabaseName("idx_subscribers_tenant"); .HasDatabaseName("idx_subscriptions_external_subscriber");
b.HasIndex("TenantId", "Email") b.HasIndex("ListId")
.HasDatabaseName("idx_subscriptions_list");
b.HasIndex("ListId", "Email")
.IsUnique(); .IsUnique();
b.ToTable("subscribers", (string)null); b.ToTable("subscriptions", (string)null);
}); });
modelBuilder.Entity("SendEngine.Domain.Entities.Tenant", b => modelBuilder.Entity("SendEngine.Domain.Entities.Tenant", b =>
@ -586,30 +545,6 @@ namespace SendEngine.Infrastructure.Data.Migrations
.HasConstraintName("fk_events_inbox_tenant"); .HasConstraintName("fk_events_inbox_tenant");
}); });
modelBuilder.Entity("SendEngine.Domain.Entities.ListMember", b =>
{
b.HasOne("SendEngine.Domain.Entities.MailingList", null)
.WithMany()
.HasForeignKey("ListId")
.OnDelete(DeleteBehavior.Cascade)
.IsRequired()
.HasConstraintName("fk_list_members_list");
b.HasOne("SendEngine.Domain.Entities.Subscriber", null)
.WithMany()
.HasForeignKey("SubscriberId")
.OnDelete(DeleteBehavior.Cascade)
.IsRequired()
.HasConstraintName("fk_list_members_subscriber");
b.HasOne("SendEngine.Domain.Entities.Tenant", null)
.WithMany()
.HasForeignKey("TenantId")
.OnDelete(DeleteBehavior.Cascade)
.IsRequired()
.HasConstraintName("fk_list_members_tenant");
});
modelBuilder.Entity("SendEngine.Domain.Entities.MailingList", b => modelBuilder.Entity("SendEngine.Domain.Entities.MailingList", b =>
{ {
b.HasOne("SendEngine.Domain.Entities.Tenant", null) b.HasOne("SendEngine.Domain.Entities.Tenant", null)
@ -661,14 +596,14 @@ namespace SendEngine.Infrastructure.Data.Migrations
.HasConstraintName("fk_send_jobs_tenant"); .HasConstraintName("fk_send_jobs_tenant");
}); });
modelBuilder.Entity("SendEngine.Domain.Entities.Subscriber", b => modelBuilder.Entity("SendEngine.Domain.Entities.Subscription", b =>
{ {
b.HasOne("SendEngine.Domain.Entities.Tenant", null) b.HasOne("SendEngine.Domain.Entities.MailingList", null)
.WithMany() .WithMany()
.HasForeignKey("TenantId") .HasForeignKey("ListId")
.OnDelete(DeleteBehavior.Cascade) .OnDelete(DeleteBehavior.Cascade)
.IsRequired() .IsRequired()
.HasConstraintName("fk_subscribers_tenant"); .HasConstraintName("fk_subscriptions_list");
}); });
modelBuilder.Entity("SendEngine.Domain.Entities.WebhookNonce", b => modelBuilder.Entity("SendEngine.Domain.Entities.WebhookNonce", b =>

View File

@ -6,7 +6,7 @@ using Microsoft.EntityFrameworkCore.Migrations;
namespace SendEngine.Infrastructure.Data.Migrations namespace SendEngine.Infrastructure.Data.Migrations
{ {
/// <inheritdoc /> /// <inheritdoc />
public partial class Initial : Migration public partial class _20260219_Rebaseline : Migration
{ {
/// <inheritdoc /> /// <inheritdoc />
protected override void Up(MigrationBuilder migrationBuilder) protected override void Up(MigrationBuilder migrationBuilder)
@ -130,29 +130,6 @@ namespace SendEngine.Infrastructure.Data.Migrations
onDelete: ReferentialAction.Cascade); onDelete: ReferentialAction.Cascade);
}); });
migrationBuilder.CreateTable(
name: "subscribers",
columns: table => new
{
id = table.Column<Guid>(type: "uuid", nullable: false),
tenant_id = table.Column<Guid>(type: "uuid", nullable: false),
email = table.Column<string>(type: "citext", nullable: false),
status = table.Column<string>(type: "text", nullable: false),
preferences = table.Column<string>(type: "jsonb", nullable: true),
created_at = table.Column<DateTimeOffset>(type: "timestamp with time zone", nullable: false),
updated_at = table.Column<DateTimeOffset>(type: "timestamp with time zone", nullable: false)
},
constraints: table =>
{
table.PrimaryKey("PK_subscribers", x => x.id);
table.ForeignKey(
name: "fk_subscribers_tenant",
column: x => x.tenant_id,
principalTable: "tenants",
principalColumn: "id",
onDelete: ReferentialAction.Cascade);
});
migrationBuilder.CreateTable( migrationBuilder.CreateTable(
name: "campaigns", name: "campaigns",
columns: table => new columns: table => new
@ -185,38 +162,27 @@ namespace SendEngine.Infrastructure.Data.Migrations
}); });
migrationBuilder.CreateTable( migrationBuilder.CreateTable(
name: "list_members", name: "subscriptions",
columns: table => new columns: table => new
{ {
id = table.Column<Guid>(type: "uuid", nullable: false), id = table.Column<Guid>(type: "uuid", nullable: false),
tenant_id = table.Column<Guid>(type: "uuid", nullable: false),
list_id = table.Column<Guid>(type: "uuid", nullable: false), list_id = table.Column<Guid>(type: "uuid", nullable: false),
subscriber_id = table.Column<Guid>(type: "uuid", nullable: false), email = table.Column<string>(type: "citext", nullable: false),
external_subscriber_id = table.Column<Guid>(type: "uuid", nullable: true),
status = table.Column<string>(type: "text", nullable: false), status = table.Column<string>(type: "text", nullable: false),
preferences = table.Column<string>(type: "jsonb", nullable: true),
created_at = table.Column<DateTimeOffset>(type: "timestamp with time zone", nullable: false), created_at = table.Column<DateTimeOffset>(type: "timestamp with time zone", nullable: false),
updated_at = table.Column<DateTimeOffset>(type: "timestamp with time zone", nullable: false) updated_at = table.Column<DateTimeOffset>(type: "timestamp with time zone", nullable: false)
}, },
constraints: table => constraints: table =>
{ {
table.PrimaryKey("PK_list_members", x => x.id); table.PrimaryKey("PK_subscriptions", x => x.id);
table.ForeignKey( table.ForeignKey(
name: "fk_list_members_list", name: "fk_subscriptions_list",
column: x => x.list_id, column: x => x.list_id,
principalTable: "lists", principalTable: "lists",
principalColumn: "id", principalColumn: "id",
onDelete: ReferentialAction.Cascade); onDelete: ReferentialAction.Cascade);
table.ForeignKey(
name: "fk_list_members_subscriber",
column: x => x.subscriber_id,
principalTable: "subscribers",
principalColumn: "id",
onDelete: ReferentialAction.Cascade);
table.ForeignKey(
name: "fk_list_members_tenant",
column: x => x.tenant_id,
principalTable: "tenants",
principalColumn: "id",
onDelete: ReferentialAction.Cascade);
}); });
migrationBuilder.CreateTable( migrationBuilder.CreateTable(
@ -363,27 +329,6 @@ namespace SendEngine.Infrastructure.Data.Migrations
table: "events_inbox", table: "events_inbox",
column: "event_type"); column: "event_type");
migrationBuilder.CreateIndex(
name: "idx_list_members_list",
table: "list_members",
column: "list_id");
migrationBuilder.CreateIndex(
name: "idx_list_members_tenant",
table: "list_members",
column: "tenant_id");
migrationBuilder.CreateIndex(
name: "IX_list_members_subscriber_id",
table: "list_members",
column: "subscriber_id");
migrationBuilder.CreateIndex(
name: "IX_list_members_tenant_id_list_id_subscriber_id",
table: "list_members",
columns: new[] { "tenant_id", "list_id", "subscriber_id" },
unique: true);
migrationBuilder.CreateIndex( migrationBuilder.CreateIndex(
name: "idx_lists_tenant", name: "idx_lists_tenant",
table: "lists", table: "lists",
@ -425,14 +370,19 @@ namespace SendEngine.Infrastructure.Data.Migrations
column: "list_id"); column: "list_id");
migrationBuilder.CreateIndex( migrationBuilder.CreateIndex(
name: "idx_subscribers_tenant", name: "idx_subscriptions_external_subscriber",
table: "subscribers", table: "subscriptions",
column: "tenant_id"); column: "external_subscriber_id");
migrationBuilder.CreateIndex( migrationBuilder.CreateIndex(
name: "IX_subscribers_tenant_id_email", name: "idx_subscriptions_list",
table: "subscribers", table: "subscriptions",
columns: new[] { "tenant_id", "email" }, column: "list_id");
migrationBuilder.CreateIndex(
name: "IX_subscriptions_list_id_email",
table: "subscriptions",
columns: new[] { "list_id", "email" },
unique: true); unique: true);
migrationBuilder.CreateIndex( migrationBuilder.CreateIndex(
@ -459,17 +409,14 @@ namespace SendEngine.Infrastructure.Data.Migrations
migrationBuilder.DropTable( migrationBuilder.DropTable(
name: "events_inbox"); name: "events_inbox");
migrationBuilder.DropTable(
name: "list_members");
migrationBuilder.DropTable( migrationBuilder.DropTable(
name: "send_batches"); name: "send_batches");
migrationBuilder.DropTable( migrationBuilder.DropTable(
name: "webhook_nonces"); name: "subscriptions");
migrationBuilder.DropTable( migrationBuilder.DropTable(
name: "subscribers"); name: "webhook_nonces");
migrationBuilder.DropTable( migrationBuilder.DropTable(
name: "send_jobs"); name: "send_jobs");

View File

@ -254,54 +254,6 @@ namespace SendEngine.Infrastructure.Data.Migrations
b.ToTable("events_inbox", (string)null); b.ToTable("events_inbox", (string)null);
}); });
modelBuilder.Entity("SendEngine.Domain.Entities.ListMember", b =>
{
b.Property<Guid>("Id")
.ValueGeneratedOnAdd()
.HasColumnType("uuid")
.HasColumnName("id");
b.Property<DateTimeOffset>("CreatedAt")
.HasColumnType("timestamp with time zone")
.HasColumnName("created_at");
b.Property<Guid>("ListId")
.HasColumnType("uuid")
.HasColumnName("list_id");
b.Property<string>("Status")
.IsRequired()
.HasColumnType("text")
.HasColumnName("status");
b.Property<Guid>("SubscriberId")
.HasColumnType("uuid")
.HasColumnName("subscriber_id");
b.Property<Guid>("TenantId")
.HasColumnType("uuid")
.HasColumnName("tenant_id");
b.Property<DateTimeOffset>("UpdatedAt")
.HasColumnType("timestamp with time zone")
.HasColumnName("updated_at");
b.HasKey("Id");
b.HasIndex("ListId")
.HasDatabaseName("idx_list_members_list");
b.HasIndex("SubscriberId");
b.HasIndex("TenantId")
.HasDatabaseName("idx_list_members_tenant");
b.HasIndex("TenantId", "ListId", "SubscriberId")
.IsUnique();
b.ToTable("list_members", (string)null);
});
modelBuilder.Entity("SendEngine.Domain.Entities.MailingList", b => modelBuilder.Entity("SendEngine.Domain.Entities.MailingList", b =>
{ {
b.Property<Guid>("Id") b.Property<Guid>("Id")
@ -434,7 +386,7 @@ namespace SendEngine.Infrastructure.Data.Migrations
b.ToTable("send_jobs", (string)null); b.ToTable("send_jobs", (string)null);
}); });
modelBuilder.Entity("SendEngine.Domain.Entities.Subscriber", b => modelBuilder.Entity("SendEngine.Domain.Entities.Subscription", b =>
{ {
b.Property<Guid>("Id") b.Property<Guid>("Id")
.ValueGeneratedOnAdd() .ValueGeneratedOnAdd()
@ -450,6 +402,14 @@ namespace SendEngine.Infrastructure.Data.Migrations
.HasColumnType("citext") .HasColumnType("citext")
.HasColumnName("email"); .HasColumnName("email");
b.Property<Guid?>("ExternalSubscriberId")
.HasColumnType("uuid")
.HasColumnName("external_subscriber_id");
b.Property<Guid>("ListId")
.HasColumnType("uuid")
.HasColumnName("list_id");
b.Property<string>("Preferences") b.Property<string>("Preferences")
.HasColumnType("jsonb") .HasColumnType("jsonb")
.HasColumnName("preferences"); .HasColumnName("preferences");
@ -459,23 +419,22 @@ namespace SendEngine.Infrastructure.Data.Migrations
.HasColumnType("text") .HasColumnType("text")
.HasColumnName("status"); .HasColumnName("status");
b.Property<Guid>("TenantId")
.HasColumnType("uuid")
.HasColumnName("tenant_id");
b.Property<DateTimeOffset>("UpdatedAt") b.Property<DateTimeOffset>("UpdatedAt")
.HasColumnType("timestamp with time zone") .HasColumnType("timestamp with time zone")
.HasColumnName("updated_at"); .HasColumnName("updated_at");
b.HasKey("Id"); b.HasKey("Id");
b.HasIndex("TenantId") b.HasIndex("ExternalSubscriberId")
.HasDatabaseName("idx_subscribers_tenant"); .HasDatabaseName("idx_subscriptions_external_subscriber");
b.HasIndex("TenantId", "Email") b.HasIndex("ListId")
.HasDatabaseName("idx_subscriptions_list");
b.HasIndex("ListId", "Email")
.IsUnique(); .IsUnique();
b.ToTable("subscribers", (string)null); b.ToTable("subscriptions", (string)null);
}); });
modelBuilder.Entity("SendEngine.Domain.Entities.Tenant", b => modelBuilder.Entity("SendEngine.Domain.Entities.Tenant", b =>
@ -583,30 +542,6 @@ namespace SendEngine.Infrastructure.Data.Migrations
.HasConstraintName("fk_events_inbox_tenant"); .HasConstraintName("fk_events_inbox_tenant");
}); });
modelBuilder.Entity("SendEngine.Domain.Entities.ListMember", b =>
{
b.HasOne("SendEngine.Domain.Entities.MailingList", null)
.WithMany()
.HasForeignKey("ListId")
.OnDelete(DeleteBehavior.Cascade)
.IsRequired()
.HasConstraintName("fk_list_members_list");
b.HasOne("SendEngine.Domain.Entities.Subscriber", null)
.WithMany()
.HasForeignKey("SubscriberId")
.OnDelete(DeleteBehavior.Cascade)
.IsRequired()
.HasConstraintName("fk_list_members_subscriber");
b.HasOne("SendEngine.Domain.Entities.Tenant", null)
.WithMany()
.HasForeignKey("TenantId")
.OnDelete(DeleteBehavior.Cascade)
.IsRequired()
.HasConstraintName("fk_list_members_tenant");
});
modelBuilder.Entity("SendEngine.Domain.Entities.MailingList", b => modelBuilder.Entity("SendEngine.Domain.Entities.MailingList", b =>
{ {
b.HasOne("SendEngine.Domain.Entities.Tenant", null) b.HasOne("SendEngine.Domain.Entities.Tenant", null)
@ -658,14 +593,14 @@ namespace SendEngine.Infrastructure.Data.Migrations
.HasConstraintName("fk_send_jobs_tenant"); .HasConstraintName("fk_send_jobs_tenant");
}); });
modelBuilder.Entity("SendEngine.Domain.Entities.Subscriber", b => modelBuilder.Entity("SendEngine.Domain.Entities.Subscription", b =>
{ {
b.HasOne("SendEngine.Domain.Entities.Tenant", null) b.HasOne("SendEngine.Domain.Entities.MailingList", null)
.WithMany() .WithMany()
.HasForeignKey("TenantId") .HasForeignKey("ListId")
.OnDelete(DeleteBehavior.Cascade) .OnDelete(DeleteBehavior.Cascade)
.IsRequired() .IsRequired()
.HasConstraintName("fk_subscribers_tenant"); .HasConstraintName("fk_subscriptions_list");
}); });
modelBuilder.Entity("SendEngine.Domain.Entities.WebhookNonce", b => modelBuilder.Entity("SendEngine.Domain.Entities.WebhookNonce", b =>

View File

@ -11,8 +11,7 @@ public sealed class SendEngineDbContext : DbContext
public DbSet<Tenant> Tenants => Set<Tenant>(); public DbSet<Tenant> Tenants => Set<Tenant>();
public DbSet<MailingList> Lists => Set<MailingList>(); public DbSet<MailingList> Lists => Set<MailingList>();
public DbSet<Subscriber> Subscribers => Set<Subscriber>(); public DbSet<Subscription> Subscriptions => Set<Subscription>();
public DbSet<ListMember> ListMembers => Set<ListMember>();
public DbSet<EventInbox> EventsInbox => Set<EventInbox>(); public DbSet<EventInbox> EventsInbox => Set<EventInbox>();
public DbSet<Campaign> Campaigns => Set<Campaign>(); public DbSet<Campaign> Campaigns => Set<Campaign>();
public DbSet<SendJob> SendJobs => Set<SendJob>(); public DbSet<SendJob> SendJobs => Set<SendJob>();
@ -47,39 +46,22 @@ public sealed class SendEngineDbContext : DbContext
entity.HasOne<Tenant>().WithMany().HasForeignKey(e => e.TenantId).HasConstraintName("fk_lists_tenant"); entity.HasOne<Tenant>().WithMany().HasForeignKey(e => e.TenantId).HasConstraintName("fk_lists_tenant");
}); });
modelBuilder.Entity<Subscriber>(entity => modelBuilder.Entity<Subscription>(entity =>
{ {
entity.ToTable("subscribers"); entity.ToTable("subscriptions");
entity.HasKey(e => e.Id); entity.HasKey(e => e.Id);
entity.Property(e => e.Id).HasColumnName("id"); entity.Property(e => e.Id).HasColumnName("id");
entity.Property(e => e.TenantId).HasColumnName("tenant_id"); entity.Property(e => e.ListId).HasColumnName("list_id");
entity.Property(e => e.Email).HasColumnName("email").HasColumnType("citext"); entity.Property(e => e.Email).HasColumnName("email").HasColumnType("citext");
entity.Property(e => e.ExternalSubscriberId).HasColumnName("external_subscriber_id");
entity.Property(e => e.Status).HasColumnName("status"); entity.Property(e => e.Status).HasColumnName("status");
entity.Property(e => e.Preferences).HasColumnName("preferences").HasColumnType("jsonb"); entity.Property(e => e.Preferences).HasColumnName("preferences").HasColumnType("jsonb");
entity.Property(e => e.CreatedAt).HasColumnName("created_at"); entity.Property(e => e.CreatedAt).HasColumnName("created_at");
entity.Property(e => e.UpdatedAt).HasColumnName("updated_at"); entity.Property(e => e.UpdatedAt).HasColumnName("updated_at");
entity.HasIndex(e => e.TenantId).HasDatabaseName("idx_subscribers_tenant"); entity.HasIndex(e => e.ListId).HasDatabaseName("idx_subscriptions_list");
entity.HasIndex(e => new { e.TenantId, e.Email }).IsUnique(); entity.HasIndex(e => new { e.ListId, e.Email }).IsUnique();
entity.HasOne<Tenant>().WithMany().HasForeignKey(e => e.TenantId).HasConstraintName("fk_subscribers_tenant"); entity.HasIndex(e => e.ExternalSubscriberId).HasDatabaseName("idx_subscriptions_external_subscriber");
}); entity.HasOne<MailingList>().WithMany().HasForeignKey(e => e.ListId).HasConstraintName("fk_subscriptions_list");
modelBuilder.Entity<ListMember>(entity =>
{
entity.ToTable("list_members");
entity.HasKey(e => e.Id);
entity.Property(e => e.Id).HasColumnName("id");
entity.Property(e => e.TenantId).HasColumnName("tenant_id");
entity.Property(e => e.ListId).HasColumnName("list_id");
entity.Property(e => e.SubscriberId).HasColumnName("subscriber_id");
entity.Property(e => e.Status).HasColumnName("status");
entity.Property(e => e.CreatedAt).HasColumnName("created_at");
entity.Property(e => e.UpdatedAt).HasColumnName("updated_at");
entity.HasIndex(e => e.TenantId).HasDatabaseName("idx_list_members_tenant");
entity.HasIndex(e => e.ListId).HasDatabaseName("idx_list_members_list");
entity.HasIndex(e => new { e.TenantId, e.ListId, e.SubscriberId }).IsUnique();
entity.HasOne<Tenant>().WithMany().HasForeignKey(e => e.TenantId).HasConstraintName("fk_list_members_tenant");
entity.HasOne<MailingList>().WithMany().HasForeignKey(e => e.ListId).HasConstraintName("fk_list_members_list");
entity.HasOne<Subscriber>().WithMany().HasForeignKey(e => e.SubscriberId).HasConstraintName("fk_list_members_subscriber");
}); });
modelBuilder.Entity<EventInbox>(entity => modelBuilder.Entity<EventInbox>(entity =>

View File

@ -0,0 +1,20 @@
FROM mcr.microsoft.com/dotnet/sdk:8.0 AS build
WORKDIR /src
COPY ["Directory.Build.props", "./"]
COPY ["NuGet.Config", "./"]
COPY ["src/SendEngine.Domain/SendEngine.Domain.csproj", "src/SendEngine.Domain/"]
COPY ["src/SendEngine.Application/SendEngine.Application.csproj", "src/SendEngine.Application/"]
COPY ["src/SendEngine.Infrastructure/SendEngine.Infrastructure.csproj", "src/SendEngine.Infrastructure/"]
COPY ["src/SendEngine.Installer/SendEngine.Installer.csproj", "src/SendEngine.Installer/"]
RUN dotnet restore "src/SendEngine.Installer/SendEngine.Installer.csproj"
COPY . .
RUN dotnet publish "src/SendEngine.Installer/SendEngine.Installer.csproj" -c Release -o /app/publish /p:UseAppHost=false
FROM mcr.microsoft.com/dotnet/runtime:8.0 AS final
WORKDIR /app
COPY --from=build /app/publish .
ENTRYPOINT ["dotnet", "SendEngine.Installer.dll"]

View File

@ -4,6 +4,9 @@ using Microsoft.Extensions.DependencyInjection;
using SendEngine.Domain.Entities; using SendEngine.Domain.Entities;
using SendEngine.Infrastructure; using SendEngine.Infrastructure;
using SendEngine.Infrastructure.Data; using SendEngine.Infrastructure.Data;
using System.Net.Http.Headers;
using System.Net.Http.Json;
using System.Text.Json;
var command = args.Length > 0 ? args[0] : "migrate"; var command = args.Length > 0 ? args[0] : "migrate";
@ -12,7 +15,8 @@ if (command is "-h" or "--help" or "help")
Console.WriteLine("SendEngine Installer"); Console.WriteLine("SendEngine Installer");
Console.WriteLine("Usage:"); Console.WriteLine("Usage:");
Console.WriteLine(" dotnet run --project src/SendEngine.Installer -- migrate"); Console.WriteLine(" dotnet run --project src/SendEngine.Installer -- migrate");
Console.WriteLine(" dotnet run --project src/SendEngine.Installer -- add-webhook-client --tenant-id <uuid> --client-id <string> --name <string> --scopes <csv>"); Console.WriteLine(" dotnet run --project src/SendEngine.Installer -- ensure-tenant --tenant-id <uuid> [--tenant-name <string>]");
Console.WriteLine(" dotnet run --project src/SendEngine.Installer -- add-webhook-client --tenant-id <uuid> [--tenant-name <string>] --client-id <string> --name <string> --scopes <csv> [--upsert-member-center --mc-base-url <url> --mc-client-id <string> --mc-client-secret <string> [--mc-scope <scope>] [--mc-token-path <path>] [--mc-upsert-path <path>] [--mc-token-url <url>] [--mc-upsert-url <url>]]");
return; return;
} }
@ -38,6 +42,7 @@ if (command == "add-webhook-client")
{ {
var options = ParseOptions(args); var options = ParseOptions(args);
options.TryGetValue("tenant-id", out var tenantIdRaw); options.TryGetValue("tenant-id", out var tenantIdRaw);
options.TryGetValue("tenant-name", out var tenantName);
options.TryGetValue("client-id", out var clientId); options.TryGetValue("client-id", out var clientId);
options.TryGetValue("name", out var name); options.TryGetValue("name", out var name);
options.TryGetValue("scopes", out var scopesRaw); options.TryGetValue("scopes", out var scopesRaw);
@ -72,6 +77,20 @@ if (command == "add-webhook-client")
using var scope = provider.CreateScope(); using var scope = provider.CreateScope();
var db = scope.ServiceProvider.GetRequiredService<SendEngineDbContext>(); var db = scope.ServiceProvider.GetRequiredService<SendEngineDbContext>();
var tenant = await db.Tenants.FirstOrDefaultAsync(x => x.Id == tenantId);
if (tenant is null)
{
tenant = new Tenant
{
Id = tenantId,
Name = string.IsNullOrWhiteSpace(tenantName) ? $"tenant-{tenantId:N}" : tenantName.Trim(),
CreatedAt = DateTimeOffset.UtcNow
};
db.Tenants.Add(tenant);
await db.SaveChangesAsync();
Console.WriteLine($"Tenant created. tenant_id={tenant.Id}");
}
var exists = await db.AuthClients.AsNoTracking().AnyAsync(x => x.ClientId == clientId); var exists = await db.AuthClients.AsNoTracking().AnyAsync(x => x.ClientId == clientId);
if (exists) if (exists)
{ {
@ -98,6 +117,94 @@ if (command == "add-webhook-client")
Console.WriteLine($"tenant_id={entity.TenantId}"); Console.WriteLine($"tenant_id={entity.TenantId}");
Console.WriteLine($"client_id={entity.ClientId}"); Console.WriteLine($"client_id={entity.ClientId}");
Console.WriteLine($"scopes={string.Join(",", entity.Scopes)}"); Console.WriteLine($"scopes={string.Join(",", entity.Scopes)}");
if (options.ContainsKey("upsert-member-center"))
{
var mcSettings = ResolveMemberCenterUpsertSettings(options);
if (mcSettings is null)
{
Console.WriteLine("Missing required Member Center options for --upsert-member-center.");
Console.WriteLine("Required: --mc-base-url, --mc-client-id, --mc-client-secret");
Environment.Exit(1);
}
var token = await ResolveMemberCenterTokenAsync(mcSettings);
if (string.IsNullOrWhiteSpace(token))
{
Console.WriteLine("Member Center upsert failed: unable to get access token.");
Environment.Exit(1);
}
using var client = new HttpClient();
client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Bearer", token);
var upsertPayload = new
{
tenant_id = tenantId,
webhook_client_id = entity.Id
};
try
{
var response = await client.PostAsJsonAsync(mcSettings.UpsertWebhookClientUrl, upsertPayload);
var body = await response.Content.ReadAsStringAsync();
if (!response.IsSuccessStatusCode)
{
Console.WriteLine($"Member Center upsert failed. status={(int)response.StatusCode} body={Truncate(body, 1000)}");
Environment.Exit(1);
}
Console.WriteLine("Member Center webhook client mapping updated.");
Console.WriteLine($"member_center_tenant_id={tenantId}");
Console.WriteLine($"member_center_webhook_client_id={entity.Id}");
}
catch (Exception ex)
{
Console.WriteLine($"Member Center upsert request failed: {ex.Message}");
Environment.Exit(1);
}
}
return;
}
if (command == "ensure-tenant")
{
var options = ParseOptions(args);
options.TryGetValue("tenant-id", out var tenantIdRaw);
options.TryGetValue("tenant-name", out var tenantName);
if (string.IsNullOrWhiteSpace(tenantIdRaw))
{
Console.WriteLine("Missing required option: --tenant-id");
Environment.Exit(1);
}
if (!Guid.TryParse(tenantIdRaw, out var tenantId))
{
Console.WriteLine("Invalid --tenant-id, expected UUID.");
Environment.Exit(1);
}
using var scope = provider.CreateScope();
var db = scope.ServiceProvider.GetRequiredService<SendEngineDbContext>();
var tenant = await db.Tenants.FirstOrDefaultAsync(x => x.Id == tenantId);
if (tenant is null)
{
tenant = new Tenant
{
Id = tenantId,
Name = string.IsNullOrWhiteSpace(tenantName) ? $"tenant-{tenantId:N}" : tenantName.Trim(),
CreatedAt = DateTimeOffset.UtcNow
};
db.Tenants.Add(tenant);
await db.SaveChangesAsync();
Console.WriteLine($"Tenant created. tenant_id={tenant.Id}");
}
else
{
Console.WriteLine($"Tenant exists. tenant_id={tenant.Id}");
}
return; return;
} }
@ -127,3 +234,94 @@ static Dictionary<string, string> ParseOptions(string[] args)
return result; return result;
} }
static MemberCenterUpsertSettings? ResolveMemberCenterUpsertSettings(IReadOnlyDictionary<string, string> options)
{
options.TryGetValue("mc-base-url", out var baseUrl);
options.TryGetValue("mc-client-id", out var clientId);
options.TryGetValue("mc-client-secret", out var clientSecret);
options.TryGetValue("mc-scope", out var scope);
options.TryGetValue("mc-token-url", out var tokenUrl);
options.TryGetValue("mc-upsert-url", out var upsertUrl);
options.TryGetValue("mc-token-path", out var tokenPath);
options.TryGetValue("mc-upsert-path", out var upsertPath);
if (string.IsNullOrWhiteSpace(baseUrl) ||
string.IsNullOrWhiteSpace(clientId) ||
string.IsNullOrWhiteSpace(clientSecret))
{
return null;
}
scope = string.IsNullOrWhiteSpace(scope) ? "newsletter:events.write.global" : scope;
tokenPath = string.IsNullOrWhiteSpace(tokenPath) ? "/oauth/token" : tokenPath;
upsertPath = string.IsNullOrWhiteSpace(upsertPath) ? "/integrations/send-engine/webhook-clients/upsert" : upsertPath;
tokenUrl = string.IsNullOrWhiteSpace(tokenUrl) ? CombineUrl(baseUrl, tokenPath) : tokenUrl;
upsertUrl = string.IsNullOrWhiteSpace(upsertUrl) ? CombineUrl(baseUrl, upsertPath) : upsertUrl;
return new MemberCenterUpsertSettings(tokenUrl, upsertUrl, clientId, clientSecret, scope);
}
static async Task<string?> ResolveMemberCenterTokenAsync(MemberCenterUpsertSettings settings)
{
using var client = new HttpClient();
var form = new Dictionary<string, string>
{
["grant_type"] = "client_credentials",
["client_id"] = settings.ClientId,
["client_secret"] = settings.ClientSecret
};
if (!string.IsNullOrWhiteSpace(settings.Scope))
{
form["scope"] = settings.Scope;
}
try
{
var response = await client.PostAsync(settings.TokenUrl, new FormUrlEncodedContent(form));
var body = await response.Content.ReadAsStringAsync();
if (!response.IsSuccessStatusCode)
{
Console.WriteLine($"Member Center token request failed. status={(int)response.StatusCode} body={Truncate(body, 1000)}");
return null;
}
using var doc = JsonDocument.Parse(body);
if (doc.RootElement.TryGetProperty("access_token", out var tokenElement))
{
var token = tokenElement.GetString();
if (!string.IsNullOrWhiteSpace(token))
{
return token;
}
}
}
catch (Exception ex)
{
Console.WriteLine($"Member Center token request error: {ex.Message}");
}
return null;
}
static string CombineUrl(string baseUrl, string path)
{
return $"{baseUrl.TrimEnd('/')}/{path.TrimStart('/')}";
}
static string Truncate(string? input, int maxLen)
{
if (string.IsNullOrWhiteSpace(input))
{
return string.Empty;
}
return input.Length <= maxLen ? input : $"{input[..maxLen]}...(truncated)";
}
sealed record MemberCenterUpsertSettings(
string TokenUrl,
string UpsertWebhookClientUrl,
string ClientId,
string ClientSecret,
string Scope);