kneo_marketplace/docs/frontend-media-storage-plan.md

# Frontend Refresh and Media Storage Plan

## Purpose

This plan defines the first major rebuild track for the storefront and media layer. The goal is to replace the current storefront theme while introducing a private S3-compatible media storage layer that can later take over nopCommerce product pictures and support a controlled production migration.

The current data volume is small, so the plan favors clear boundaries and reversible migration steps over heavy batch infrastructure.

## Fixed Decisions

- The existing storefront theme can be abandoned for the public frontend rebuild.
- A new theme will be created instead of modifying `DefaultClean` in place.
- The first frontend implementation is desktop-first. Responsive behavior can be added later, but HTML and CSS structure should leave room for responsive breakpoints.
- The storefront rebuild is not only visual CSS work. It may require Razor view changes, model/factory changes, controller changes, admin settings, widget changes, and seed/config data.
- Product-related visuals should use existing product picture data where practical instead of being recreated as static theme images.
- Media objects must not expose raw S3, MinIO, or bucket URLs to browsers.
- AWS S3 will be used for development/test and production object storage.
- MinIO will be used in stage as an S3-compatible replacement.
- All browser-facing media URLs must go through the application, using a media proxy.
- CloudFront is intentionally out of scope because DNS is not controlled by the team.
- Existing production migration can use a planned maintenance window of roughly half a day to one day.

## Target Architecture

### Storage

Create an application-level media storage abstraction instead of mounting S3 as a filesystem.

The storage layer should support:

- AWS S3 for test/prod.
- MinIO for stage.
- Private buckets only.
- No public object ACLs.
- Configurable endpoint, bucket, region, access key, secret, and path-style addressing.

Proposed interface:

```csharp
public interface IMediaStorage
{
    Task<MediaObjectInfo> PutAsync(MediaPutRequest request, CancellationToken cancellationToken = default);
    Task<MediaReadResult> GetAsync(string objectKey, CancellationToken cancellationToken = default);
    Task<MediaObjectInfo?> HeadAsync(string objectKey, CancellationToken cancellationToken = default);
    Task DeleteAsync(string objectKey, CancellationToken cancellationToken = default);
    Task<bool> ExistsAsync(string objectKey, CancellationToken cancellationToken = default);
}
```

### Browser-Facing URLs

Media URLs should be stable application URLs, not object storage URLs.

Examples:

```text
/media/assets/{assetId}/{seoName}
/media/pictures/{pictureId}/{size}/{seoName}
/media/pictures/{pictureId}/original/{seoName}
```

Optional compatibility route for existing rich editor HTML:

```text
/images/uploaded/{**path}
```

This route can temporarily proxy legacy uploaded paths while old HTML is migrated.

### Media Proxy

Add a media proxy controller responsible for reading from storage and streaming the response.

Required behavior:

- Stream object content without buffering entire files in memory.
- Return correct `Content-Type`.
- Return `Content-Length` when available.
- Return `ETag` and/or `Last-Modified` when available.
- Support conditional requests with `304 Not Modified`.
- Apply browser cache headers.
- Keep private object storage credentials server-side only.

Later behavior:

- Support HTTP `Range` requests for video and large media.
- Add authorization rules if private customer/vendor media is introduced.
- Add image transformation variants if needed beyond nopCommerce thumbnail behavior.

## Data Model Direction

Use explicit metadata tables rather than relying on physical paths inside HTML or S3 object keys alone.

### Managed Media Assets

Add a new managed media asset model for non-product-picture assets such as homepage visuals, banners, rich editor images, icons, downloadable presentation media, and future marketing media.

Suggested fields:

- `Id`
- `StorageProvider`
- `Bucket`
- `ObjectKey`
- `FileName`
- `SeoFileName`
- `MimeType`
- `FileSize`
- `Width`
- `Height`
- `Checksum`
- `AltText`
- `Title`
- `UsageType`
- `CreatedOnUtc`
- `UpdatedOnUtc`
- `Deleted`

### Product Pictures

Keep nopCommerce product picture concepts intact:

- Keep `Picture`.
- Keep `ProductPicture`.
- Keep existing picture metadata such as MIME type, SEO filename, alt, title, and display order.

Add storage mapping for picture binaries, either with a dedicated `PictureStorage` table or by reusing the managed media asset table with a product-picture usage type.

Suggested picture storage fields:

- `PictureId`
- `StorageProvider`
- `Bucket`
- `ObjectKey`
- `OriginalFileName`
- `MimeType`
- `FileSize`
- `Width`
- `Height`
- `Checksum`
- `CreatedOnUtc`

`PictureBinary` should remain untouched during initial rollout and migration dry runs. It can be cleared only after production validation and rollback risk is acceptable.

## Phase 1: New Frontend Theme and Media Storage Foundation

This is the first major step.

### Scope

- Create the new frontend theme.
- Build the initial `IMediaStorage` abstraction.
- Implement S3-compatible storage for AWS S3 and MinIO.
- Add media proxy routes.
- Add admin-managed media assets for assets currently handled as static files.
- Update new frontend views to use managed media asset URLs instead of hardcoded static files.
- Add or adjust application data needed by the new storefront, such as homepage section configuration, featured product selections, category/application links, and managed media references.
- Adjust public model factories/controllers only when the new frontend requires data that existing models do not expose.

### Theme Work

- Create `Themes/Kneo`.
- Add a new `theme.json`.
- Add new frontend CSS/JS asset structure.
- Override only the required public views and partials.
- Keep `DefaultClean` available as a reference and fallback.
- Build the first version as a desktop-first layout matching the supplied design screenshots.
- Use semantic section/partial boundaries so mobile breakpoints can be added later without replacing the markup.
- Prioritize the main storefront paths:
  - Home page
  - Header/navigation
  - Footer
  - Category/listing pages
  - Product cards
  - Product detail page
  - Cart entry points and purchase CTA surfaces

### Media Storage Work

- Add configuration for S3-compatible storage:
  - Provider
  - Endpoint
  - Region
  - Bucket
  - Access key
  - Secret key
  - Force path style
  - Public proxy base path
- Implement AWS S3/MinIO client code behind `IMediaStorage`.
- Add managed media metadata persistence.
- Add media proxy controller and routes.
- Add admin upload/list/edit/delete flow for managed assets.
- Add rich editor insertion path for new managed media assets.
- Add controlled ways to reference managed media from storefront sections, rather than hardcoding object keys or bucket URLs in Razor views.

### Storefront Data Work

The redesigned home page and other new storefront surfaces may require data that the current theme does not model explicitly.

Possible data/config needs:

- Home hero copy and media references.
- Featured hardware/product list.
- Recommended books/product list.
- Application domain tiles and links.
- Section display order.
- Header navigation entries.
- CTA target URLs.

Implementation options, from simplest to most flexible:

- Use existing nopCommerce categories/products/manufacturers where the content naturally maps to catalog data.
- Use settings for small global values such as hero title, subtitle, and CTA links.
- Use a custom table/model for repeatable home page sections if the content needs admin management.
- Use widgets only when the content needs to be independently pluggable.

Avoid encoding business-editable homepage content directly in CSS or static HTML unless it is clearly temporary.

### Static Asset Conversion

Convert frontend-owned non-CSS media from static files into managed media where practical.

Examples:

- Homepage hero images
- Banner images
- Landing/media blocks
- Editorial image assets
- Future video poster images

CSS-only assets can remain under the theme if they are purely presentational and not expected to change through admin workflows.

### Out of Scope

- Existing `PictureService` replacement.
- Existing product picture migration.
- Bulk rewriting existing product descriptions.
- Deleting `PictureBinary`.
- CloudFront integration.

### Acceptance Criteria

- New theme can be selected and renders core storefront pages.
- New frontend assets can be uploaded to S3/MinIO through the application.
- New frontend views render media through application proxy URLs.
- No browser-visible raw S3 or MinIO URLs.
- Stage can switch between AWS S3-compatible config and MinIO config.
- Existing product pictures still work through current nopCommerce behavior.
- Required home page data can be configured or seeded without editing Razor/CSS for normal content updates.

## Phase 2: Apply Media Storage to Picture Service

### Scope

Replace product picture binary storage/read behavior with the new media storage layer while keeping existing public and admin behavior as stable as possible.

### Work Items

- Extend or replace `IPictureService` implementation.
- Make new product picture uploads write to S3/MinIO.
- Make `GetPictureUrlAsync` return media proxy URLs.
- Make original and generated thumbnail reads flow through the proxy.
- Decide whether thumbnails are:
  - generated on demand and stored in S3/MinIO, or
  - generated during upload/update and stored as variants.
- Preserve existing product picture metadata and product associations.
- Keep compatibility for pictures still stored in DB during rollout.

### Compatibility Strategy

During rollout, product pictures may exist in two states:

- Legacy DB-backed picture: `PictureBinary.BinaryData` exists and no storage mapping exists.
- New storage-backed picture: storage mapping exists and object is in S3/MinIO.

The picture service should support both until migration is complete.

### Acceptance Criteria

- New product image uploads are stored in S3/MinIO.
- Existing DB-backed product images still render.
- Product image URLs are application proxy URLs.
- Admin product picture add/edit/delete behavior still works.
- Category pages, product detail pages, carts, orders, and search thumbnails render correctly.

## Phase 3: Migration Plan and Migration Tooling

### Scope

Build and test tools to migrate existing DB-backed product pictures and legacy rich editor uploads into the new storage system.

### Product Picture Migration

Migration tool responsibilities:

- Scan `Picture` records.
- Read current binary data from `PictureBinary`.
- Generate deterministic object keys.
- Upload originals to S3/MinIO.
- Create storage mapping records.
- Verify object exists and size/checksum matches.
- Mark migration status.
- Produce a migration report.

Suggested object key format:

```text
pictures/{pictureId}/original/{safeSeoName-or-pictureId}.{extension}
pictures/{pictureId}/thumbs/{size}/{safeSeoName-or-pictureId}.{extension}
```

Thumbs may be generated lazily after migration unless performance testing shows preload is needed.

### Rich Editor Upload Migration

Migration tool responsibilities:

- Scan product descriptions and other rich editor fields for `/images/uploaded/...`.
- Resolve matching local files from the current production filesystem.
- Upload files to managed media storage.
- Create managed media asset records.
- Optionally rewrite HTML to `/media/assets/...`.
- Produce a report of missing local files and rewritten references.

If direct rewrite is risky, keep `/images/uploaded/{**path}` as a legacy proxy route during the transition.

### Production Cutover

Expected maintenance-window flow:

1. Enable maintenance mode.
2. Freeze product/media edits.
3. Run product picture migration.
4. Run rich editor media migration.
5. Validate migration report.
6. Switch production config to storage-backed picture service.
7. Clear application cache.
8. Run smoke tests on critical pages.
9. Disable maintenance mode.

### Rollback Strategy

- Do not delete or clear `PictureBinary` during initial production cutover.
- Keep legacy local uploaded files until product verification is complete.
- Keep DB and filesystem backups from immediately before migration.
- If needed, switch config back to DB-backed picture behavior and legacy static uploaded files.

### Acceptance Criteria

- All existing product pictures have storage mapping records.
- All migrated objects exist in S3/MinIO.
- Product pages render after clearing browser/application cache.
- Missing rich editor files are reported explicitly.
- No raw object storage URLs appear in rendered storefront HTML.

## Risks and Controls

### Web Server Bandwidth

Without CloudFront, all media flows through the application.

Controls:

- Stream responses.
- Use browser caching.
- Use conditional GET.
- Add Range support before serving large videos.
- Keep image sizes reasonable.

### Cache Invalidation

Stable URLs can cause stale browser caches when media is replaced.

Controls:

- Prefer immutable object keys for replaced files.
- Include version or content hash in proxy URLs when needed.
- Use `ETag` and `Last-Modified`.

### Legacy HTML References

Existing rich editor HTML may point to missing local files.

Controls:

- Build a scanner before production migration.
- Report missing files before cutover.
- Keep a compatibility proxy route.

### S3-Compatible Differences

AWS S3 and MinIO differ in endpoint style, path style, metadata, and local TLS behavior.

Controls:

- Keep all provider differences inside `S3MediaStorage`.
- Test against MinIO in stage and AWS S3 in test.
- Avoid provider-specific URLs in persisted HTML.

## Suggested Implementation Order

1. Add configuration and `IMediaStorage`.
2. Add S3/MinIO implementation.
3. Add media metadata tables and migrations.
4. Add media proxy read endpoint.
5. Add admin-managed media upload/list/edit flow.
6. Create `Themes/Kneo`.
7. Convert new frontend non-CSS media to managed assets.
8. Integrate managed assets with rich editor for new content.
9. Add storage-backed picture service compatibility.
10. Build migration scanner/report.
11. Build product picture migration tool.
12. Build rich editor media migration tool.
13. Run dry runs on stage.
14. Execute production migration during maintenance window.

## Deployment Memo

This section tracks deploy-time work that should not be forgotten as the project evolves.

### Phase 1 Deploy Checklist

- Add S3/MinIO configuration values for the target environment.
- Create the target bucket in AWS S3 or MinIO.
- Keep the bucket private.
- Create or provision application credentials with only the required bucket permissions.
- Confirm application config does not expose raw S3/MinIO URLs to rendered HTML.
- Run database migrations for managed media metadata.
- Seed required storefront settings/data:
  - active theme set to `Kneo`
  - home hero configuration
  - featured product/book selections
  - application tiles
  - navigation entries if they are data-driven
- Upload required non-product homepage media into managed media storage.
- Verify media proxy can read from S3/MinIO in the deployed environment.
- Verify cache headers and conditional GET behavior on media responses.
- Confirm existing product pictures still render through the legacy picture path.
- Clear application cache after theme/settings changes.
- Smoke test:
  - home page
  - category/listing page
  - product detail page
  - cart entry point
  - admin media upload/list

### Phase 2 Deploy Checklist

- Run database migrations for picture storage mapping.
- Enable storage-backed picture service in the target environment.
- Confirm new product picture uploads write to S3/MinIO.
- Confirm legacy DB-backed product pictures still render.
- Verify generated thumbnail behavior.
- Clear picture/model caches.
- Smoke test:
  - product image upload in admin
  - product detail gallery
  - product card thumbnails
  - cart/order thumbnails

### Phase 3 Migration Checklist

- Take database backup.
- Back up legacy local media directories, especially `wwwroot/images/uploaded`.
- Enable maintenance mode.
- Freeze product and media edits.
- Run migration dry-run/report first if possible.
- Run product picture migration.
- Run rich editor media migration.
- Verify migration counts and missing-file report.
- Switch production config to storage-backed picture service.
- Clear application and CDN/browser-relevant caches where applicable.
- Smoke test critical storefront and admin pages.
- Keep `PictureBinary` data and legacy uploaded files until post-cutover validation is complete.
- Disable maintenance mode.