kneo_marketplace/docs/frontend-media-storage-plan.md

Frontend Refresh and Media Storage Plan

Purpose

This plan defines the first major rebuild track for the storefront and media layer. The goal is to replace the current storefront theme while introducing a private S3-compatible media storage layer that can later take over nopCommerce product pictures and support a controlled production migration.

The current data volume is small, so the plan favors clear boundaries and reversible migration steps over heavy batch infrastructure.

Fixed Decisions

• The existing storefront theme can be abandoned for the public frontend rebuild.
• A new theme will be created instead of modifying DefaultClean in place.
• The first frontend implementation is desktop-first. Responsive behavior can be added later, but HTML and CSS structure should leave room for responsive breakpoints.
• The storefront rebuild is not only visual CSS work. It may require Razor view changes, model/factory changes, controller changes, admin settings, widget changes, and seed/config data.
• Product-related visuals should use existing product picture data where practical instead of being recreated as static theme images.
• Media objects must not expose raw S3, MinIO, or bucket URLs to browsers.
• AWS S3 will be used for development/test and production object storage.
• MinIO will be used in stage as an S3-compatible replacement.
• All browser-facing media URLs must go through the application, using a media proxy.
• CloudFront is intentionally out of scope because DNS is not controlled by the team.
• Existing production migration can use a planned maintenance window of roughly half a day to one day.

Target Architecture

Storage

Create an application-level media storage abstraction instead of mounting S3 as a filesystem.

The storage layer should support:

• AWS S3 for test/prod.
• MinIO for stage.
• Private buckets only.
• No public object ACLs.
• Configurable endpoint, bucket, region, access key, secret, and path-style addressing.

Proposed interface:

public interface IMediaStorage
{
    Task<MediaObjectInfo> PutAsync(MediaPutRequest request, CancellationToken cancellationToken = default);
    Task<MediaReadResult> GetAsync(string objectKey, CancellationToken cancellationToken = default);
    Task<MediaObjectInfo?> HeadAsync(string objectKey, CancellationToken cancellationToken = default);
    Task DeleteAsync(string objectKey, CancellationToken cancellationToken = default);
    Task<bool> ExistsAsync(string objectKey, CancellationToken cancellationToken = default);
}

Browser-Facing URLs

Media URLs should be stable application URLs, not object storage URLs.

Examples:

/media/assets/{assetId}/{seoName}
/media/pictures/{pictureId}/{size}/{seoName}
/media/pictures/{pictureId}/original/{seoName}

Optional compatibility route for existing rich editor HTML:

/images/uploaded/{**path}

This route can temporarily proxy legacy uploaded paths while old HTML is migrated.

Media Proxy

Add a media proxy controller responsible for reading from storage and streaming the response.

Required behavior:

• Stream object content without buffering entire files in memory.
• Return correct Content-Type.
• Return Content-Length when available.
• Return ETag and/or Last-Modified when available.
• Support conditional requests with 304 Not Modified.
• Apply browser cache headers.
• Keep private object storage credentials server-side only.

Later behavior:

• Support HTTP Range requests for video and large media.
• Add authorization rules if private customer/vendor media is introduced.
• Add image transformation variants if needed beyond nopCommerce thumbnail behavior.

Data Model Direction

Use explicit metadata tables rather than relying on physical paths inside HTML or S3 object keys alone.

Managed Media Assets

Add a new managed media asset model for non-product-picture assets such as homepage visuals, banners, rich editor images, icons, downloadable presentation media, and future marketing media.

Suggested fields:

• Id
• StorageProvider
• Bucket
• ObjectKey
• FileName
• SeoFileName
• MimeType
• FileSize
• Width
• Height
• Checksum
• AltText
• Title
• UsageType
• CreatedOnUtc
• UpdatedOnUtc
• Deleted

Product Pictures

Keep nopCommerce product picture concepts intact:

• Keep Picture.
• Keep ProductPicture.
• Keep existing picture metadata such as MIME type, SEO filename, alt, title, and display order.

Add storage mapping for picture binaries, either with a dedicated PictureStorage table or by reusing the managed media asset table with a product-picture usage type.

Suggested picture storage fields:

• PictureId
• StorageProvider
• Bucket
• ObjectKey
• OriginalFileName
• MimeType
• FileSize
• Width
• Height
• Checksum
• CreatedOnUtc

PictureBinary should remain untouched during initial rollout and migration dry runs. It can be cleared only after production validation and rollback risk is acceptable.

Phase 1: New Frontend Theme and Media Storage Foundation

This is the first major step.

Scope

• Create the new frontend theme.
• Build the initial IMediaStorage abstraction.
• Implement S3-compatible storage for AWS S3 and MinIO.
• Add media proxy routes.
• Add admin-managed media assets for assets currently handled as static files.
• Update new frontend views to use managed media asset URLs instead of hardcoded static files.
• Add or adjust application data needed by the new storefront, such as homepage section configuration, featured product selections, category/application links, and managed media references.
• Adjust public model factories/controllers only when the new frontend requires data that existing models do not expose.

Theme Work

• Create Themes/Kneo.
• Add a new theme.json.
• Add new frontend CSS/JS asset structure.
• Override only the required public views and partials.
• Keep DefaultClean available as a reference and fallback.
• Build the first version as a desktop-first layout matching the supplied design screenshots.
• Use semantic section/partial boundaries so mobile breakpoints can be added later without replacing the markup.
• Prioritize the main storefront paths:
  • Home page
  • Header/navigation
  • Footer
  • Category/listing pages
  • Product cards
  • Product detail page
  • Cart entry points and purchase CTA surfaces

Media Storage Work

• Add configuration for S3-compatible storage:
  • Provider
  • Endpoint
  • Region
  • Bucket
  • Access key
  • Secret key
  • Force path style
  • Public proxy base path
• Implement AWS S3/MinIO client code behind IMediaStorage.
• Add managed media metadata persistence.
• Add media proxy controller and routes.
• Add admin upload/list/edit/delete flow for managed assets.
• Add rich editor insertion path for new managed media assets.
• Add controlled ways to reference managed media from storefront sections, rather than hardcoding object keys or bucket URLs in Razor views.

Storefront Data Work

The redesigned home page and other new storefront surfaces may require data that the current theme does not model explicitly.

Possible data/config needs:

• Home hero copy and media references.
• Featured hardware/product list.
• Recommended books/product list.
• Application domain tiles and links.
• Section display order.
• Header navigation entries.
• CTA target URLs.

Implementation options, from simplest to most flexible:

• Use existing nopCommerce categories/products/manufacturers where the content naturally maps to catalog data.
• Use settings for small global values such as hero title, subtitle, and CTA links.
• Use a custom table/model for repeatable home page sections if the content needs admin management.
• Use widgets only when the content needs to be independently pluggable.

Avoid encoding business-editable homepage content directly in CSS or static HTML unless it is clearly temporary.

Static Asset Conversion

Convert frontend-owned non-CSS media from static files into managed media where practical.

Examples:

• Homepage hero images
• Banner images
• Landing/media blocks
• Editorial image assets
• Future video poster images

CSS-only assets can remain under the theme if they are purely presentational and not expected to change through admin workflows.

Out of Scope

• Existing PictureService replacement.
• Existing product picture migration.
• Bulk rewriting existing product descriptions.
• Deleting PictureBinary.
• CloudFront integration.

Acceptance Criteria

• New theme can be selected and renders core storefront pages.
• New frontend assets can be uploaded to S3/MinIO through the application.
• New frontend views render media through application proxy URLs.
• No browser-visible raw S3 or MinIO URLs.
• Stage can switch between AWS S3-compatible config and MinIO config.
• Existing product pictures still work through current nopCommerce behavior.
• Required home page data can be configured or seeded without editing Razor/CSS for normal content updates.

Phase 2: Apply Media Storage to Picture Service

Scope

Replace product picture binary storage/read behavior with the new media storage layer while keeping existing public and admin behavior as stable as possible.

Work Items

• Extend or replace IPictureService implementation.
• Make new product picture uploads write to S3/MinIO.
• Make GetPictureUrlAsync return media proxy URLs.
• Make original and generated thumbnail reads flow through the proxy.
• Decide whether thumbnails are:
  • generated on demand and stored in S3/MinIO, or
  • generated during upload/update and stored as variants.
• Preserve existing product picture metadata and product associations.
• Keep compatibility for pictures still stored in DB during rollout.

Compatibility Strategy

During rollout, product pictures may exist in two states:

• Legacy DB-backed picture: PictureBinary.BinaryData exists and no storage mapping exists.
• New storage-backed picture: storage mapping exists and object is in S3/MinIO.

The picture service should support both until migration is complete.

Acceptance Criteria

• New product image uploads are stored in S3/MinIO.
• Existing DB-backed product images still render.
• Product image URLs are application proxy URLs.
• Admin product picture add/edit/delete behavior still works.
• Category pages, product detail pages, carts, orders, and search thumbnails render correctly.

Phase 3: Migration Plan and Migration Tooling

Scope

Build and test tools to migrate existing DB-backed product pictures and legacy rich editor uploads into the new storage system.

Product Picture Migration

Migration tool responsibilities:

• Scan Picture records.
• Read current binary data from PictureBinary.
• Generate deterministic object keys.
• Upload originals to S3/MinIO.
• Create storage mapping records.
• Verify object exists and size/checksum matches.
• Mark migration status.
• Produce a migration report.

Suggested object key format:

pictures/{pictureId}/original/{safeSeoName-or-pictureId}.{extension}
pictures/{pictureId}/thumbs/{size}/{safeSeoName-or-pictureId}.{extension}

Thumbs may be generated lazily after migration unless performance testing shows preload is needed.

Rich Editor Upload Migration

Migration tool responsibilities:

• Scan product descriptions and other rich editor fields for /images/uploaded/....
• Resolve matching local files from the current production filesystem.
• Upload files to managed media storage.
• Create managed media asset records.
• Optionally rewrite HTML to /media/assets/....
• Produce a report of missing local files and rewritten references.

If direct rewrite is risky, keep /images/uploaded/{**path} as a legacy proxy route during the transition.

Production Cutover

Expected maintenance-window flow:

1. Enable maintenance mode.
2. Freeze product/media edits.
3. Run product picture migration.
4. Run rich editor media migration.
5. Validate migration report.
6. Switch production config to storage-backed picture service.
7. Clear application cache.
8. Run smoke tests on critical pages.
9. Disable maintenance mode.

Rollback Strategy

• Do not delete or clear PictureBinary during initial production cutover.
• Keep legacy local uploaded files until product verification is complete.
• Keep DB and filesystem backups from immediately before migration.
• If needed, switch config back to DB-backed picture behavior and legacy static uploaded files.

Acceptance Criteria

• All existing product pictures have storage mapping records.
• All migrated objects exist in S3/MinIO.
• Product pages render after clearing browser/application cache.
• Missing rich editor files are reported explicitly.
• No raw object storage URLs appear in rendered storefront HTML.

Risks and Controls

Web Server Bandwidth

Without CloudFront, all media flows through the application.

Controls:

• Stream responses.
• Use browser caching.
• Use conditional GET.
• Add Range support before serving large videos.
• Keep image sizes reasonable.

Cache Invalidation

Stable URLs can cause stale browser caches when media is replaced.

Controls:

• Prefer immutable object keys for replaced files.
• Include version or content hash in proxy URLs when needed.
• Use ETag and Last-Modified.

Legacy HTML References

Existing rich editor HTML may point to missing local files.

Controls:

• Build a scanner before production migration.
• Report missing files before cutover.
• Keep a compatibility proxy route.

S3-Compatible Differences

AWS S3 and MinIO differ in endpoint style, path style, metadata, and local TLS behavior.

Controls:

• Keep all provider differences inside S3MediaStorage.
• Test against MinIO in stage and AWS S3 in test.
• Avoid provider-specific URLs in persisted HTML.

Suggested Implementation Order

1. Add configuration and IMediaStorage.
2. Add S3/MinIO implementation.
3. Add media metadata tables and migrations.
4. Add media proxy read endpoint.
5. Add admin-managed media upload/list/edit flow.
6. Create Themes/Kneo.
7. Convert new frontend non-CSS media to managed assets.
8. Integrate managed assets with rich editor for new content.
9. Add storage-backed picture service compatibility.
10. Build migration scanner/report.
11. Build product picture migration tool.
12. Build rich editor media migration tool.
13. Run dry runs on stage.
14. Execute production migration during maintenance window.

Deployment Memo

This section tracks deploy-time work that should not be forgotten as the project evolves.

Phase 1 Deploy Checklist

• Add S3/MinIO configuration values for the target environment.
• Create the target bucket in AWS S3 or MinIO.
• Keep the bucket private.
• Create or provision application credentials with only the required bucket permissions.
• Confirm application config does not expose raw S3/MinIO URLs to rendered HTML.
• Run database migrations for managed media metadata.
• Seed required storefront settings/data:
  • active theme set to Kneo
  • home hero configuration
  • featured product/book selections
  • application tiles
  • navigation entries if they are data-driven
• Upload required non-product homepage media into managed media storage.
• Verify media proxy can read from S3/MinIO in the deployed environment.
• Verify cache headers and conditional GET behavior on media responses.
• Confirm existing product pictures still render through the legacy picture path.
• Clear application cache after theme/settings changes.
• Smoke test:
  • home page
  • category/listing page
  • product detail page
  • cart entry point
  • admin media upload/list

Phase 2 Deploy Checklist

• Run database migrations for picture storage mapping.
• Enable storage-backed picture service in the target environment.
• Confirm new product picture uploads write to S3/MinIO.
• Confirm legacy DB-backed product pictures still render.
• Verify generated thumbnail behavior.
• Clear picture/model caches.
• Smoke test:
  • product image upload in admin
  • product detail gallery
  • product card thumbnails
  • cart/order thumbnails

Phase 3 Migration Checklist

• Take database backup.
• Back up legacy local media directories, especially wwwroot/images/uploaded.
• Enable maintenance mode.
• Freeze product and media edits.
• Run migration dry-run/report first if possible.
• Run product picture migration.
• Run rich editor media migration.
• Verify migration counts and missing-file report.
• Switch production config to storage-backed picture service.
• Clear application and CDN/browser-relevant caches where applicable.
• Smoke test critical storefront and admin pages.
• Keep PictureBinary data and legacy uploaded files until post-cutover validation is complete.
• Disable maintenance mode.