cluster4npu/cluster4npu_ui

Cluster4NPU UI - Visual Pipeline Designer

A visual pipeline designer for creating parallel AI inference workflows using Kneron NPU dongles. Build complex multi-stage inference pipelines through an intuitive drag-and-drop interface without coding knowledge.

Features

• Visual Pipeline Design: Drag-and-drop node-based interface using NodeGraphQt
• Multi-Stage Pipelines: Chain multiple AI models for complex workflows
• Real-time Performance Monitoring: Live FPS, latency, and throughput tracking
• Hardware Integration: Automatic Kneron NPU dongle detection and management
• Professional UI: Three-panel layout with integrated configuration and monitoring
• Pipeline Validation: Real-time pipeline structure analysis and error detection

Installation

This project uses uv for fast Python package management.

# Install uv if you haven't already
curl -LsSf https://astral.sh/uv/install.sh | sh

# Create and activate virtual environment
uv venv
source .venv/bin/activate  # On Windows: .venv\Scripts\activate

# Install dependencies
uv pip install -r requirements.txt

Requirements

Python Dependencies:

• PyQt5 (GUI framework)
• NodeGraphQt (visual node editor)
• OpenCV (image processing)
• NumPy (array operations)
• Kneron KP SDK (NPU communication)

Hardware Requirements:

• Kneron NPU dongles (KL520, KL720, KL1080)
• USB 3.0 ports for device connections
• Compatible firmware files (fw_scpu.bin, fw_ncpu.bin)
• Trained model files (.nef format)

Quick Start

Launching the Application

# Activate virtual environment
source .venv/bin/activate  # On Windows: .venv\Scripts\activate

# Launch the visual pipeline designer
python main.py

Creating Your First Pipeline

1. Start the Application: Launch main.py to open the login/project manager
2. Create New Project: Click "Create New Pipeline" or load an existing .mflow file
3. Design Pipeline: Use the 3-panel interface:
   • Left Panel: Drag nodes from the template palette
   • Middle Panel: Connect nodes to build your pipeline flow
   • Right Panel: Configure node properties and monitor performance

Basic Pipeline Structure

Input Node → Preprocess Node → Model Node → Postprocess Node → Output Node

Node Types:

• Input Node: Camera, video file, or image source
• Preprocess Node: Data transformation (resize, normalize, format conversion)
• Model Node: AI inference on Kneron NPU dongles
• Postprocess Node: Result processing (classification, detection formatting)
• Output Node: Display, file output, or network streaming

Visual Pipeline Design Workflow

1. Node Placement: Drag nodes from the left template palette
2. Connection: Connect nodes by dragging from output to input ports
3. Configuration: Select nodes and configure properties in the right panel
4. Validation: Real-time pipeline validation with stage counting
5. Deployment: Export configured pipeline for execution

User Interface

Three-Panel Layout

The main dashboard provides an integrated development environment with three main panels:

Left Panel (25% width):

• Node Templates: Drag-and-drop node palette
  • Input Node (camera, video, image sources)
  • Model Node (AI inference on NPU dongles)
  • Preprocess Node (data transformation)
  • Postprocess Node (result processing)
  • Output Node (display, file, stream output)
• Pipeline Operations: Validation and management tools
• Instructions: Context-sensitive help

Middle Panel (50% width):

• Visual Pipeline Editor: NodeGraphQt-based visual editor
• Real-time Validation: Instant pipeline structure analysis
• Node Connection: Drag from output to input ports to connect nodes
• Global Status Bar: Shows stage count and pipeline statistics

Right Panel (25% width):

• Properties Tab: Node-specific configuration panels
• Performance Tab: Real-time performance monitoring and estimation
• Dongles Tab: Hardware device management and allocation

Project Management

Login/Startup Window:

• Recent projects list with quick access
• Create new pipeline projects
• Load existing .mflow pipeline files
• Project location management

Real-time Feedback

• Stage Counting: Automatic detection of pipeline stages
• Connection Analysis: Real-time validation of node connections
• Error Highlighting: Visual indicators for configuration issues
• Performance Metrics: Live FPS, latency, and throughput display

Architecture

Core Components

Pipeline Analysis Engine (core/pipeline.py):

• Automatic stage detection and validation
• Connection path analysis between nodes
• Real-time pipeline structure summarization
• Configuration export for deployment

Node System (core/nodes/):

• Extensible node architecture with type-specific properties
• Business logic separation from UI presentation
• Dynamic property validation and configuration panels

Inference Engine (core/functions/InferencePipeline.py):

• Multi-stage pipeline orchestration with thread-based processing
• Real-time performance monitoring and FPS calculation
• Inter-stage data flow and result aggregation

Hardware Abstraction (core/functions/Multidongle.py):

• Kneron NPU dongle management and auto-detection
• Multi-device support with load balancing
• Async inference processing with result queuing

Data Flow

1. Design Phase: Visual pipeline creation using drag-and-drop interface
2. Validation Phase: Real-time analysis of pipeline structure and configuration
3. Export Phase: Generate executable configuration from visual design
4. Execution Phase: Deploy pipeline to hardware with performance monitoring

File Formats

Pipeline Files (.mflow)

JSON-based format storing:

• Node definitions and properties
• Connection relationships
• Stage configurations
• Export settings

Hardware Configuration

• Firmware files: fw_scpu.bin, fw_ncpu.bin
• Model files: .nef format for Kneron NPUs
• Device mapping: USB port assignment to pipeline stages

Performance Monitoring

Real-time Metrics

• FPS (Frames Per Second): Processing throughput
• Latency: End-to-end processing time
• Stage Performance: Per-stage processing statistics
• Device Utilization: NPU dongle usage monitoring

Statistics Collection

• Pipeline input/output counts
• Processing time distributions
• Error rates and failure analysis
• Resource utilization tracking

Testing and Validation

Run the test suite to verify functionality:

# Test core pipeline functionality
python tests/test_pipeline_editor.py

# Test UI components
python tests/test_ui_fixes.py

# Test integration
python tests/test_integration.py

Troubleshooting

Common Issues

Node creation fails:

• Verify NodeGraphQt installation and compatibility
• Check node template definitions in core/nodes/

Pipeline validation errors:

• Ensure all model nodes are connected between input and output
• Verify node property configurations are complete

Hardware detection issues:

• Check USB connections and dongles power
• Verify firmware files are accessible
• Ensure proper Kneron SDK installation

Performance issues:

• Monitor device utilization in Dongles tab
• Adjust queue sizes for throughput vs. latency tradeoffs
• Check for processing bottlenecks in stage statistics

Development

Project Structure

cluster4npu_ui/
├── main.py                 # Application entry point
├── config/                 # Configuration and theming
├── core/                   # Core processing engine
│   ├── functions/          # Inference and hardware abstraction
│   ├── nodes/             # Node type definitions
│   └── pipeline.py        # Pipeline analysis and validation
├── ui/                    # User interface components
│   ├── windows/           # Main windows (login, dashboard)
│   ├── components/        # Reusable UI widgets
│   └── dialogs/           # Modal dialogs
├── tests/                 # Test suite
└── resources/             # Assets and styling

Contributing

1. Follow the TDD workflow defined in CLAUDE.md
2. Run tests before committing changes
3. Maintain the three-panel UI architecture
4. Document new node types and their properties

License

This project is part of the Cluster4NPU ecosystem for parallel AI inference on Kneron NPU hardware.