cluster4npu/cluster4npu_ui/README.md

# Cluster4NPU UI - Visual Pipeline Designer

A visual pipeline designer for creating parallel AI inference workflows using Kneron NPU dongles. Build complex multi-stage inference pipelines through an intuitive drag-and-drop interface without coding knowledge.

## Features

- **Visual Pipeline Design**: Drag-and-drop node-based interface using NodeGraphQt
- **Multi-Stage Pipelines**: Chain multiple AI models for complex workflows  
- **Real-time Performance Monitoring**: Live FPS, latency, and throughput tracking
- **Hardware Integration**: Automatic Kneron NPU dongle detection and management
- **Professional UI**: Three-panel layout with integrated configuration and monitoring
- **Pipeline Validation**: Real-time pipeline structure analysis and error detection

## Installation

This project uses [uv](https://github.com/astral-sh/uv) for fast Python package management.

```bash
# Install uv if you haven't already
curl -LsSf https://astral.sh/uv/install.sh | sh

# Create and activate virtual environment
uv venv
source .venv/bin/activate  # On Windows: .venv\Scripts\activate

# Install dependencies
uv pip install -r requirements.txt
```

### Requirements

**Python Dependencies:**
- PyQt5 (GUI framework)
- NodeGraphQt (visual node editor)
- OpenCV (image processing)
- NumPy (array operations)
- Kneron KP SDK (NPU communication)

**Hardware Requirements:**
- Kneron NPU dongles (KL520, KL720, KL1080)
- USB 3.0 ports for device connections
- Compatible firmware files (`fw_scpu.bin`, `fw_ncpu.bin`)
- Trained model files (`.nef` format)

## Quick Start

### Launching the Application

```bash
# Activate virtual environment
source .venv/bin/activate  # On Windows: .venv\Scripts\activate

# Launch the visual pipeline designer
python main.py
```

### Creating Your First Pipeline

1. **Start the Application**: Launch `main.py` to open the login/project manager
2. **Create New Project**: Click "Create New Pipeline" or load an existing `.mflow` file
3. **Design Pipeline**: Use the 3-panel interface:
   - **Left Panel**: Drag nodes from the template palette
   - **Middle Panel**: Connect nodes to build your pipeline flow
   - **Right Panel**: Configure node properties and monitor performance

### Basic Pipeline Structure

```
Input Node → Preprocess Node → Model Node → Postprocess Node → Output Node
```

**Node Types:**
- **Input Node**: Camera, video file, or image source
- **Preprocess Node**: Data transformation (resize, normalize, format conversion)
- **Model Node**: AI inference on Kneron NPU dongles
- **Postprocess Node**: Result processing (classification, detection formatting)  
- **Output Node**: Display, file output, or network streaming

### Visual Pipeline Design Workflow

1. **Node Placement**: Drag nodes from the left template palette
2. **Connection**: Connect nodes by dragging from output to input ports
3. **Configuration**: Select nodes and configure properties in the right panel
4. **Validation**: Real-time pipeline validation with stage counting
5. **Deployment**: Export configured pipeline for execution

## User Interface

### Three-Panel Layout

The main dashboard provides an integrated development environment with three main panels:

**Left Panel (25% width):**
- **Node Templates**: Drag-and-drop node palette
  - Input Node (camera, video, image sources)
  - Model Node (AI inference on NPU dongles)  
  - Preprocess Node (data transformation)
  - Postprocess Node (result processing)
  - Output Node (display, file, stream output)
- **Pipeline Operations**: Validation and management tools
- **Instructions**: Context-sensitive help

**Middle Panel (50% width):**
- **Visual Pipeline Editor**: NodeGraphQt-based visual editor
- **Real-time Validation**: Instant pipeline structure analysis
- **Node Connection**: Drag from output to input ports to connect nodes
- **Global Status Bar**: Shows stage count and pipeline statistics

**Right Panel (25% width):**
- **Properties Tab**: Node-specific configuration panels
- **Performance Tab**: Real-time performance monitoring and estimation
- **Dongles Tab**: Hardware device management and allocation

### Project Management

**Login/Startup Window:**
- Recent projects list with quick access
- Create new pipeline projects
- Load existing `.mflow` pipeline files
- Project location management

### Real-time Feedback

- **Stage Counting**: Automatic detection of pipeline stages
- **Connection Analysis**: Real-time validation of node connections
- **Error Highlighting**: Visual indicators for configuration issues
- **Performance Metrics**: Live FPS, latency, and throughput display

## Architecture

### Core Components

**Pipeline Analysis Engine (`core/pipeline.py`):**
- Automatic stage detection and validation
- Connection path analysis between nodes
- Real-time pipeline structure summarization
- Configuration export for deployment

**Node System (`core/nodes/`):**
- Extensible node architecture with type-specific properties
- Business logic separation from UI presentation
- Dynamic property validation and configuration panels

**Inference Engine (`core/functions/InferencePipeline.py`):**
- Multi-stage pipeline orchestration with thread-based processing
- Real-time performance monitoring and FPS calculation
- Inter-stage data flow and result aggregation

**Hardware Abstraction (`core/functions/Multidongle.py`):**
- Kneron NPU dongle management and auto-detection
- Multi-device support with load balancing
- Async inference processing with result queuing

### Data Flow

1. **Design Phase**: Visual pipeline creation using drag-and-drop interface
2. **Validation Phase**: Real-time analysis of pipeline structure and configuration
3. **Export Phase**: Generate executable configuration from visual design
4. **Execution Phase**: Deploy pipeline to hardware with performance monitoring

## File Formats

### Pipeline Files (`.mflow`)

JSON-based format storing:
- Node definitions and properties
- Connection relationships
- Stage configurations
- Export settings

### Hardware Configuration

- Firmware files: `fw_scpu.bin`, `fw_ncpu.bin`
- Model files: `.nef` format for Kneron NPUs
- Device mapping: USB port assignment to pipeline stages

## Performance Monitoring

### Real-time Metrics

- **FPS (Frames Per Second)**: Processing throughput
- **Latency**: End-to-end processing time
- **Stage Performance**: Per-stage processing statistics
- **Device Utilization**: NPU dongle usage monitoring

### Statistics Collection

- Pipeline input/output counts
- Processing time distributions
- Error rates and failure analysis
- Resource utilization tracking

## Testing and Validation

Run the test suite to verify functionality:

```bash
# Test core pipeline functionality
python tests/test_pipeline_editor.py

# Test UI components
python tests/test_ui_fixes.py

# Test integration
python tests/test_integration.py
```

## Troubleshooting

### Common Issues

**Node creation fails:**
- Verify NodeGraphQt installation and compatibility
- Check node template definitions in `core/nodes/`

**Pipeline validation errors:**
- Ensure all model nodes are connected between input and output
- Verify node property configurations are complete

**Hardware detection issues:**
- Check USB connections and dongles power
- Verify firmware files are accessible
- Ensure proper Kneron SDK installation

**Performance issues:**
- Monitor device utilization in Dongles tab
- Adjust queue sizes for throughput vs. latency tradeoffs
- Check for processing bottlenecks in stage statistics

## Development

### Project Structure

```
cluster4npu_ui/
├── main.py                 # Application entry point
├── config/                 # Configuration and theming
├── core/                   # Core processing engine
│   ├── functions/          # Inference and hardware abstraction
│   ├── nodes/             # Node type definitions
│   └── pipeline.py        # Pipeline analysis and validation
├── ui/                    # User interface components
│   ├── windows/           # Main windows (login, dashboard)
│   ├── components/        # Reusable UI widgets
│   └── dialogs/           # Modal dialogs
├── tests/                 # Test suite
└── resources/             # Assets and styling
```

### Contributing

1. Follow the TDD workflow defined in `CLAUDE.md`
2. Run tests before committing changes
3. Maintain the three-panel UI architecture
4. Document new node types and their properties

## License

This project is part of the Cluster4NPU ecosystem for parallel AI inference on Kneron NPU hardware.