For help on running internal solutions, you can refer to the corresponding solution README for more details. Some general details are outlined in this section.

Running internal solutions

Clone the solution under the app directory.

cd app && git clone git@gitlab.kneron.tw:modelshare/solution_xxx.git

Follow the steps outlined in the solution README to intialize the necessary submodules and models.
Return to the root folder of the E2E simulator and test run the solution using the provided test folder. The following is the general command to run, where solution_xxx is the cloned solution and everything else is the same as is.

python3 simulator.py app/solution_xxx app/solution_xxx/alg/golden application -d -j app/solution_xxx/run_config.json

General tips:

app/solution_xxx/alg/golden may be substituted for your own image folder
-d enables print output to the terminal
You may need to checkout other versions of the solution if you want to run, e.g. sys520
- When doing so, you will need to change the -p option to the corresponding version
Take a look at the run_config.json for some of the parameters used to run the solution.
If you would like to run binary input files, you will need to add -i binary to the command. It defaults to image inputs.
- If you need to supply image size and format with the binary inputs, look at this section for more info.
If the solutions fail, it is probably because it is not complete yet or environment issues. Please check that you have all of the E2E Simulator dependencies as well as the specific solution depedencies.

Binary information

If you are using binary inputs and need to supply information into flow.py, you may provide the information through an input JSON file.

The size key should be a list of lists, where the inner lists are of the format [w, h], indicating the binary input sizes. The format key should be a list of strings, where the valid strings are "RGB565", "NIR888", "YUV422", and "Depth". Each value should be a list for each different value that you need to access inside your flow.py.

For specifying the order of images if you are using fusion input, you can modify the sequence key. This will look for files that contain the given strings in that order. Otherwise, the default order will look for RGB then NIR.

To specify the specific file order for your fusion input, you can modify the file_type key. This will look for files with that extension along with the provided sequence in that order. Otherwise, the default order will look for png and bin.

{
    "size": [[640, 480], [1080, 720], [200, 200], [480, 640]],
    "format": ["RGB565", "NIR888", "YUV422", "Depth"],
    "sequence": ["NIR", "RGB"],
    "file_type": ["bin", "png", "wav"]
}

You can then pass this JSON file through the command line option using -b and the path to your JSON file. Do note that this will only work if you specify the binary file inputs using -i binary. Note: sequence will be used regardless if you specify fusion and it is found in the JSON file.

In your flow.py, you can then access this data using user_config["bin_input"]["size"][index] and user_config["bin_input"]["format"][index] to grab the corresponding information that you need. You may add more keys if necessary.

There is an example under app/template_app/example_bin_input.json to use as reference.

Compiling new C code

Here is a general guideline of files that will be modified when adding a new postprocess or preprocess C function under c_interface. All of the following file paths will be relative to c_interface.

Add the postprocess file to src/postprocess or preprocess file to src/preprocess.
Add the function call to src/postprocess/post_processing.h or src/preprocess/pre_processing.h.
Add any result structure changes to include/model_res.h. You will need to create a corresponding class in Python under wrappers.py.
Add a C wrapper call to src/wrappers.c. This is where extra memory initialization should be done.
Add a Python file under src_python. This should include the function call that the system runner will use and a Python wrapper function that calls the C function. You may look at any of the existing files for reference.
Add your new C file to CMakeLists.txt.

If you do modify the C code under c_interface, you will need to recompile the shared libraries used in the E2E Simulator. To do so, run the following commands.

cd c_interface
mkdir build
cd build
cmake ..
make

The necessary files will be automatically copied into their specific locations, so no further actions will be needed.

Calling E2E function wrapper

We have provided a function wrapper for a simpler method to running a solution. This will involve a Python function call rather than a terminal command with different arguments. There are two required arguments and one optional argument:

solution_path: String path to the solution.
run_config: String path to the JSON file holding the configurations to run the solution.
image_directory (optional): String path to the input image directory. This will only be needed if the
    run_config does not have an input image list JSON specified.

Create another file in the same directory as simulator.py (base folder for the E2E). Example call:

import simulator

solution_path = "app/solution_xxx"
run_config = "app/solution_xxx/run_config.json"
image_dir = "app/solution_xxx/alg/golden" # optional, only if run_config does not specify input image list
simulator.run_solution(solution_path, run_config, image_dir=image_dir)

Example run_config JSON can be found at app/template_app/example_options.json. Results will be stored in bin/out/solution_xxx/....

Dependencies

PPP library: 22d82dc0

Regression library (working): 8b9c33fa37c83ed3220921fc636fdf70f64e102e (0.23.0_dev branch)

Python Dependencies

E2E:

matplotlib>=3.3.4
numpy>=1.21.0
Pillow>=7.2.0
tqdm>=4.56.0

Dongle:

grpcio>=1.35.0
grpcio-tools>=1.16.0
numpy>=1.20.0

To install, you can either use python_flow/internal/scripts/conda_install.sh or python_flow/internal/scripts/pip_install.sh from the root directory of this project.

./python_flow/internal/scripts/conda_install.sh

./python_flow/internal/scripts/pip_install.sh