ML Inference

Generic inference module powered by ONNX Runtime. Loads any .onnx model file, reads its port layout from an optional model-description.md companion file, and maps model inputs/outputs to Usine ports in real time using a dedicated background worker thread.

ONNX (Open Neural Network Exchange) is an open format for machine-learning models. Models trained in PyTorch, TensorFlow, scikit-learn, or any compatible framework can be exported to .onnx and run directly in Usine without any Python runtime. The module handles all type conversions between Usine flow types and the flat float tensors expected by the model.

Settings

Settings

model

File path to the .onnx model file. Click to open a file browser filtered to *.onnx. When a file is selected, the module:

1. Loads the model into ONNX Runtime.
2. Looks for a model-description.md file in the same directory as the .onnx file.
3. Configures input and output ports according to the description (or falls back to array ports if no description is found).
4. Starts a background inference thread.

The model is reloaded automatically when the threads setting changes.

threads

Number of CPU threads allocated to ONNX Runtime's intra-op parallelism. Range 1 – 8, default 1. Increase for large models or on multi-core systems; for small models a single thread usually gives lower latency.

Inputs

The module provides 2 input ports (input-0, input-1). Their captions and flow types are set automatically when a model is loaded.

Port flow types depend on the flow field in the model description:

Outputs

The module provides 5 output ports (output-0 to output-4). Their captions and flow types are configured by the model description, following the same rules as inputs.

time

Inference latency in milliseconds, measured from the moment inputs are submitted to the worker thread until the result is written back. Useful for performance profiling.

model-description.md

Place a file named model-description.md next to your .onnx file to control port layout, captions, and flow types. Without this file the module still works but all ports default to array flow and use the raw ONNX tensor names as captions.

The file uses a YAML frontmatter block (between --- markers). Any text after the second --- is loaded as the model description and shown in the module's Properties panel.

Frontmatter structure

---
usine:
  version: 1
  name: "My Model"
  description: "Short description shown in Properties."

  inputs:
    - name: input_tensor_name     # must match the ONNX tensor name exactly
      label: "friendly caption"   # optional, shown on the port
      flow: array                 # array | data | audio | video
      size: 128                   # number of float elements

  outputs:
    - name: output_tensor_name
      label: "result"
      flow: data
---

Longer description shown in the Properties panel.

Audio-specific fields

When flow: audio, additional fields control windowing and resampling:

  inputs:
    - name: audio_input
      label: "microphone"
      flow: audio
      size: 16000        # 1 s window at 16 kHz
      sample_rate: 16000
      hop: 8000          # inference every 0.5 s (50% overlap)
      normalize: zero-mean-unit-variance

Video-specific fields

When flow: video, the input frame is resized to the model's expected resolution and converted to a flat float tensor in [0, 1]:

  inputs:
    - name: image
      label: "camera"
      flow: video
      width: 224
      height: 224
      channels: 3
      format: chw       # PyTorch convention

How it works

Inference pipeline

Usine Process thread          Background worker thread
─────────────────────         ─────────────────────────
[input data ready]
  → copy to worker buffer
  → signal worker
                              ← wake up
                              ← run ONNX session
                              ← write outputs to buffer
[next process cycle]
  → check HasOutput
  → copy outputs to ports
  → write InferenceTime

The worker thread runs asynchronously: the Process thread never blocks. If inference is slower than the process block rate, the module silently skips frames. This ensures Usine's audio engine is never stalled.

Audio windowing

For audio ports, samples are written into a ring buffer as they arrive. When the ring buffer has accumulated enough samples to fill one window:

1. The window is extracted and optionally normalized.
2. If the model sample rate differs from Usine's rate, the window is resampled using cubic interpolation.
3. The resampled buffer is submitted to the worker thread.
4. The next window starts hop samples later.

Output audio is upsampled back to Usine's sample rate after inference.

Video resize

If the incoming frame dimensions differ from the model's expected width × height, the frame is rescaled using a fast low-quality resize on the worker thread. Pixel values are converted to [0, 1] floats (dividing by 255). Channel ordering follows the format field.

Workflow — step by step

1. Export your model to ONNX

From PyTorch:

torch.onnx.export(model, dummy_input, "my_model.onnx",
                  input_names=["features"],
                  output_names=["label_scores"])

2. Write a model-description.md

Create model-description.md next to my_model.onnx:

---
usine:
  version: 1
  name: "My Classifier"
  inputs:
    - name: features
      flow: array
      size: 32
  outputs:
    - name: label_scores
      flow: array
      size: 10
---
Classifies a 32-element feature vector into 10 categories.

3. Load in Usine

• Drop an ML Inference module into your patch.
• In Settings, click model and select my_model.onnx.
• The two input ports reconfigure to match your description.
• Wire your data source to the input port and read the output array.

4. Monitor

• Connect the time output to a display to watch inference latency.
• Check the Usine trace panel for load errors or fallback messages.

Notes and limits

See also

• Machine Learning
• ML Train SOM
• ArrayDistance (USER MODULE)
• ML Inference
• MotionRecognition (USER MODULE)