Free Dimensions Overrides

ONNX models support dynamic input shapes through free dimensions. A free dimension allows the model to accept inputs of varying sizes along that dimension. For instance, a model with input shape [batch, 3, height, width] can process any number of RGB images (3 channels) of any size.

While flexible, you can optimize performance by fixing these dimensions when your application uses consistent input sizes. For example, if your WebNN app always processes single 224×224 images, you can override the free dimensions from [batch, 3, height, width] to [1, 3, 224, 224] using the freeDimensionOverrides option. This configuration locks the input dimensions to specific values, potentially improving model performance.

Setting Free Dimension Overrides in Transformers.js 3.5.0+

When working with dynamic shapes in Transformers.js  3.5.0 or later, you can specify free dimension overrides using one of the following methods, listed in order of precedence:

1. Directly in your code using the freeDimensionOverrides
2. In the device_config level of transformers.js_config within a model’s config.js file
3. In the model level of transformers.js_config within a model’s config.js file

The first method that provides valid overrides will be used, with earlier methods taking priority over later ones.

Priority 1

Setting freeDimensionOverrides in your code with Transformers.js.

import * as transformers from from '@huggingface/transformers@3.5.0';
 
const options = {
  device: "webnn-gpu", // webnn, webnn-cpu, webnn-npu
  dtype: "fp32",
  session_options: {
    freeDimensionOverrides: {
      batch: 1,
      height: 224,
      width: 224
    }
  }
}
 
const classifier = await transformers.pipeline('image-classification', path, options);

Priority 2

Setting free_dimension_overrides in the device_config level of transformers.js_config in config.js file of Huggingface models, e.g. Xenova/mobileclip_s0 

{
  "transformers.js_config": {
    "device_config": {
      "webnn-gpu": {
        "dtype": "fp16",
        "free_dimension_overrides": {
          "batch": 1,
          "height": 224,
          "width": 224
        }
      }
    }
  }
}

Priority 3

Setting free_dimension_overrides in a model level of transformers.js_config in config.js file of Huggingface models, e.g. Xenova/mobileclip_s0 

{
  "transformers.js_config": {
    "device": "webnn-gpu",
    "dtype": "fp16",
    "free_dimension_overrides": {
      "batch": 1,
      "height": 224,
      "width": 224
    }
  }
}

Symbolic Dimension

Example model without symbolic dimensions

Example model with three symbolic dimensions called batch_size, height and width

WebNN Execution Provider needs specify fixed integer values via freeDimensionOverrides for all the symbolic dimensions to avoid fallbacks.

Additional Resources

• Transformers.js - Free dimension overrides - Support device-level configuration across all devices