API reference

Inference benchmarking

Signatures, parameters, return contracts, and source for inference benchmarking.

Overview

The benchmark measures model sequence execution across batch sizes, horizons, and precision choices. It does not include the complete connected population runtime. Preserve the warmup, repetitions, synchronization, compilation mode, device, and shape contract with every reported timing.

Source revision: caef7be03abd. Public export index.

benchmark_inference_matrix

axosim.inference_benchmark.benchmark_inference_matrix(model: torch.nn.Module, *, batch_sizes: Iterable[int], time_steps: Iterable[int], input_dim: int, device: str='cpu', precision: str='float32', warmup_runs: int=5, runs: int=20, compile_model: bool=False, accuracy_metrics_path: str | Path | None=None) -> dict[str, Any]

Source

Benchmark dense full-window inference for deployment-oriented comparisons.

Parameters

model torch.nn.Module
required. Model to execute, optimize, count, or serialize for this operation.
batch_sizes Iterable[int]
keyword-only, required. Batch sizes included in the benchmark matrix.
time_steps Iterable[int]
keyword-only, required. Native sequence horizon.
input_dim int
keyword-only, required. Native input-channel count.
device str
keyword-only, default='cpu'. Execution or allocation device.
precision str
keyword-only, default='float32'. Floating-point execution precision.
warmup_runs int
keyword-only, default=5. Unmeasured iterations before the timing repetitions.
runs int
keyword-only, default=20. Measured repetitions after the declared warmup.
compile_model bool
keyword-only, default=False. Enable the benchmark's compilation path.
accuracy_metrics_path str | Path | None
keyword-only, default=None. Optional metrics JSON included with the inference timing report.

Returns

report dict[str, Any]
Timing rows for each batch/horizon pair, device/precision and repetition settings, resident parameter count, optional accuracy metadata, and the fastest successful row. Out-of-memory rows contain error details.

count_parameters

axosim.inference_benchmark.count_parameters(model: torch.nn.Module) -> int

Source

Count all resident model parameter elements, including frozen parameters.

Parameters

model torch.nn.Module
required. Model to execute, optimize, count, or serialize for this operation.

Returns

count int
Sum of numel() over every model parameter, independent of requires_grad.

write_inference_benchmark

axosim.inference_benchmark.write_inference_benchmark(report: dict[str, Any], path: str | Path) -> None

Source

Write the inference timing report as JSON, creating the parent directory when needed.

Parameters

report dict[str, Any]
required. Inference benchmark report to serialize.
path str | Path
required. Filesystem location to read or write; see the operation's persistence contract.

Returns

result None
No return value.

parse_int_list

axosim.inference_benchmark.parse_int_list(value: str) -> list[int]

Source

Parse a comma-separated integer list, ignoring empty entries and rejecting an empty result.

Parameters

value str
required. Comma-separated string of integer values.

Returns

values list[int]
Parsed integers in their original order.

Search guides, examples, and API signatures.