API reference
Inference benchmarking
Signatures, parameters, return contracts, and source for inference benchmarking.
Overview
The benchmark measures model sequence execution across batch sizes, horizons, and precision choices. It does not include the complete connected population runtime. Preserve the warmup, repetitions, synchronization, compilation mode, device, and shape contract with every reported timing.
Source revision: caef7be03abd. Public export index.
benchmark_inference_matrix
axosim.inference_benchmark.benchmark_inference_matrix(model: torch.nn.Module, *, batch_sizes: Iterable[int], time_steps: Iterable[int], input_dim: int, device: str='cpu', precision: str='float32', warmup_runs: int=5, runs: int=20, compile_model: bool=False, accuracy_metrics_path: str | Path | None=None) -> dict[str, Any]
Benchmark dense full-window inference for deployment-oriented comparisons.
Parameters
modeltorch.nn.Module- required. Model to execute, optimize, count, or serialize for this operation.
batch_sizesIterable[int]- keyword-only, required. Batch sizes included in the benchmark matrix.
time_stepsIterable[int]- keyword-only, required. Native sequence horizon.
input_dimint- keyword-only, required. Native input-channel count.
devicestr- keyword-only, default='cpu'. Execution or allocation device.
precisionstr- keyword-only, default='float32'. Floating-point execution precision.
warmup_runsint- keyword-only, default=5. Unmeasured iterations before the timing repetitions.
runsint- keyword-only, default=20. Measured repetitions after the declared warmup.
compile_modelbool- keyword-only, default=False. Enable the benchmark's compilation path.
accuracy_metrics_pathstr | Path | None- keyword-only, default=None. Optional metrics JSON included with the inference timing report.
Returns
reportdict[str, Any]- Timing rows for each batch/horizon pair, device/precision and repetition settings, resident parameter count, optional accuracy metadata, and the fastest successful row. Out-of-memory rows contain error details.
count_parameters
axosim.inference_benchmark.count_parameters(model: torch.nn.Module) -> int
Count all resident model parameter elements, including frozen parameters.
Parameters
modeltorch.nn.Module- required. Model to execute, optimize, count, or serialize for this operation.
Returns
countint- Sum of numel() over every model parameter, independent of requires_grad.
write_inference_benchmark
axosim.inference_benchmark.write_inference_benchmark(report: dict[str, Any], path: str | Path) -> None
Write the inference timing report as JSON, creating the parent directory when needed.
Parameters
reportdict[str, Any]- required. Inference benchmark report to serialize.
pathstr | Path- required. Filesystem location to read or write; see the operation's persistence contract.
Returns
resultNone- No return value.
parse_int_list
axosim.inference_benchmark.parse_int_list(value: str) -> list[int]
Parse a comma-separated integer list, ignoring empty entries and rejecting an empty result.
Parameters
valuestr- required. Comma-separated string of integer values.
Returns
valueslist[int]- Parsed integers in their original order.