API reference

Local metrics and dataset evaluation

Signatures, parameters, return contracts, and source for local metrics and dataset evaluation.

Overview

These local utility functions support RMSE, AUC, and dataset evaluation. They do not implement the AxoBench protocol: AxoBench defines Voltage SERA, spike Mean F1, the subthreshold voltage RMSE, and Response SERA, and axosim-evaluate-model applies them to a checkpoint. AxoBench 0.2.0 reports SERA per teacher spike in mV^2/spike, which differs from the technical report’s earlier SERA definition.

Source revision: caef7be03abd. Public export index.

soma_rmse

axosim.metrics.soma_rmse(prediction: np.ndarray, target: np.ndarray) -> float

Source

Compute root-mean-square error from channel 1 of matched prediction and target arrays.

Parameters

prediction np.ndarray
required. Prediction array with a final channel axis; channel 1 contains soma values.
target np.ndarray
required. Matched reference array in the same soma coordinate as prediction.

Returns

rmse float
Root-mean-square error in the supplied soma target coordinate, aggregated over channel 1.

binary_auc

axosim.metrics.binary_auc(scores: np.ndarray, labels: np.ndarray) -> float

Source

Compute rank-based binary ROC AUC with averaged tied ranks; return NaN when either label class is absent.

Parameters

scores np.ndarray
required. Continuous scores; larger values indicate greater evidence for a positive label.
labels np.ndarray
required. Binary reference labels, classified as positive above 0.5.

Returns

auc float
ROC AUC over all flattened entries, or NaN when labels contain only one class.

spike_auc

axosim.metrics.spike_auc(prediction: np.ndarray, target: np.ndarray) -> float

Source

Compute binary ROC AUC from channel 0 of prediction and target arrays.

Parameters

prediction np.ndarray
required. Prediction array whose channel 0 contains continuous spike scores.
target np.ndarray
required. Matched reference array whose channel 0 contains spike labels.

Returns

auc float
ROC AUC over channel 0, or NaN when reference labels contain only one class.

evaluate_dataset

axosim.evaluate.evaluate_dataset(model: BranchELM, dataset: ShardedNeuronIODataset, *, batch_size: int=8, device: str='cpu', soma_units: str='millivolts', y_train_soma_scale: float=DEFAULT_Y_TRAIN_SOMA_SCALE, ignore_start: int=0, mask_mode: str='ignore-start', stitch_burn_in: int=150, soma_affine_calibration: bool=False, pin_memory: bool=False, non_blocking: bool=True) -> dict[str, float | int]

Source

Evaluate local spike AUC and soma RMSE over dataset batches, with the selected temporal mask and soma coordinate convention.

Parameters

model BranchELM
required. Model to execute, optimize, count, or serialize for this operation.
dataset ShardedNeuronIODataset
required. Trace dataset supporting the batching contract used by this operation.
batch_size int
keyword-only, default=8. Examples processed per batch.
device str
keyword-only, default='cpu'. Execution or allocation device.
soma_units str
keyword-only, default='millivolts'. Coordinate convention used when computing the local soma metric.
y_train_soma_scale float
keyword-only, default=0.1. Scale converting the biased voltage to the training target coordinate.
ignore_start int
keyword-only, default=0. Initial native timesteps excluded by the declared path.
mask_mode str
keyword-only, default='ignore-start'. Initial-timestep exclusion or official overlap-stitching mask.
stitch_burn_in int
keyword-only, default=150. Native overlap timesteps excluded from subsequent stitched windows.
soma_affine_calibration bool
keyword-only, default=False. Rescale predictions to the evaluation targets' mean and standard deviation; declare this calibration when comparing metrics.
pin_memory bool
keyword-only, default=False. Stage CPU arrays in pinned memory before a CUDA transfer.
non_blocking bool
keyword-only, default=True. Request asynchronous tensor transfers where supported.

Returns

metrics dict[str, float | int]
Sample count, temporal-mask and calibration settings, elapsed time, throughput, soma RMSE and units, spike AUC, and related local spike metrics.

write_metrics

axosim.evaluate.write_metrics(metrics: dict[str, float | int], output: str | Path) -> None

Source

Write the local metrics dictionary as JSON, creating its parent directory when needed.

Parameters

metrics dict[str, float | int]
required. Metrics dictionary to serialize.
output str | Path
required.

Returns

result None
No return value.

Search guides, examples, and API signatures.