API reference
Local metrics and dataset evaluation
Signatures, parameters, return contracts, and source for local metrics and dataset evaluation.
Overview
These local utility functions support RMSE, AUC, and dataset evaluation. They do not implement the AxoBench protocol: AxoBench defines Voltage SERA, spike Mean F1, the subthreshold voltage RMSE, and Response SERA, and axosim-evaluate-model applies them to a checkpoint. AxoBench 0.2.0 reports SERA per teacher spike in mV^2/spike, which differs from the technical report’s earlier SERA definition.
Source revision: caef7be03abd. Public export index.
soma_rmse
axosim.metrics.soma_rmse(prediction: np.ndarray, target: np.ndarray) -> float
Compute root-mean-square error from channel 1 of matched prediction and target arrays.
Parameters
predictionnp.ndarray- required. Prediction array with a final channel axis; channel 1 contains soma values.
targetnp.ndarray- required. Matched reference array in the same soma coordinate as prediction.
Returns
rmsefloat- Root-mean-square error in the supplied soma target coordinate, aggregated over channel 1.
binary_auc
axosim.metrics.binary_auc(scores: np.ndarray, labels: np.ndarray) -> float
Compute rank-based binary ROC AUC with averaged tied ranks; return NaN when either label class is absent.
Parameters
scoresnp.ndarray- required. Continuous scores; larger values indicate greater evidence for a positive label.
labelsnp.ndarray- required. Binary reference labels, classified as positive above 0.5.
Returns
aucfloat- ROC AUC over all flattened entries, or NaN when labels contain only one class.
spike_auc
axosim.metrics.spike_auc(prediction: np.ndarray, target: np.ndarray) -> float
Compute binary ROC AUC from channel 0 of prediction and target arrays.
Parameters
predictionnp.ndarray- required. Prediction array whose channel 0 contains continuous spike scores.
targetnp.ndarray- required. Matched reference array whose channel 0 contains spike labels.
Returns
aucfloat- ROC AUC over channel 0, or NaN when reference labels contain only one class.
evaluate_dataset
axosim.evaluate.evaluate_dataset(model: BranchELM, dataset: ShardedNeuronIODataset, *, batch_size: int=8, device: str='cpu', soma_units: str='millivolts', y_train_soma_scale: float=DEFAULT_Y_TRAIN_SOMA_SCALE, ignore_start: int=0, mask_mode: str='ignore-start', stitch_burn_in: int=150, soma_affine_calibration: bool=False, pin_memory: bool=False, non_blocking: bool=True) -> dict[str, float | int]
Evaluate local spike AUC and soma RMSE over dataset batches, with the selected temporal mask and soma coordinate convention.
Parameters
modelBranchELM- required. Model to execute, optimize, count, or serialize for this operation.
datasetShardedNeuronIODataset- required. Trace dataset supporting the batching contract used by this operation.
batch_sizeint- keyword-only, default=8. Examples processed per batch.
devicestr- keyword-only, default='cpu'. Execution or allocation device.
soma_unitsstr- keyword-only, default='millivolts'. Coordinate convention used when computing the local soma metric.
y_train_soma_scalefloat- keyword-only, default=0.1. Scale converting the biased voltage to the training target coordinate.
ignore_startint- keyword-only, default=0. Initial native timesteps excluded by the declared path.
mask_modestr- keyword-only, default='ignore-start'. Initial-timestep exclusion or official overlap-stitching mask.
stitch_burn_inint- keyword-only, default=150. Native overlap timesteps excluded from subsequent stitched windows.
soma_affine_calibrationbool- keyword-only, default=False. Rescale predictions to the evaluation targets' mean and standard deviation; declare this calibration when comparing metrics.
pin_memorybool- keyword-only, default=False. Stage CPU arrays in pinned memory before a CUDA transfer.
non_blockingbool- keyword-only, default=True. Request asynchronous tensor transfers where supported.
Returns
metricsdict[str, float | int]- Sample count, temporal-mask and calibration settings, elapsed time, throughput, soma RMSE and units, spike AUC, and related local spike metrics.
write_metrics
axosim.evaluate.write_metrics(metrics: dict[str, float | int], output: str | Path) -> None
Write the local metrics dictionary as JSON, creating its parent directory when needed.
Parameters
metricsdict[str, float | int]- required. Metrics dictionary to serialize.
outputstr | Path- required.
Returns
resultNone- No return value.