AxoBench API
Diagnostic rows
Signatures, parameters, return values, and source for diagnostic rows.
Overview
Dataframe-first diagnostic rows for teacher QC, model diagnostics, paired interventions, morphology routing, and morphology contrasts, with runners that read an evaluation root and a bundle that writes all of them. Every row carries the columns in DIAGNOSTIC_ROW_COLUMNS. A row is kept when its diagnostic belongs to one of the suites in the context. See diagnostics.
Source revision: 496e0c1a2dfd. Export index.
DiagnosticContext
axobench.benchmark.diagnostic_rows.DiagnosticContext(suites: tuple[str, ...] = ('paper-summary',), model_id: str = DEFAULT_MODEL_ID, teacher_id: str = DEFAULT_TEACHER_ID, morphology_id: str = 'unknown', split: str = 'val', condition: str = 'val', intervention: str | None = None, trace_id: str | None = None, target_view: str = 'neuronio_clipped_normalized', soma_scale: float = 0.1, soma_bias_mv: float = DEFAULT_Y_TRAIN_SOMA_BIAS, ignore_start: int = 500, calibrated_spike_threshold: float | None = None)
Metadata attached to every row and the suites that select rows. Exported from axobench.benchmark.
Parameters
suitestuple[str, ...]- default=('paper-summary',). Suites whose diagnostics are kept.
model_idstr- default=DEFAULT_MODEL_ID = 'model'. Model name.
teacher_idstr- default=DEFAULT_TEACHER_ID = 'teacher'. Teacher name.
morphology_idstr- default='unknown'. Morphology label of the rows.
splitstr- default='val'. Split label.
conditionstr- default='val'. Split or interventions/<name>.
interventionstr | None- default=None. Condition name.
trace_idstr | None- default=None. Trace label for per-trace rows.
target_viewstr- default='neuronio_clipped_normalized'. Target view label.
soma_scalefloat- default=0.1. Scale of the soma target coordinate.
soma_bias_mvfloat- default=DEFAULT_Y_TRAIN_SOMA_BIAS = -67.7. Millivolt offset of the soma target coordinate.
ignore_startint- default=500. Initial steps excluded.
calibrated_spike_thresholdfloat | None- default=None. Spike threshold for model rows, calibrated on other traces.
Attributes
Constructor fields are retained as read-only attributes.
evaluate_model_diagnostic_rows
axobench.benchmark.diagnostic_rows.evaluate_model_diagnostic_rows(model_fn: Callable[[np.ndarray], np.ndarray], inputs: np.ndarray, targets: np.ndarray, *, context: DiagnosticContext | None=None, trace_ids: list[str] | None=None, raw_soma_mv: np.ndarray | None=None) -> list[dict[str, Any]]
Evaluate a model on supplied arrays and return rows for trace fidelity, state-conditioned dynamics, spike behavior, feature fidelity, and mechanistic response. Exported from axobench.benchmark.
Parameters
model_fnCallable[[np.ndarray], np.ndarray]- required. Prediction function mapping float32 inputs (batch, time, 1278) to (batch, time, 2) outputs: a spike score and the soma target coordinate. With supports_morphology_ids=True it is called with a morphology_ids keyword.
inputsnp.ndarray- required. Inputs, (trace, time, channels).
targetsnp.ndarray- required. Teacher targets, (trace, time, 2).
contextDiagnosticContext | None- keyword-only, default=None. Row metadata and suite selection; None uses the function's default context.
trace_idslist[str] | None- keyword-only, default=None. Sample IDs, one per trace; the text before the first ':' is the morphology or cell-type group.
raw_soma_mvnp.ndarray | None- keyword-only, default=None. Optional unclipped teacher soma voltage in millivolts, shaped (trace, time); replaces the voltage reconstructed from the target.
Returns
rowslist[dict[str, Any]]- Rows of the selected suites.
teacher_qc_diagnostic_rows
axobench.benchmark.diagnostic_rows.teacher_qc_diagnostic_rows(inputs: np.ndarray, targets: np.ndarray, *, context: DiagnosticContext | None=None, raw_soma_mv: np.ndarray | None=None, trace_ids: list[str] | None=None) -> list[dict[str, Any]]
Teacher distribution rows and, with raw voltage, clipping-distortion rows. Exported from axobench.benchmark.
Parameters
inputsnp.ndarray- required. Inputs, (trace, time, channels).
targetsnp.ndarray- required. Teacher targets, (trace, time, 2).
contextDiagnosticContext | None- keyword-only, default=None. Row metadata and suite selection; None uses the function's default context.
raw_soma_mvnp.ndarray | None- keyword-only, default=None. Optional unclipped teacher soma voltage in millivolts, shaped (trace, time); replaces the voltage reconstructed from the target.
trace_idslist[str] | None- keyword-only, default=None. Sample IDs, one per trace; the text before the first ':' is the morphology or cell-type group.
Returns
rowslist[dict[str, Any]]- Rows of the selected suites.
paired_intervention_diagnostic_rows
axobench.benchmark.diagnostic_rows.paired_intervention_diagnostic_rows(baseline_inputs: np.ndarray, baseline_targets: np.ndarray, intervention_inputs: np.ndarray, intervention_targets: np.ndarray, *, model_fn: Callable[[np.ndarray], np.ndarray] | None=None, context: DiagnosticContext | None=None, trace_ids: list[str] | None=None) -> list[dict[str, Any]]
Teacher response rows for paired inputs and, with a model, model response rows: voltage-change errors, Response SERA, correlation, gain, spike-count deltas, and direction agreement. The model’s spike threshold is context.calibrated_spike_threshold, or 0. Exported from axobench.benchmark.
Parameters
baseline_inputsnp.ndarray- required. Baseline inputs.
baseline_targetsnp.ndarray- required. Baseline teacher targets.
intervention_inputsnp.ndarray- required. Edited inputs.
intervention_targetsnp.ndarray- required. Edited teacher targets.
model_fnCallable[[np.ndarray], np.ndarray] | None- keyword-only, default=None. Optional prediction function; None writes teacher rows only.
contextDiagnosticContext | None- keyword-only, default=None. Row metadata and suite selection; None uses the function's default context.
trace_idslist[str] | None- keyword-only, default=None. Sample IDs, one per trace; the text before the first ':' is the morphology or cell-type group.
Returns
rowslist[dict[str, Any]]- Rows of the selected suites.
morphology_routing_qc_rows
axobench.benchmark.diagnostic_rows.morphology_routing_qc_rows(*, context: DiagnosticContext | None=None) -> list[dict[str, Any]]
Rows of branch counts and shape ratios for the v1 morphology registry. Exported from axobench.benchmark.
Parameters
contextDiagnosticContext | None- keyword-only, default=None. Row metadata and suite selection; None uses the function's default context.
Returns
rowslist[dict[str, Any]]- Rows with role, Cre line, reconstruction type, SWC path, and caveat.
morphology_contrast_diagnostic_rows
axobench.benchmark.diagnostic_rows.morphology_contrast_diagnostic_rows(source_rows: Iterable[dict[str, Any]], *, context: DiagnosticContext | None=None, min_morphologies: int=2) -> list[dict[str, Any]]
Mean, standard deviation, range, minimum, and maximum across morphologies of every numeric metric present for at least min_morphologies morphologies, grouped by diagnostic, metric, state, model, and condition. Exported from axobench.benchmark.
Parameters
source_rowsIterable[dict[str, Any]]- required. Rows with morphology labels.
contextDiagnosticContext | None- keyword-only, default=None. Row metadata and suite selection; None uses the function's default context.
min_morphologiesint- keyword-only, default=2. Minimum number of morphologies per contrast; at least 2.
Returns
rowslist[dict[str, Any]]- morphology_contrast rows with the per-morphology values attached.
validate_diagnostic_rows
axobench.benchmark.diagnostic_rows.validate_diagnostic_rows(rows: Iterable[dict[str, Any]]) -> None
Raise ValueError if a row lacks a column of DIAGNOSTIC_ROW_COLUMNS. Exported from axobench.benchmark.
Parameters
rowsIterable[dict[str, Any]]- required. Diagnostic row dictionaries.
write_diagnostic_rows
axobench.benchmark.diagnostic_rows.write_diagnostic_rows(rows: Iterable[dict[str, Any]], path: str | Path) -> None
Validate and write rows as .json, .jsonl, or .csv. Exported from axobench.benchmark.
Parameters
rowsIterable[dict[str, Any]]- required. Diagnostic row dictionaries.
pathstr | Path- required. Destination ending in .json, .jsonl, or .csv.
run_teacher_qc_on_selector
axobench.benchmark.runner.run_teacher_qc_on_selector(selector: DatasetSelector, *, context: DiagnosticContext | None=None, raw_soma_mv: np.ndarray | None=None, max_samples: int | None=None, output: str | Path | None=None, cache_shards: int=1) -> list[dict[str, Any]]
Teacher QC rows for the first max_samples traces of a selector, in index order. Exported from axobench.benchmark.
Parameters
selectorDatasetSelector- required. Resolved split or intervention directory from V1DatasetLayout.
contextDiagnosticContext | None- keyword-only, default=None. Row metadata and suite selection; None uses the function's default context.
raw_soma_mvnp.ndarray | None- keyword-only, default=None. Optional raw voltage; None reads soma_mv from the shards.
max_samplesint | None- keyword-only, default=None. Maximum traces; None reads all.
outputstr | Path | None- keyword-only, default=None. Optional .json, .jsonl, or .csv path for the rows.
cache_shardsint- keyword-only, default=1. Decoded shards kept in memory by the reader.
Returns
rowslist[dict[str, Any]]- Teacher QC rows.
run_model_diagnostics_on_selector
axobench.benchmark.runner.run_model_diagnostics_on_selector(model_fn: Callable[[np.ndarray], np.ndarray], selector: DatasetSelector, *, context: DiagnosticContext | None=None, max_samples: int | None=None, output: str | Path | None=None, cache_shards: int=1) -> list[dict[str, Any]]
Model diagnostic rows for the first max_samples traces of a selector, with raw voltage when stored. Exported from axobench.benchmark.
Parameters
model_fnCallable[[np.ndarray], np.ndarray]- required. Prediction function mapping float32 inputs (batch, time, 1278) to (batch, time, 2) outputs: a spike score and the soma target coordinate. With supports_morphology_ids=True it is called with a morphology_ids keyword.
selectorDatasetSelector- required. Resolved split or intervention directory from V1DatasetLayout.
contextDiagnosticContext | None- keyword-only, default=None. Row metadata and suite selection; None uses the function's default context.
max_samplesint | None- keyword-only, default=None. Maximum traces; None reads all.
outputstr | Path | None- keyword-only, default=None. Optional .json, .jsonl, or .csv path for the rows.
cache_shardsint- keyword-only, default=1. Decoded shards kept in memory by the reader.
Returns
rowslist[dict[str, Any]]- Model diagnostic rows.
run_paired_intervention_on_selector
axobench.benchmark.runner.run_paired_intervention_on_selector(selector: DatasetSelector, *, model_fn: Callable[[np.ndarray], np.ndarray] | None=None, context: DiagnosticContext | None=None, max_samples: int | None=None, output: str | Path | None=None) -> list[dict[str, Any]]
Paired intervention rows for the first max_samples pairs of a condition directory, across its shards in sorted file order. Exported from axobench.benchmark.
Parameters
selectorDatasetSelector- required. Resolved split or intervention directory from V1DatasetLayout.
model_fnCallable[[np.ndarray], np.ndarray] | None- keyword-only, default=None. Optional prediction function; None writes teacher rows only.
contextDiagnosticContext | None- keyword-only, default=None. Row metadata and suite selection; None uses the function's default context.
max_samplesint | None- keyword-only, default=None. Maximum pairs; None reads all.
outputstr | Path | None- keyword-only, default=None. Optional .json, .jsonl, or .csv path for the rows.
Returns
rowslist[dict[str, Any]]- Intervention rows.
run_paired_intervention_on_npz
axobench.benchmark.runner.run_paired_intervention_on_npz(path: str | Path, *, model_fn: Callable[[np.ndarray], np.ndarray] | None=None, context: DiagnosticContext | None=None, max_samples: int | None=None, output: str | Path | None=None) -> list[dict[str, Any]]
Paired intervention rows for one pair file. Exported from axobench.benchmark.
Parameters
pathstr | Path- required. Pair shard.
model_fnCallable[[np.ndarray], np.ndarray] | None- keyword-only, default=None. Optional prediction function; None writes teacher rows only.
contextDiagnosticContext | None- keyword-only, default=None. Row metadata and suite selection; None uses the function's default context.
max_samplesint | None- keyword-only, default=None. Maximum pairs; None reads all.
outputstr | Path | None- keyword-only, default=None. Optional .json, .jsonl, or .csv path for the rows.
Returns
rowslist[dict[str, Any]]- Intervention rows.
run_v1_benchmark_bundle
axobench.benchmark.bundle.run_v1_benchmark_bundle(*, dataset_root: str | Path, output_dir: str | Path, base_context: DiagnosticContext | None=None, model_fn: Callable[[np.ndarray], np.ndarray] | None=None, model_id: str | None=None, intervention_pair_paths: Iterable[str | Path]=(), suite: SuiteProfile='paper-summary', max_val_samples: int | None=None, max_intervention_samples: int | None=None, cache_shards: int=1, write_summaries: bool=True) -> dict[str, Any]
Write teacher QC rows for val, morphology routing rows, intervention rows per condition and morphology, morphology-contrast rows, and, with a model, model rows, together with Markdown summaries and a manifest.json. Exported from axobench.benchmark.
Parameters
dataset_rootstr | Path- keyword-only, required. Evaluation root containing val/ and optionally interventions/<condition>/.
output_dirstr | Path- keyword-only, required. Directory for the rows, summaries, and manifest.
base_contextDiagnosticContext | None- keyword-only, default=None. Context copied into every row; its morphology_id labels the val rows.
model_fnCallable[[np.ndarray], np.ndarray] | None- keyword-only, default=None. Optional prediction function; None runs the teacher bundle only.
model_idstr | None- keyword-only, default=None. Model name; defaults to base_context.model_id.
intervention_pair_pathsIterable[str | Path]- keyword-only, default=(). Explicit pair files; empty discovers every condition under <root>/interventions.
suiteSuiteProfile- keyword-only, default='paper-summary'. teacher-qc, paper-summary, or full.
max_val_samplesint | None- keyword-only, default=None. Maximum validation traces, read in index order.
max_intervention_samplesint | None- keyword-only, default=None. Maximum pairs per condition and morphology.
cache_shardsint- keyword-only, default=1. Decoded shards kept in memory by the reader.
write_summariesbool- keyword-only, default=True. Also write a Markdown summary per row file.
Returns
manifestdict[str, Any]- Settings and the written artifacts with row counts.
summarize_diagnostic_rows
axobench.benchmark.reports.summarize_diagnostic_rows(rows: list[dict[str, Any]]) -> dict[str, Any]
Compact summary of rows: counts, diagnostics, suites, models, teachers, morphologies, conditions, example trace IDs, and numeric and label metric summaries. Exported from axobench.benchmark.
Parameters
rowslist[dict[str, Any]]- required. Diagnostic row dictionaries.
Returns
summarydict[str, Any]- JSON-compatible summary.
write_diagnostic_summary
axobench.benchmark.reports.write_diagnostic_summary(rows: list[dict[str, Any]], path: str | Path) -> None
Write summarize_diagnostic_rows as Markdown or JSON. Exported from axobench.benchmark.
Parameters
rowslist[dict[str, Any]]- required. Diagnostic row dictionaries.
pathstr | Path- required. Destination ending in .md or .json.