AxoBench API

Diagnostic rows

Signatures, parameters, return values, and source for diagnostic rows.

Overview

Dataframe-first diagnostic rows for teacher QC, model diagnostics, paired interventions, morphology routing, and morphology contrasts, with runners that read an evaluation root and a bundle that writes all of them. Every row carries the columns in DIAGNOSTIC_ROW_COLUMNS. A row is kept when its diagnostic belongs to one of the suites in the context. See diagnostics.

Source revision: 496e0c1a2dfd. Export index.

DiagnosticContext

axobench.benchmark.diagnostic_rows.DiagnosticContext(suites: tuple[str, ...] = ('paper-summary',), model_id: str = DEFAULT_MODEL_ID, teacher_id: str = DEFAULT_TEACHER_ID, morphology_id: str = 'unknown', split: str = 'val', condition: str = 'val', intervention: str | None = None, trace_id: str | None = None, target_view: str = 'neuronio_clipped_normalized', soma_scale: float = 0.1, soma_bias_mv: float = DEFAULT_Y_TRAIN_SOMA_BIAS, ignore_start: int = 500, calibrated_spike_threshold: float | None = None)

Source

Metadata attached to every row and the suites that select rows. Exported from axobench.benchmark.

Parameters

suites tuple[str, ...]
default=('paper-summary',). Suites whose diagnostics are kept.
model_id str
default=DEFAULT_MODEL_ID = 'model'. Model name.
teacher_id str
default=DEFAULT_TEACHER_ID = 'teacher'. Teacher name.
morphology_id str
default='unknown'. Morphology label of the rows.
split str
default='val'. Split label.
condition str
default='val'. Split or interventions/<name>.
intervention str | None
default=None. Condition name.
trace_id str | None
default=None. Trace label for per-trace rows.
target_view str
default='neuronio_clipped_normalized'. Target view label.
soma_scale float
default=0.1. Scale of the soma target coordinate.
soma_bias_mv float
default=DEFAULT_Y_TRAIN_SOMA_BIAS = -67.7. Millivolt offset of the soma target coordinate.
ignore_start int
default=500. Initial steps excluded.
calibrated_spike_threshold float | None
default=None. Spike threshold for model rows, calibrated on other traces.

Attributes

Constructor fields are retained as read-only attributes.

evaluate_model_diagnostic_rows

axobench.benchmark.diagnostic_rows.evaluate_model_diagnostic_rows(model_fn: Callable[[np.ndarray], np.ndarray], inputs: np.ndarray, targets: np.ndarray, *, context: DiagnosticContext | None=None, trace_ids: list[str] | None=None, raw_soma_mv: np.ndarray | None=None) -> list[dict[str, Any]]

Source

Evaluate a model on supplied arrays and return rows for trace fidelity, state-conditioned dynamics, spike behavior, feature fidelity, and mechanistic response. Exported from axobench.benchmark.

Parameters

model_fn Callable[[np.ndarray], np.ndarray]
required. Prediction function mapping float32 inputs (batch, time, 1278) to (batch, time, 2) outputs: a spike score and the soma target coordinate. With supports_morphology_ids=True it is called with a morphology_ids keyword.
inputs np.ndarray
required. Inputs, (trace, time, channels).
targets np.ndarray
required. Teacher targets, (trace, time, 2).
context DiagnosticContext | None
keyword-only, default=None. Row metadata and suite selection; None uses the function's default context.
trace_ids list[str] | None
keyword-only, default=None. Sample IDs, one per trace; the text before the first ':' is the morphology or cell-type group.
raw_soma_mv np.ndarray | None
keyword-only, default=None. Optional unclipped teacher soma voltage in millivolts, shaped (trace, time); replaces the voltage reconstructed from the target.

Returns

rows list[dict[str, Any]]
Rows of the selected suites.

teacher_qc_diagnostic_rows

axobench.benchmark.diagnostic_rows.teacher_qc_diagnostic_rows(inputs: np.ndarray, targets: np.ndarray, *, context: DiagnosticContext | None=None, raw_soma_mv: np.ndarray | None=None, trace_ids: list[str] | None=None) -> list[dict[str, Any]]

Source

Teacher distribution rows and, with raw voltage, clipping-distortion rows. Exported from axobench.benchmark.

Parameters

inputs np.ndarray
required. Inputs, (trace, time, channels).
targets np.ndarray
required. Teacher targets, (trace, time, 2).
context DiagnosticContext | None
keyword-only, default=None. Row metadata and suite selection; None uses the function's default context.
raw_soma_mv np.ndarray | None
keyword-only, default=None. Optional unclipped teacher soma voltage in millivolts, shaped (trace, time); replaces the voltage reconstructed from the target.
trace_ids list[str] | None
keyword-only, default=None. Sample IDs, one per trace; the text before the first ':' is the morphology or cell-type group.

Returns

rows list[dict[str, Any]]
Rows of the selected suites.

paired_intervention_diagnostic_rows

axobench.benchmark.diagnostic_rows.paired_intervention_diagnostic_rows(baseline_inputs: np.ndarray, baseline_targets: np.ndarray, intervention_inputs: np.ndarray, intervention_targets: np.ndarray, *, model_fn: Callable[[np.ndarray], np.ndarray] | None=None, context: DiagnosticContext | None=None, trace_ids: list[str] | None=None) -> list[dict[str, Any]]

Source

Teacher response rows for paired inputs and, with a model, model response rows: voltage-change errors, Response SERA, correlation, gain, spike-count deltas, and direction agreement. The model’s spike threshold is context.calibrated_spike_threshold, or 0. Exported from axobench.benchmark.

Parameters

baseline_inputs np.ndarray
required. Baseline inputs.
baseline_targets np.ndarray
required. Baseline teacher targets.
intervention_inputs np.ndarray
required. Edited inputs.
intervention_targets np.ndarray
required. Edited teacher targets.
model_fn Callable[[np.ndarray], np.ndarray] | None
keyword-only, default=None. Optional prediction function; None writes teacher rows only.
context DiagnosticContext | None
keyword-only, default=None. Row metadata and suite selection; None uses the function's default context.
trace_ids list[str] | None
keyword-only, default=None. Sample IDs, one per trace; the text before the first ':' is the morphology or cell-type group.

Returns

rows list[dict[str, Any]]
Rows of the selected suites.

morphology_routing_qc_rows

axobench.benchmark.diagnostic_rows.morphology_routing_qc_rows(*, context: DiagnosticContext | None=None) -> list[dict[str, Any]]

Source

Rows of branch counts and shape ratios for the v1 morphology registry. Exported from axobench.benchmark.

Parameters

context DiagnosticContext | None
keyword-only, default=None. Row metadata and suite selection; None uses the function's default context.

Returns

rows list[dict[str, Any]]
Rows with role, Cre line, reconstruction type, SWC path, and caveat.

morphology_contrast_diagnostic_rows

axobench.benchmark.diagnostic_rows.morphology_contrast_diagnostic_rows(source_rows: Iterable[dict[str, Any]], *, context: DiagnosticContext | None=None, min_morphologies: int=2) -> list[dict[str, Any]]

Source

Mean, standard deviation, range, minimum, and maximum across morphologies of every numeric metric present for at least min_morphologies morphologies, grouped by diagnostic, metric, state, model, and condition. Exported from axobench.benchmark.

Parameters

source_rows Iterable[dict[str, Any]]
required. Rows with morphology labels.
context DiagnosticContext | None
keyword-only, default=None. Row metadata and suite selection; None uses the function's default context.
min_morphologies int
keyword-only, default=2. Minimum number of morphologies per contrast; at least 2.

Returns

rows list[dict[str, Any]]
morphology_contrast rows with the per-morphology values attached.

validate_diagnostic_rows

axobench.benchmark.diagnostic_rows.validate_diagnostic_rows(rows: Iterable[dict[str, Any]]) -> None

Source

Raise ValueError if a row lacks a column of DIAGNOSTIC_ROW_COLUMNS. Exported from axobench.benchmark.

Parameters

rows Iterable[dict[str, Any]]
required. Diagnostic row dictionaries.

write_diagnostic_rows

axobench.benchmark.diagnostic_rows.write_diagnostic_rows(rows: Iterable[dict[str, Any]], path: str | Path) -> None

Source

Validate and write rows as .json, .jsonl, or .csv. Exported from axobench.benchmark.

Parameters

rows Iterable[dict[str, Any]]
required. Diagnostic row dictionaries.
path str | Path
required. Destination ending in .json, .jsonl, or .csv.

run_teacher_qc_on_selector

axobench.benchmark.runner.run_teacher_qc_on_selector(selector: DatasetSelector, *, context: DiagnosticContext | None=None, raw_soma_mv: np.ndarray | None=None, max_samples: int | None=None, output: str | Path | None=None, cache_shards: int=1) -> list[dict[str, Any]]

Source

Teacher QC rows for the first max_samples traces of a selector, in index order. Exported from axobench.benchmark.

Parameters

selector DatasetSelector
required. Resolved split or intervention directory from V1DatasetLayout.
context DiagnosticContext | None
keyword-only, default=None. Row metadata and suite selection; None uses the function's default context.
raw_soma_mv np.ndarray | None
keyword-only, default=None. Optional raw voltage; None reads soma_mv from the shards.
max_samples int | None
keyword-only, default=None. Maximum traces; None reads all.
output str | Path | None
keyword-only, default=None. Optional .json, .jsonl, or .csv path for the rows.
cache_shards int
keyword-only, default=1. Decoded shards kept in memory by the reader.

Returns

rows list[dict[str, Any]]
Teacher QC rows.

run_model_diagnostics_on_selector

axobench.benchmark.runner.run_model_diagnostics_on_selector(model_fn: Callable[[np.ndarray], np.ndarray], selector: DatasetSelector, *, context: DiagnosticContext | None=None, max_samples: int | None=None, output: str | Path | None=None, cache_shards: int=1) -> list[dict[str, Any]]

Source

Model diagnostic rows for the first max_samples traces of a selector, with raw voltage when stored. Exported from axobench.benchmark.

Parameters

model_fn Callable[[np.ndarray], np.ndarray]
required. Prediction function mapping float32 inputs (batch, time, 1278) to (batch, time, 2) outputs: a spike score and the soma target coordinate. With supports_morphology_ids=True it is called with a morphology_ids keyword.
selector DatasetSelector
required. Resolved split or intervention directory from V1DatasetLayout.
context DiagnosticContext | None
keyword-only, default=None. Row metadata and suite selection; None uses the function's default context.
max_samples int | None
keyword-only, default=None. Maximum traces; None reads all.
output str | Path | None
keyword-only, default=None. Optional .json, .jsonl, or .csv path for the rows.
cache_shards int
keyword-only, default=1. Decoded shards kept in memory by the reader.

Returns

rows list[dict[str, Any]]
Model diagnostic rows.

run_paired_intervention_on_selector

axobench.benchmark.runner.run_paired_intervention_on_selector(selector: DatasetSelector, *, model_fn: Callable[[np.ndarray], np.ndarray] | None=None, context: DiagnosticContext | None=None, max_samples: int | None=None, output: str | Path | None=None) -> list[dict[str, Any]]

Source

Paired intervention rows for the first max_samples pairs of a condition directory, across its shards in sorted file order. Exported from axobench.benchmark.

Parameters

selector DatasetSelector
required. Resolved split or intervention directory from V1DatasetLayout.
model_fn Callable[[np.ndarray], np.ndarray] | None
keyword-only, default=None. Optional prediction function; None writes teacher rows only.
context DiagnosticContext | None
keyword-only, default=None. Row metadata and suite selection; None uses the function's default context.
max_samples int | None
keyword-only, default=None. Maximum pairs; None reads all.
output str | Path | None
keyword-only, default=None. Optional .json, .jsonl, or .csv path for the rows.

Returns

rows list[dict[str, Any]]
Intervention rows.

run_paired_intervention_on_npz

axobench.benchmark.runner.run_paired_intervention_on_npz(path: str | Path, *, model_fn: Callable[[np.ndarray], np.ndarray] | None=None, context: DiagnosticContext | None=None, max_samples: int | None=None, output: str | Path | None=None) -> list[dict[str, Any]]

Source

Paired intervention rows for one pair file. Exported from axobench.benchmark.

Parameters

path str | Path
required. Pair shard.
model_fn Callable[[np.ndarray], np.ndarray] | None
keyword-only, default=None. Optional prediction function; None writes teacher rows only.
context DiagnosticContext | None
keyword-only, default=None. Row metadata and suite selection; None uses the function's default context.
max_samples int | None
keyword-only, default=None. Maximum pairs; None reads all.
output str | Path | None
keyword-only, default=None. Optional .json, .jsonl, or .csv path for the rows.

Returns

rows list[dict[str, Any]]
Intervention rows.

run_v1_benchmark_bundle

axobench.benchmark.bundle.run_v1_benchmark_bundle(*, dataset_root: str | Path, output_dir: str | Path, base_context: DiagnosticContext | None=None, model_fn: Callable[[np.ndarray], np.ndarray] | None=None, model_id: str | None=None, intervention_pair_paths: Iterable[str | Path]=(), suite: SuiteProfile='paper-summary', max_val_samples: int | None=None, max_intervention_samples: int | None=None, cache_shards: int=1, write_summaries: bool=True) -> dict[str, Any]

Source

Write teacher QC rows for val, morphology routing rows, intervention rows per condition and morphology, morphology-contrast rows, and, with a model, model rows, together with Markdown summaries and a manifest.json. Exported from axobench.benchmark.

Parameters

dataset_root str | Path
keyword-only, required. Evaluation root containing val/ and optionally interventions/<condition>/.
output_dir str | Path
keyword-only, required. Directory for the rows, summaries, and manifest.
base_context DiagnosticContext | None
keyword-only, default=None. Context copied into every row; its morphology_id labels the val rows.
model_fn Callable[[np.ndarray], np.ndarray] | None
keyword-only, default=None. Optional prediction function; None runs the teacher bundle only.
model_id str | None
keyword-only, default=None. Model name; defaults to base_context.model_id.
intervention_pair_paths Iterable[str | Path]
keyword-only, default=(). Explicit pair files; empty discovers every condition under <root>/interventions.
suite SuiteProfile
keyword-only, default='paper-summary'. teacher-qc, paper-summary, or full.
max_val_samples int | None
keyword-only, default=None. Maximum validation traces, read in index order.
max_intervention_samples int | None
keyword-only, default=None. Maximum pairs per condition and morphology.
cache_shards int
keyword-only, default=1. Decoded shards kept in memory by the reader.
write_summaries bool
keyword-only, default=True. Also write a Markdown summary per row file.

Returns

manifest dict[str, Any]
Settings and the written artifacts with row counts.

summarize_diagnostic_rows

axobench.benchmark.reports.summarize_diagnostic_rows(rows: list[dict[str, Any]]) -> dict[str, Any]

Source

Compact summary of rows: counts, diagnostics, suites, models, teachers, morphologies, conditions, example trace IDs, and numeric and label metric summaries. Exported from axobench.benchmark.

Parameters

rows list[dict[str, Any]]
required. Diagnostic row dictionaries.

Returns

summary dict[str, Any]
JSON-compatible summary.

write_diagnostic_summary

axobench.benchmark.reports.write_diagnostic_summary(rows: list[dict[str, Any]], path: str | Path) -> None

Source

Write summarize_diagnostic_rows as Markdown or JSON. Exported from axobench.benchmark.

Parameters

rows list[dict[str, Any]]
required. Diagnostic row dictionaries.
path str | Path
required. Destination ending in .md or .json.

Search guides, examples, and API signatures.