Setup workflow

Module contract

Setup planning separates the declared installation/download/conversion steps from execution. Default setup creates local directories and installs the checkout; downloads occur only when explicitly requested. A dry run prints the plan without executing its commands.

Source revision: 306a51ed950b. Public export index.

DatasetSpec

Source

DatasetSpec(name: str, slug: str) -> None

Fields

Parameter	Type	Default
name	str	required
slug	str	required

SetupOptions

Source

SetupOptions(project_root: Path, data_dir: Path, install_kaggle: bool = False, download_data: bool = False, convert_raw: bool = False, raw_dir: Path | None = None, shard_dir: Path | None = None, shard_size: int = 128, skip_install: bool = False, datasets: tuple[DatasetSpec, ...] = DEFAULT_DATASETS, python_executable: str = sys.executable) -> None

Fields

Parameter	Type	Default
project_root	Path	required
data_dir	Path	required
install_kaggle	bool	False
download_data	bool	False
convert_raw	bool	False
raw_dir	Path | None	None
shard_dir	Path | None	None
shard_size	int	128
skip_install	bool	False
datasets	tuple[DatasetSpec, ...]	DEFAULT_DATASETS
python_executable	str	sys.executable

SetupStep

Source

SetupStep(description: str, command: str | None = None, argv: tuple[str, ...] | None = None, mkdir: Path | None = None) -> None

Fields

Parameter	Type	Default
description	str	required
command	str | None	None
argv	tuple[str, ...] | None	None
mkdir	Path | None	None

SetupPlan

Source

SetupPlan(steps: tuple[SetupStep, ...], next_steps: tuple[str, ...] = field(default_factory=tuple)) -> None

Fields

Parameter	Type	Default
steps	tuple[SetupStep, ...]	required
next_steps	tuple[str, ...]	field(default_factory=tuple)

build_setup_plan

Source

build_setup_plan(options: SetupOptions) -> SetupPlan

Parameter	Type	Default
options	SetupOptions	required

Returns SetupPlan.

run_setup_plan

Source

run_setup_plan(plan: SetupPlan, *, dry_run: bool=False) -> None

Parameter	Type	Default
plan	SetupPlan	required
dry_run	bool	False

Returns None.