Catalog#

class hydromodpy.results.Catalog(workspace_path, *, catalog_path=None, runs_dir=None, persistence=None, read_only=False)[source]#

Bases: LifecycleMixin, RegistrationMixin, WritesMixin, ReadsMixin, DiscoveryMixin, SchemaDiscoveryMixin, PackageIOMixin

Workspace-level catalog of finished simulations.

Backed by DuckDB for tabular state (simulations, parameters, metrics, provenance, calibration sessions) and by Zarr / Parquet for field arrays and timeseries written under <project>/runs/<run>/.

Inspecting is reading: every caller that only reads (listings, show, SQL queries, figures, reports) opens with read_only=True, which installs no schema, runs no migration and leaves the index file byte for byte as it found it. A writable handle is for code that produces runs.

The facade owns four pieces of state:

_db: a CatalogConnection (lazy DuckDB connection released while idle, re-acquired with retry on contention).
_workspace: the project catalog root.
_paths: a StoragePathResolver translating simulation ids to run directories and the Parquet/Zarr paths they hold.
_open_zarr_handles: live SimulationZarr handles, tracked so finalize and close can release them deterministically.

Parameters#

workspace_path: Workspace directory, or direct path to a .duckdb catalog file.
catalog_path: Optional explicit catalog database path.
runs_dir: Optional directory containing the per-run directories.
persistence: Which sinks the catalog writes to (index rows, Zarr fields, Parquet tables) and how they are compressed. Field arrays are always a directory store: there is no packed form.
read_only: Open the index read-only. The database file must already exist: an inspection never creates a phantom index.

Raises#

hydromodpy.core.exceptions.CatalogError: If the DuckDB catalog cannot be opened or the schema migration fails.
hydromodpy.results.errors.SchemaVersionMismatchError: If the stored catalog schema version is older than the runtime expects.

Examples#

>>> import hydromodpy as hmp
>>> catalog = hmp.open("~/hmp_workspace")  
>>> latest = catalog.latest()  
>>> latest.summary()  

See Also#

hydromodpy.results.run.Run: Per-simulation view returned by catalog queries.
hydromodpy.results.run.group.RunSet: Multi-run view returned by cohort queries.

param workspace_path:
type workspace_path:: Path | str
param catalog_path:
type catalog_path:: Path | str | None
param runs_dir:
type runs_dir:: Path | str | None
param persistence:
type persistence:: PersistenceConfig | None
param read_only:
type read_only:: bool

classmethod from_workspace(workspace, *, persistence=None)[source]#

Open the project catalog declared by a runtime workspace object.

Return type:

Catalog

Parameters:

workspace (object)
persistence (PersistenceConfig | None)

Parameters#

workspace: Object exposing project_root, catalog_path, and runs_dir.
persistence: Optional storage policy.

Returns#

Catalog: Open catalog connected to the workspace database.

param workspace:
type workspace:: object
param persistence:
type persistence:: PersistenceConfig | None

classmethod from_workspace_config(workspace, *, persistence=None)[source]#

Open the project catalog declared by a workspace configuration.

Return type:

Catalog

Parameters:

workspace (object)
persistence (PersistenceConfig | None)

Parameters#

workspace: Workspace configuration object.
persistence: Optional storage policy.

Returns#

Catalog: Open catalog connected to the configured workspace.

param workspace:
type workspace:: object
param persistence:
type persistence:: PersistenceConfig | None

classmethod from_toml(toml_path)[source]#

Open the project catalog declared in a TOML config.

Return type:: Catalog
Parameters:: toml_path (str | Path)

Parameters#

toml_path: HydroModPy TOML file with a workspace section.

Returns#

Catalog: Open catalog connected to the resolved workspace.

Raises#

FileNotFoundError: If the TOML path does not exist.
ConfigValidationError: If the TOML payload fails Pydantic validation.

param toml_path:
type toml_path:: str | Path

classmethod from_json(payload)[source]#

Open the project catalog declared in a JSON config string.

Return type:: Catalog
Parameters:: payload (str | bytes)

Parameters#

payload: JSON payload validated against HydroModPyConfig.

Returns#

Catalog: Open catalog connected to the resolved workspace.

Raises#

ConfigValidationError: If the JSON payload fails validation.

param payload:
type payload:: str | bytes

classmethod from_dict(payload)[source]#

Open the project catalog declared in a dict config payload.

Return type:: Catalog
Parameters:: payload (dict)

Parameters#

payload: Mapping validated against HydroModPyConfig.

Returns#

Catalog: Open catalog connected to the resolved workspace.

Raises#

ConfigValidationError: If the mapping fails Pydantic validation.

param payload:
type payload:: dict

property connection: CatalogConnection#: Return the DuckDB connection handle for protocol-based integrations.

property backend: CatalogBackend#: Return the storage backend port driving SQL reads and writes.

property sessions: CalibrationSessionNamespace#: Session-level verbs, composed rather than stapled onto this facade.

property workspace_path: Path#: Workspace directory that owns this catalog.

property catalog_path: Path#: Path to the project index database.

property runs_dir: Path#: Directory holding one sub-directory per run.

property project_path: Path#: Alias for workspace_path kept for protocol integrations.

run_dir_for(sim_id)[source]#

Return the run directory runs/<name> (public accessor).

Parameters:: sim_id (str | UUID)
Return type:: Path

fields_path_for(sim_id)[source]#

Return the run’s Zarr directory store (public accessor).

Parameters:: sim_id (str | UUID)
Return type:: Path

tables_dir_for(sim_id)[source]#

Return the run’s Parquet payload directory (public accessor).

Parameters:: sim_id (str | UUID)
Return type:: Path

load_dataset(filters=None, *, fields=None, include_params=True, include_metrics=True, include_env=True)[source]#

Return one xr.Dataset joining scalars and Zarr fields.

Composition entry-point for ML/DL pipelines: bundles parameters, metrics, runs_environment metadata, and (optionally) lazy Zarr fields into a single xarray.Dataset indexed by sim_id. See hydromodpy.results.catalog.dataset_loader.DatasetLoader.

Return type:

Dataset

Parameters:

filters (dict[str, Any] | None)
fields (list[str] | None)
include_params (bool)
include_metrics (bool)
include_env (bool)

Parameters#

filters: Catalog filters used to select the simulation cohort.
fields: Optional field names to include from Zarr stores.
include_params: Include parameter columns in the dataset.
include_metrics: Include metric columns in the dataset.
include_env: Include runtime environment metadata.

Returns#

xarray.Dataset: Dataset indexed by sim_id with scalar tables and optional lazy field arrays.

param filters:
type filters:: dict[str, Any] | None
param fields:
type fields:: list[str] | None
param include_params:
type include_params:: bool
param include_metrics:
type include_metrics:: bool
param include_env:
type include_env:: bool

property frame#: Every simulation as one DataFrame row (alias of list_simulations).

read(ref, var, *, time=None, layer=None, sel=None, bbox=None)[source]#

Read var for the run referenced by ref (the by-id read path).

ref is resolved through resolve() (full UUID, unique prefix, or name). For an already-resolved Run, use hydromodpy.read().

Parameters:

ref (str)
var (str)
time (int | slice | None)
layer (int | None)
sel (dict | None)
bbox (tuple[float, float, float, float] | None)

Return type:

Any

Parameters:

workspace_path (Path | str)
catalog_path (Path | str | None)
runs_dir (Path | str | None)
persistence (PersistenceConfig | None)
read_only (bool)