hmp.Project#

Project is the object-oriented face of the Python API. Where the verbs of Python API take a config and return a result, hydromodpy.project.Project keeps the resolved config, the workspace, the geographic runtime, the loaded data, the mesh and the catalog handle alive between calls. It is the setup-once, run-many surface used by notebooks, calibration loops and custom analysis scripts.

Project is a facade, not an engine. The execution engine is hydromodpy.workflow.runner.Pipeline, which runs ordered steps with checkpoint and resume support. Both drive the same hydromodpy.workflow.steps helpers; Project only offers a more interactive way to call them.

Construction#

Construction is cheap. __init__ validates the configuration, resolves the time grid and the data plan, and builds an empty runtime context. It performs no heavy I/O.

import hydromodpy as hmp

project = hmp.Project("project.toml")

The constructor is polymorphic: it accepts a TOML path, a HydroModPyConfig, a dict payload, or a JSON string, auto-detected. Keyword options are solver (auto-detected from the config otherwise), headless (disable display and postprocess runners, useful in calibration loops) and no_display.

The heavy model phase builds lazily on the first simulate() call or on the first accessor that needs it. Build it eagerly with prepare(), which returns self, or call the phase verbs one by one.

Model phase#

The model phase turns a validated config into a runnable model. Each verb is callable on its own, and each one runs the phases below it when they have not happened yet.

Method	Role
`setup_workspace()`	Bootstrap the shared runtime anchor: workspace, geographic context, domain and process objects. Opens the catalog as a side effect. Idempotent: calling it twice resets those objects.
`build_geographic(reuse_dem=False)`	Mark the geographic and domain runtime ready, record the project phase, and invalidate downstream data and mesh state. Runs `setup_workspace()` first when needed.
`rebuild_geographic(reuse_dem=False)`	Drop the setup products and rerun the geographic pipeline, invalidating the mesh.
`load_data(types=None)`	Load the external forcings declared in the `[data]` section. Restrict the work with `types`.
`reload_data(types=...)`	Reload a named subset of data variables.
`build_mesh(**overrides)`	Build the mesh used by the solver. Keyword overrides patch the mesh configuration before the step runs.
`prepare()`	Run geographic, data and mesh in order. Returns the project.

The fields of the [data] section are defined in [data] DataManagersConfig, not here.

Run phase#

simulate() runs one simulation through the configured workflow and returns its Run. It builds the model phase on first call, then runs the Pipeline. Flow parameter overrides such as Sy, K and Ss, plus the special keys thickness, first_clim and properties, are applied to the plan before the Pipeline runs. Other keywords cover the run name, resume from a workflow journal, from_step and until_step bounds, dry_run, frozen input references, no_display and parallel. A dry run, and some non-simulation workflows, return None.

run = project.simulate(name="baseline", Sy=0.05)

There is no sweep verb. A sweep is a plain Python loop over simulate():

for value in [1e-3, 5e-3, 1e-2]:
    project.simulate(name=f"sy_{value}", Sy=value)

calibrate() runs a calibration campaign on the project, either from a TOML path passed as config_path or from parameters, outputs, objective blocks and a method given in Python.

spinup() runs the cyclic spin-up loop, restarting the representative window each cycle from the previous cycle’s state until heads and lake stage converge. It defaults to the [spinup] section of the project config. The returned SpinupResult carries restart_from, ready to feed a production run’s flow.restart_from.

rerun() is a classmethod. It takes a persisted Run, rebuilds the configuration from its snapshot, applies an optional config_overrides deep-merge patch, and launches a new simulation whose parent_sim_id points at the original. A run with no config snapshot raises ConfigMissingError.

Overview, comparison, testbed and site selection are TOML workflows, not Project methods. They run through hydromodpy.run() on the [workflow] mode selector, whose values are simulation, calibration, overview, comparison, testbed and site_selection. Use Project for repeated simulations, calibration and spin-up; use hydromodpy.run() for the one-shot workflows.

Lifecycle#

A project owns an open DuckDB catalog handle and a few cached preprocessing files. close() closes the catalog and cleans the preprocessing tree. It reads the geographic object straight from the context, so closing an unused project never triggers a lazy build. The preprocessing tree survives when the geographic configuration asked for the intermediate rasters on disk.

Use the project as a context manager so close runs even when an exception is raised:

import hydromodpy as hmp

with hmp.Project("project.toml") as project:
    project.setup_workspace()
    project.build_geographic()
    project.load_data()
    project.build_mesh()
    run = project.simulate(name="baseline", Sy=0.05)

Project state#

hydromodpy.project.state.ProjectState is the typed dataclass that owns the runtime state of a project. The twenty-one private attributes that were once mutated directly on the Project instance live in this container.

The runtime state mixes immutable inputs (the resolved configuration, the solver name), cached preprocessing results (the geographic context, the mesh inputs) and live counters (the run counter, the run history). Spread as dunder-attributes on Project they made the surface noisy and resisted static typing. In one slots=True dataclass they give mypy and pyright a single typed view and read as state rather than behaviour.

The fields group by concern:

Configuration: config_path, cfg, solver, time_grid.
Display flags: headless, no_display.
Mesh inputs: mesh_section_data, external_mesh_input, mesh_constraints_mode, spatial_support_registry, requested_support_ids, requested_domain_supports.
Workflow runtime: ctx, the WorkflowContext, and store, the open Catalog.
Bookkeeping: project_name, run_counter, active_runs, last_wall_seconds, phase, data_loaded, run_history.

Project proxies private reads and writes through __getattr__ and __setattr__, so call sites that touch the legacy names keep working:

project._cfg          # reads project._state.cfg
project._config_path  # reads project._state.config_path
project._run_history  # reads project._state.run_history

The mapping lives in hydromodpy.project.state.PROJECT_ATTR_TO_STATE_FIELD. A name absent from the map falls back to the regular attribute machinery, so project._runner and project._catalog keep living on the instance. The public read-only view of the config is project.config.

Read-only properties#

config, data, runs, data_loaded, has_mesh, geographic, domain, store, time_grid, loaded_data and workflow_context expose the same runtime state that TOML workflows populate through the Pipeline. geographic, domain, store, loaded_data, runs and project[sim_id] trigger the lazy model build; config, has_mesh and time_grid do not. project[sim_id] returns the Run for that identifier.

Accessors#

Two properties return small accessor objects. They scope catalog queries and data introspection to the current project, which keeps the facade surface small while staying explicit at the call site.

project.data returns a ProjectDataAccessor. It lists the input-data variables already loaded and reports the ones the declared plan still expects.

import hydromodpy as hmp

with hmp.Project("project.toml") as project:
    df = project.data.list()        # variables loaded in cache
    todo = project.data.missing()   # declared but not loaded

Use it when a workflow step complains about a missing variable, or to confirm that a manual project.load_data(types=...) covered the expected set.

project.runs returns a ProjectRunsAccessor. It wraps the project’s Catalog and pre-filters every query by the current project name.

list() returns a DataFrame summary of every persisted run for the project.
find(**filters) filters by metadata such as solver, status or run name.
latest() returns the most recent run, or None when the project has none yet.
best(metric) returns the run ranking first on a metric stored in the catalog, highest value first.
delete(sim_id, remove_storage=True) removes a simulation from the catalog and, by default, its artefacts.

with hmp.Project("project.toml") as project:
    project.simulate(Sy=0.05, name="probe-1")
    project.simulate(Sy=0.08, name="probe-2")

    last = project.runs.latest()
    probes = project.runs.find(name="probe-1")
    best = project.runs.best("nse")

The accessor yields full Run objects, not identifiers, so the caller can chain into run.field(...), run.timeseries(...) or hmp.read(run, "head").

The catalog without a project#

A project handle is convenient for the run loop but unnecessary when the caller only wants to read previously persisted runs. hydromodpy.open() returns a Catalog rooted at one project. It is the read-side complement of simulate() and mirrors the xarray.open_dataset intent: one call, a ready-to-query object.

The argument is a project directory, the one holding project.toml and .hmp/index.duckdb. A workspace root such as ~/hydromodpy is not a project and owns no index: pass ~/hydromodpy/projects/<name> instead. With the default create=False the call raises FileNotFoundError when no index exists; pass create=True to initialise an empty catalog.

import hydromodpy as hmp

cat = hmp.open("~/hmp_workspace/projects/naizin")
last = cat.latest()
da = hmp.read(last, "head")

cat is scoped to that one project: it sees every run persisted under the project root and nothing from its neighbours. Federation across projects is the job of hydromodpy.index(), which registers one row per project root and expands a workspace root into the project roots it holds.

The catalog is the single door for queries. cat.find is the one filtered entry point and returns a RunSet; an unknown filter key raises ValueError listing the valid filters. cat.frame returns the full DataFrame. Schema discovery and selectors live on the same object: cat.describe, cat.tables, cat.columns, cat.variables, cat.metrics, cat.stations, then cat.latest, cat.best, cat.worst, cat.rank, cat[ref], cat.resolve, cat.sql and cat.read for the by-id read path. Input data is reached through hydromodpy.catalog.InputsNamespace or the hmp data CLI.

import hydromodpy as hmp

cat = hmp.open("~/proj/naizin")
sims = cat.find(solver="modflow6")
frame = cat.frame
projects = hmp.index()

Catalog and reader compose naturally inside a notebook session:

import hydromodpy as hmp

cat = hmp.open("~/hmp_workspace/projects/naizin")
run = cat.latest()

head_t0 = hmp.read(run, "head", time=0)
head_all = hmp.read(run, "head")
q_out = hmp.read(run, "discharge", sel={"station": "outlet"})

hydromodpy.read() auto-dispatches the variable name through the field registry (Zarr), the timeseries table (DuckDB) and the geographic features table (GeoParquet), so a single call handles the three storage kinds.

When to prefer TOML and the CLI#

Use TOML plus hmp run for reproducible research, teaching material and CI. Use Project when a notebook, a calibration method, a custom analysis loop or an application needs to orchestrate the same steps directly. The two surfaces execute the same workflow steps, so a script and its TOML equivalent produce the same run.

Reference#

class hydromodpy.project.Project(config, *, solver=None, headless=False, no_display=False)[source]

Setup-once, run-many interface for HydroModPy simulations.

Project(config) is cheap: it validates the configuration and builds an empty runtime context. The heavy model phase (geographic, data, mesh) is built lazily on the first simulate() (or eagerly via prepare() or the per-phase verbs build_geographic / load_data / build_mesh). Run many simulations with parameter overrides; inspect past runs through runs and inputs through data.

Parameters#

configstr, Path, HydroModPyConfig, dict, or JSON str: A TOML path, a fully-built HydroModPyConfig, a dict payload, or a JSON string (auto-detected).
solverstr, optional: Flow solver name. Auto-detected from the config, defaults to "modflow_nwt".
headlessbool, optional: Disable display and postprocess runners (useful for calibration loops where generating figures per iteration is wasteful).
no_displaybool, optional: Skip display generation for later run phases.

Examples#

>>> import hydromodpy as hmp
>>> project = hmp.Project("project.toml")  
>>> run = project.simulate(Sy=0.05)  
>>> project.close()  

param config:
type config:: str | Path | object | dict
param solver:
type solver:: str | None
param headless:
type headless:: bool
param no_display:
type no_display:: bool

Validate config and build an empty runtime context (cheap).

No heavy I/O: geographic delineation, data download and meshing are deferred to the first simulate() (or prepare()).

param config:
type config:: str | Path | object | dict
param solver:
type solver:: str | None
param headless:
type headless:: bool
param no_display:
type no_display:: bool

build_geographic(*, reuse_dem=False)[source]

Mark geographic/domain runtime ready and invalidate downstream state.

Parameters:: reuse_dem (bool)
Return type:: None

build_mesh(**overrides)[source]

Build the catchment mesh from the current geographic context.

Return type:: None

calibrate(*, config_path=None, parameters=None, outputs=None, objective_blocks=None, method=None, max_iter=None, save_runs=None, seed=None, phase=None, **kwargs)[source]

Run a calibration campaign on this project.

Three modes are supported:

TOML mode (config_path supplied): delegate to run_calibration_cli with the given TOML path. Extra keyword arguments are forwarded.
Python mode (parameters supplied): build a CalibrationConfig in memory from the declarations and run the same loop.
Embedded mode (neither supplied): use the [calibration] section carried by this project’s config, so a fully in-memory HydroModPyConfig calibrates without re-declaring parameters.

A configuration declaring [[calibration.phases]] routes to run_staged_calibration() in TOML mode, and in embedded mode when this project was built from a file. An embedded declaration on a project built in memory is refused: each phase forks a fresh configuration from the source file, and there is none. Python mode declares its own parameter space, so the phases of the project config do not apply to it.

Parameters#

config_path: Calibration TOML path for TOML mode.
parameters: Python-mode parameter declarations.
outputs: Python-mode output declarations.
objective_blocks: Python-mode objective block declarations.
method: Optimizer method name.
max_iter: Maximum number of optimizer iterations.
save_runs: Policy controlling which trial runs remain persisted.
seed: Optional optimizer seed.
phase: Run only the named phase of a staged calibration.
kwargs: Extra options forwarded to the calibration runner.

Returns#

CalibrationReport or StagedCalibrationReport or Any: Structured calibration report when return_report is true, otherwise the runner-specific result.

Raises#

ConfigMissingError: Raised when neither config_path nor parameters is supplied.
ConfigError: Raised when phase is given and config_path cannot be read, because the answer is what the file says.
CalibrationError: Raised when [[calibration.phases]] cannot be run as declared, and when phase names a phase no configuration declares.

param config_path:
type config_path:: str | Path | None
param parameters:
type parameters:: dict[str, dict] | None
param outputs:
type outputs:: dict[str, dict] | None
param objective_blocks:
type objective_blocks:: list[dict] | None
param method:
type method:: str | None
param max_iter:
type max_iter:: int | None
param save_runs:
type save_runs:: str | None
param seed:
type seed:: int | None
param phase:
type phase:: str | None

Parameters:

config_path (str | Path | None)
parameters (dict[str, dict] | None)
outputs (dict[str, dict] | None)
objective_blocks (list[dict] | None)
method (str | None)
max_iter (int | None)
save_runs (str | None)
seed (int | None)
phase (str | None)

close()[source]

Close the Catalog and clean up preprocessing files.

Return type:: None

property config: HydroModPyConfig: Validated configuration driving this project (read-only).

property data: ProjectDataAccessor: Accessor for the input-data cache scoped to this project.

property data_loaded: set[str]: Set of data types already loaded for this project.

property domain: Domain | None: Spatial domain (mesh, layers, zones). Triggers build.

property geographic: CatchmentDelineation | None: Geographic runtime object (DEM, watershed, CRS). Triggers build.

property has_mesh: bool: True once the mesh has been built for the project.

load_data(*, types=None)[source]

Load the external forcings declared in [data].

Parameters:: types (list[str] | None)
Return type:: None

property loaded_data: LoadedDataContext: Loaded data context (recharge, geology, hydrometry, etc.). Triggers build.

prepare()[source]

Eagerly build the model phase (geographic, data, mesh). Returns self.

Return type:: Project

rebuild_geographic(*, reuse_dem=False)[source]

Rerun the geographic pipeline and invalidate the mesh.

Parameters:: reuse_dem (bool)
Return type:: None

reload_data(*, types)[source]

Reload a subset of data variables without touching the others.

Parameters:: types (list[str])
Return type:: None

classmethod rerun(run, *, name=None, config_overrides=None, solver=None, headless=False, no_display=False, **overrides)[source]

Launch a new simulation from a persisted run snapshot.

run remains a read-only result view; this Project-level helper owns the orchestration required to rebuild the configuration, execute the workflow, and record the new run with parent_sim_id pointing to the original simulation.

Return type:

Run

Parameters:

run (Run)
name (str | None)
config_overrides (Mapping[str, Any] | None)
solver (str | None)
headless (bool)
no_display (bool)

Parameters#

run: Persisted run to use as the reproducible source snapshot.
name: Optional name for the derived run.
config_overrides: Deep-merge patch applied to the stored config snapshot. Keys must match HydroModPyConfig top-level fields; the merged payload is validated by Pydantic, so unknown keys raise.
solver, headless, no_display: Options forwarded to the derived Project.
overrides: Flow parameter overrides forwarded to Project.simulate().

Returns#

Run: Persisted run view for the derived simulation.

Raises#

ConfigMissingError: If run has no persisted config snapshot.
PipelineError: If the derived pipeline produces no new Run, e.g. dry_run mode.

param run:
type run:: Run
param name:
type name:: str | None
param config_overrides:
type config_overrides:: Mapping[str, Any] | None
param solver:
type solver:: str | None
param headless:
type headless:: bool
param no_display:
type no_display:: bool

property runs: ProjectRunsAccessor: Accessor for the simulation catalog scoped to this project.

setup_workspace()[source]

Bootstrap shared runtime state for the project session.

This Project-level verb prepares the workspace/catalog anchor and the shared geographic/domain/process objects used by later data, mesh, and solver phases. It is not a standalone Pipeline step; Pipeline runs get the same setup through BuildGeographicStep.

Return type:: None

simulate(*, name=None, resume=None, from_step=None, until_step=None, dry_run=False, frozen=False, no_display=False, parallel=True, **overrides)[source]

Run one simulation through the configured workflow and return its result.

Builds the model phase on first call (lazy), then runs the Pipeline. Flow parameter overrides (Sy, K, Ss) and the special keys thickness, first_clim, properties are applied to the plan before the Pipeline runs. Call once per point to sweep a parameter.

Return type:

Run | None

Parameters:

name (str | None)
resume (str | None)
from_step (str | int | None)
until_step (str | int | None)
dry_run (bool)
frozen (bool)
no_display (bool)
parallel (bool)

Parameters#

name: Optional run name persisted in the catalog.
resume: Existing run identifier to resume from the workflow journal.
from_step, until_step: Optional step bounds for partial workflow execution.
dry_run: Build and validate the workflow without executing solver work.
frozen: Require frozen input-data references.
no_display: Skip display rendering for this run.
overrides: Parameter overrides applied to the simulation plan.

Returns#

Run or None: Persisted run view for simulation workflows. Dry runs and some non-simulation workflows may return None.

Raises#

PipelineError: If a workflow step fails during execution.
SolverError: If the configured solver crashes or fails to converge.
ResumeError: If resume references an incompatible journal state.

Examples#

>>> run = project.simulate(Sy=0.05, name="probe")  
>>> run.summary()  

Parameters#

spinup: Spin-up settings override. None uses config.spinup.
name_prefix: Prefix for the per-cycle run names recorded in the catalog.

Returns#

hydromodpy.project.spinup.SpinupResult: The loop outcome (converged state, restart_from handle).

param spinup:
type spinup:: SpinupConfig | None
param name_prefix:
type name_prefix:: str

property store: Catalog | None: Open Catalog for direct queries across all runs. Triggers build.

property time_grid: ResolvedSimulationTimeGrid | ResolvedSteadySimulationTimeGrid | None: Resolved simulation time grid.

property workflow_context: WorkflowContext: Mutable workflow runtime state threaded through workflow steps.

Parameters:

config (str | Path | object | dict)
solver (str | None)
headless (bool)
no_display (bool)

hmp.Project#

Construction#

Model phase#

Run phase#

Lifecycle#

Project state#

Read-only properties#

Accessors#

The catalog without a project#

When to prefer TOML and the CLI#

Reference#

Parameters#

Examples#

Parameters#

Returns#

Raises#

Parameters#

Returns#

Raises#

Parameters#

Returns#

Raises#

Examples#

See Also#

Parameters#

Returns#

See Also#