validity_frame#

validity_frame is experimental observability tooling shipped as a standalone distribution at validity_frame/src/validity_frame/, next to hydromodpy/ and never inside it. It records runtime metadata and can ingest JSONL capture files into a DuckDB database for downstream inspection.

V1 status#

validity_frame is outside the stable V1 public API. It remains installed as an experimental package because the current code and scripts use it directly, but it has no user-facing compatibility guarantee. It must stay isolated from the modeling layers.

Role#

The package captures execution context, probes runtime and hardware metadata, and ingests JSONL records into an observability DuckDB file. It does not own scientific outputs, project catalogs or user-facing workflow state.

Sub-modules#

  • validity_frame/auto_capture/ – recorder, context and collector utilities.

  • validity_frame/probes/ – runtime, hardware, solver and system probes.

  • validity_frame/loader.py – JSONL-to-DuckDB ingestion helper.

  • validity_frame/validity.py – the capture entry point.

Sequence diagram#

Runtime envelope showing capture before, during and after simulation:

@startuml title Process - Validity Frame Runtime Sequence actor User participant "Workflow Runner\n(example script / API)" as Runner participant "RuntimeAutoCapture" as Capture participant "Adapter / Entry-point\n(create_validity_frame / external adapter)" as Adapter participant "AutoCaptureCollector" as Collector participant "System / Hardware / Runtime / Solver probes" as Probes participant "Simulation / Solver" as Simulation participant "DuckDB / JSONL sidecar" as Storage User -> Runner: launch simulation workflow Runner -> Capture: initialize context and output_dir Runner -> Adapter: resolve collector implementation Adapter -> Collector: instantiate collector (entry-point) Capture -> Collector: capture_start() Collector -> Probes: collect initial system/runtime state Probes --> Collector: start snapshot data Collector --> Capture: start snapshot Runner -> Capture: run_with_capture(work) Capture -> Simulation: execute simulation Simulation --> Capture: results or exception alt success Capture -> Collector: capture_end(start_time, solver_source, logs) Collector -> Probes: collect end state and solver metrics Probes --> Collector: completed snapshot data Collector --> Capture: completed snapshot Capture -> Runner: return snapshot Runner -> Storage: write runtime_capture_success.json\n+append runtime_capture.jsonl else failure Capture -> Collector: capture_exception(start_time, exc, solver_source, logs) Collector -> Probes: collect end state and exception details Probes --> Collector: failed snapshot data Collector --> Capture: failed snapshot Capture -> Runner: return failed snapshot Runner -> Storage: write runtime_capture_failure.json\n+append runtime_capture.jsonl end note right of Capture Collector failures are non-fatal by default: errors are logged and collection is attempted but will not stop the simulation run unless explicitly configured. end note Storage --> User: observability artifacts for inspection @enduml

Storage contract#

JSONL and the observability DuckDB file are experimental sidecars. They are not canonical scientific outputs. The allowed stable V1 stores remain DuckDB project/catalog databases, Zarr field stores and Parquet tabular outputs, as described in Storage Layout and Artifact Policy.

Layer-matrix neighbours#

  • Allowed targets: validity_frame only.

  • Allowed sources: none in the stable modeling DAG.

See also#