Skip to content

Case Conversion (novomodelo-bridge)

novomodelo-bridge is a standalone Python package that converts NEWAVE (long-term) and DECOMP (short-term) hydrothermal dispatch cases into Novomodelo case directories and compares the source model’s published results with Novomodelo’s simulation output.

The package is maintained in a separate repository: github.com/ons-ccee-epe/novomodelo-bridge.


Terminal window
uv tool install novomodelo-bridge # isolated, on-PATH CLI (recommended)
pipx install novomodelo-bridge # alternative
pip install novomodelo-bridge # into the current environment

novomodelo-bridge requires Python 3.12 or newer. The novomodelo CLI (the solver) is installed separately (see installing novomodelo) and is needed only to run novomodelo validate and novomodelo run on the converted case.

novomodelo-python is a required runtime dependency, not an optional extra — it provides the Python bindings that convert --validate, dashboard, convert decomp’s boundary-FCF import and export dessem need, so installing novomodelo-bridge pulls it in automatically. (compare does not use the bindings: it reads Novomodelo’s simulation-output parquet files directly.)

A novomodelo-bridge release X.Y.Z targets novomodelo X.Y.Z: the converted case follows that release’s input contract. The bridge records its minimum novomodelo version (0.21.0) in the conversion manifest and uses it to decide whether the installed novomodelo-python can run convert --validate. The Dependencies section lists the pin that tracks that minimum. If the installed novomodelo-python is older than the minimum novomodelo version, convert --validate skips its validation step (printing a note) instead of failing.


A NEWAVE case goes from preflight check to results comparison in six steps.

Terminal window
novomodelo-bridge check newave <newave-dir> # preflight check; exit 2 = will not convert
novomodelo-bridge convert newave <newave-dir> <case-dir> --dry-run # convert in memory, write nothing
novomodelo-bridge convert newave <newave-dir> <case-dir> --validate # convert, then validate with novomodelo-python
novomodelo validate <case-dir> # validate the converted case
novomodelo run <case-dir> # solve; results go to <case-dir>/output/
novomodelo-bridge compare newave <newave-dir> <case-dir>/output # published NEWAVE results vs Novomodelo simulation

For a DECOMP deck, replace newave with decomp in the novomodelo-bridge commands; Converting a DECOMP Deck covers the boundary-FCF import that convert decomp runs by default.


The convert subcommand reads a source case directory and writes a complete Novomodelo case directory:

Terminal window
novomodelo-bridge convert newave /path/to/source/case /path/to/output/case

convert newave flags:

FlagDescription
--validateAfter conversion, validate the output with the novomodelo package.
--forceOverwrite the destination directory if it already contains files.
--diagnostics-json PATHAlso write the conversion diagnostics (counts + findings) as JSON.
--jsonEmit a single machine-readable JSON verdict to stdout and suppress the human-readable (Rich) output.
--dry-runRun the full conversion in memory and report what would be written, without creating or modifying the destination.
--json-float-digits NWrite every float in the case’s JSON files with N significant digits, 1 to 17. Default: 15; 17 keeps full precision. Parquet files are unaffected. Below 10 digits novomodelo can reject the case, since it cross-checks JSON values against Parquet tables and unit-group sums to a 1e-9 relative tolerance.
-v, --verbose INTEGERPrint warnings and notes in full and raise the console log level (-v INFO, -vv DEBUG). Default: 0.
--log-file PATHWrite the full DEBUG log to PATH (the console verbosity is unaffected). Missing parent directories are created; a path that cannot be opened is reported as a --log-file option error and the command exits 2.
--no-colorDisable coloured output (also honoured via the NO_COLOR env var).
--quietSuppress the summary and info notes; warnings/errors still show.

convert decomp (below) accepts this same flag set plus --no-fcf.

CodeMeaning
0Conversion completed.
1Conversion error.
2--validate reported a load failure.

A conversion that fails with exit 1 still reports the warnings found before the failure, in the console notes and in the --json diagnostics.

convert decomp uses the same 0/1/2 exit-code set. On every novomodelo-bridge command, a usage error (an unknown option, a missing argument, or an invalid option value such as an unwritable --log-file path) also exits 2, with a usage message on stderr. Under --json it prints the verdict envelope to stdout instead, with status error and one usage-error diagnostic.

With --json, convert (and every other novomodelo-bridge command) emits one JSON document to stdout instead of the Rich console output, with exactly five top-level keys in this fixed order: schema_version (2), command, status, summary, diagnostics. summary holds the command-specific payload (entity counts, for convert); diagnostics is the same findings list the Rich panels render. A convert that fails lists the warnings it found before the failure next to the error, in the conversion notes and in the --json diagnostics.

For the complete flag reference for every command, see the novomodelo-bridge CLI reference.

Terminal window
novomodelo-bridge convert decomp /path/to/deck /path/to/output/case

By default, when the deck declares its cortes/cortesh cut files (its FC records), convert decomp imports the deck’s boundary FCF as a terminal-stage novomodelo policy checkpoint via an in-process, one-iteration novomodelo pass. This import is slow and requires novomodelo-python. It is always skipped under --dry-run, and is skipped — with an informational note, not an error — when the deck declares no cut files. Pass --no-fcf to skip the boundary-FCF import for a quicker conversion with no terminal FCF.

The conversion pipeline transforms the source case’s input files into a complete Novomodelo case directory, organised by output sub-directory:

  • system/ — plant and network configuration: hydros, thermals, buses, lines, pumping stations, non-controllable sources, hydro production models, and hydro geometry.
  • scenarios/ — stochastic inputs: inflow, load, and non-controllable seasonal statistics and factors.
  • constraints/ — per-stage bounds, penalty overrides, and generic constraints.
  • Top level — study configuration, stage definitions, penalties, and initial conditions.

For the authoritative, field-level source-to-Novomodelo lineage — which deck file, record, and column feeds which output field, and what transformation is applied — see the bridge’s own generated data maps. They are produced by scripts/gen-lineage-docs.py, verified by tests against a real conversion, marked “Do not edit by hand,” and written in Portuguese: newave-data-map.md and decomp-data-map.md.

convert newave writes this file set:

output/
config.json
stages.json
penalties.json
initial_conditions.json
conversion_manifest.json
system/
hydros.json
thermals.json (conditional)
buses.json
lines.json (conditional)
pumping_stations.json
non_controllable_sources.json
hydro_production_models.json
hydro_geometry.parquet
tailrace_curves.parquet (conditional)
hydro_energy_productivity.parquet (conditional)
scenarios/
load_factors.json
non_controllable_factors.json
inflow_history.parquet
inflow_seasonal_stats.parquet
load_seasonal_stats.parquet
non_controllable_stats.parquet
constraints/
generic_parameters.json
line_bounds.parquet
hydro_bounds.parquet (conditional)
thermal_bounds.parquet (conditional)
pumping_bounds.parquet (conditional)
penalty_overrides_bus.parquet (conditional)
penalty_overrides_hydro.parquet (conditional)
generic_constraints.json (conditional: deck has generic constraints)
generic_constraint_bounds.parquet (conditional: deck has generic constraints)

Files marked (conditional) are written only when the source deck supplies the corresponding data; --dry-run writes nothing at all (the conversion report’s would_write_paths previews the set a real run would write).

A case written by convert decomp does not write scenarios/inflow_history.parquet, constraints/penalty_overrides_bus.parquet or constraints/penalty_overrides_hydro.parquet, and it also writes files that convert newave never writes:

  • system/energy_contracts.json
  • scenarios/external_inflow_scenarios.parquet, scenarios/external_load_scenarios.parquet and scenarios/external_ncs_scenarios.parquet
  • constraints/contract_bounds.parquet and constraints/hydro_unit_group_bounds.parquet
  • post_study_stages.json at the top level
  • the imported boundary under boundary/ (see Converting a DECOMP Deck)

After running both the source tool and Novomodelo on the same case, the compare subcommand aligns the source tool’s published results against Novomodelo’s simulation-output parquets:

Terminal window
novomodelo-bridge compare newave /path/to/source/case /path/to/novomodelo/output

compare newave <NEWAVE_DIR> <NOVOMODELO_OUTPUT_DIR> compares NEWAVE’s published results (MEDIAS-*.CSV, pmo.dat) against Novomodelo simulation output. compare decomp <DECOMP_DIR> <NOVOMODELO_OUTPUT_DIR> compares a DECOMP run’s published operation (dec_oper_*.csv) the same way, reading each file in the layout of the DECOMP release its header names. Both source directories must hold the case/deck and its result files directly (not nested in a subdirectory). The Productivity tab of compare newave places both models at their operating points only when the NEWAVE run has its per-plant nwlistop tables, which the nwlistop.dat plant list set to 999 produces; without them it leaves the operating points out, with a note.

compare newave compares the inflows Novomodelo reports. Those carry the raw PAR draw, which can be negative; the correction that lifts a negative draw in the water balance is reported apart, as inflow_nonnegativity_slack_m3s (simulation/hydros/). Where that correction acts, the Novomodelo incremental inflow, total inflow and natural inflow energy sit below the inflow the water balance uses.

compare and dashboard read each generic constraint’s term coefficients and row bounds from Novomodelo’s generic_constraints/resolved_echo.parquet output, as the LP received them.

FlagDescription
--tolerance FLOATLargest gap, as a fraction, at which a compared variable counts as close. Default 0.05. Env NOVOMODELO_BRIDGE_RESULTS_TOLERANCE.
--format FORMATOutput format(s): console,html,csv,parquet,json,all (comma-separated and/or repeatable). Default console,parquet,json. Env NOVOMODELO_BRIDGE_FORMAT.
--out-dir PATHDirectory for file artifacts. Default <novomodelo_output_dir>/comparison_artifacts. Env NOVOMODELO_BRIDGE_OUT_DIR.
--jsonEmit a single machine-readable JSON verdict to stdout instead of the Rich tables.

compare also shares convert’s -v/--verbose, --log-file, --no-color, and --quiet flags.

--tolerance, --format and --out-dir each take their value from the first source that sets it: the flag, then the environment variable, then the config file, then the built-in default. compare reads one config file, the first of these that exists: novomodelo-bridge.toml in the current directory or any parent; $XDG_CONFIG_HOME/novomodelo-bridge/config.toml, only when XDG_CONFIG_HOME is set and non-empty; ~/.config/novomodelo-bridge/config.toml. It is not merged with the others. It can set [compare.results] tolerance, [compare] format and [compare] out_dir; a malformed file, or a key of the wrong type, is ignored with a warning on stderr.

compare judges each variable by its gap: the summed absolute difference between the two models over every compared entity and stage, relative to the summed absolute source-model values. A variable is close up to --tolerance, far beyond four times it, and moderate in between. The --json status is ok only when every compared variable of the simulated operation is close, mismatch otherwise, and no-comparable-rows when no variable was compared.

compare is informational: it reports every divergence it finds, beyond tolerance or not, but a divergence never fails the run. On both the newave and decomp tracks the exit code reports only whether the comparison could run:

Exit codeMeaning
0The comparison ran (whether or not any divergence was found).
1The source case or deck is not found: its directory is missing or holds no NEWAVE case or DECOMP deck. On compare decomp, also a deck the bridge refuses (source-unsupported, source-field-parse or source-file-missing).
2An invalid --format token; a Novomodelo output file the comparison needs that cannot be read (missing or malformed data); or, on compare decomp, a case converted from another deck (case-deck-mismatch).

A result file the report can go without, on either side, does not stop compare when it is missing or cannot be read: the report leaves out what the file feeds and says so in a note or a warning. On compare decomp these are the DECOMP result files, such as relato.rvN or a dec_oper_*.csv written by a DECOMP release older than every layout the bridge reads.

A NEWAVE case whose dger.dat requests training without a final simulation (tipo_execucao 1 with tipo_simulacao_final 0) converts with simulation.enabled set to false. novomodelo run then writes no simulation output, so compare newave finds no Novomodelo simulation output to read and exits 2, naming the missing simulation/hydro_bus_generation/ directory.


For programmatic use, install novomodelo-bridge into your project environment (pip install novomodelo-bridge; an isolated uv tool or pipx install exposes only the CLI), then import the conversion pipeline directly:

from pathlib import Path
from novomodelo_bridge.newave.pipeline import convert_newave_case
report = convert_newave_case(
Path("/path/to/source/case"),
Path("/path/to/output/case"),
on_phase=lambda phase: print(f"phase: {phase}"),
dry_run=False,
)
print(report)

convert_newave_case(src, dst, *, force=False, on_phase=None, dry_run=False, float_digits=15) -> ConversionReport takes src/dst as positional Paths, plus four keyword-only arguments: force (default False), without which a dst that already holds files raises OutputExistsError; an optional on_phase progress callback invoked with the current conversion phase’s name; dry_run (default False) — set it to True to run the full conversion in memory and populate the report without writing anything to dst; and float_digits, the significant digits of the floats in the case’s JSON files (None keeps full precision).

The returned ConversionReport carries hydro_count, thermal_count, bus_count, line_count, stage_count, diagnostics (structured findings), warnings (flat warning strings), and would_write_paths (the output paths written, or that would have been written under dry_run=True).

DECOMP conversions use the equivalent novomodelo_bridge.decomp.pipeline.convert_decomp_case(src, dst, *, force=False, on_phase=None, dry_run=False, fcf_inputs_out=None, float_digits=15) -> ConversionReport.


check validates a source case or deck without converting or writing any output — a preflight check before committing to a full convert run:

Terminal window
novomodelo-bridge check newave /path/to/source/case
novomodelo-bridge check decomp /path/to/deck

check decomp also reports what the conversion will leave behind (deck features convert decomp does not carry over), so a limitation is never a silent omission. check and convert warn when a fixed-column input file holds a tab character, which shifts every field after it on its line.

The text files arquivos.dat names on the NEWAVE track, and dadger and dadgnl on the DECOMP track, are read by fixed columns, so a tab where the layout expects spaces shifts the fields after it, and a DECOMP register line that starts with a tab is not read at all. check and convert warn about every tab in those files, naming each file and its lines (DECOMP comment lines, whose first non-blank character is &, are skipped), and a check that finds one reports ready with warnings.

check shares convert’s --json, -v/--verbose, --log-file, --no-color, and --quiet flags.

CodeMeaning
0Ready to convert.
1Ready, with warnings.
2Will not convert.

dashboard builds an interactive HTML dashboard from Novomodelo simulation results:

Terminal window
novomodelo-bridge dashboard /path/to/novomodelo/case
FlagDescription
--output, -o PATHOutput HTML file path. Default <case_dir>/dashboard.html.
--openOpen the generated dashboard in the default web browser after writing.
--jsonEmit a single machine-readable JSON verdict to stdout.

dashboard also shares convert’s -v/--verbose, --log-file, --no-color, and --quiet flags. Exit codes: 0 written, 1 error.

dashboard also builds for a case run with training or simulation disabled, and leaves out the tabs and sections of the phase that did not run. Its Constraints tab, like those of compare, shows each generic constraint as Novomodelo resolved it, read from the run’s generic_constraints/resolved_echo.parquet (the resolved echo).

export dessem writes the policy of a solved case converted from a DECOMP deck as the future cost function DESSEM reads:

Terminal window
novomodelo-bridge export dessem /path/to/deck /path/to/novomodelo/case

export dessem <DECOMP_DIR> <CASE_DIR> reads the policy checkpoint under <CASE_DIR>/output/policy/ through novomodelo-python and writes mapcut.<rev>, cortdeco.<rev> and export_manifest.json. DECOMP_DIR is the deck the case was converted from: its caso.dat names the revision <rev>, its hidr.dat codes label the plants, and the order of its VI records numbers the plants with a water travel time, whose in-transit water the cuts carry. A plant with a travel time but no VI record in the deck, a travel time that is not a whole number of hours, a plant with a travel time but no downstream plant, and a case converted from another deck (case-deck-mismatch) are refused. Plants marked NW are left out of the cuts, and an operated plant that drains into one is refused. The cuts of the first cut-building node carry each anticipated-thermal (GNL) plant’s coefficient for the delivery of the case’s second stage, the same on every load block, and mapcut record 9 carries that stage’s load-block hours; the policy’s other GNL coefficients are left out. A plant with no coefficient for that delivery, or a coefficient of exactly zero, is written as 0.0 with a warning. Each stage’s mapcut record 10 carries the cost of the GNL generation committed before the study, priced from the commitment windows in the case’s initial_conditions.json.

FlagDescription
--output, -o PATHOutput directory. Default <case_dir>/dessem_fcf/.
--forceOverwrite the outputs if the output directory already holds them.
--audit-csvAlso write the cuts as audit CSV files.
--include-terminalInclude the terminal pool in the audit CSV files.
--jsonEmit a single machine-readable JSON verdict to stdout.

export dessem also shares convert’s -v/--verbose, --log-file, --no-color, and --quiet flags. Exit codes: 0 written, 1 error.


NEWAVE identifies subsystems and plants by arbitrary 1-based codes; Novomodelo uses 0-based ids with no gaps. novomodelo-bridge sorts the source codes in ascending order and assigns the ids 0, 1, 2 and so on in that order, so an id does not depend on how the deck orders its entities, and an entity keeps the same id in every output file. compare newave rebuilds the same numbering from the source case, so results trace back to the source codes. convert decomp sorts the deck’s codes the same way and adds one converter-created transshipment bus after the declared subsystems.

In a NEWAVE case, a hydro becomes a Novomodelo entity when confhd.dat marks it in service: operating (EX) or operating with an expansion still to come (EE). An EE plant operates from the first stage at the machine configuration modif.dat declares for the study start, and gains the machines listed in exph.dat as they enter service, through per-stage bounds. Plants that do not exist yet are left out, except a future plant whose exph.dat schedule fills its dead volume. Fictitious accounting plants are removed; they are identified structurally, as a plant with zero productivity that shares its inflow gauge with a generating plant and that no pumping station uses, and a cascade link through a removed plant is rewired to the next real plant downstream.

convert decomp converts the deck’s operated hydros, with the deck’s registry overrides applied, and its small plants as must-run non-controllable sources. Its UH plants marked NW become hydros that enter after the horizon, so the boundary cuts price them; an NW reservoir whose deck volume is above its minimum storage, or a VI record on or into an NW plant, stops the conversion. The renewable parks of its renovaveis file, with their generation on the PEE-GER-PER-PAT-CEN or the PEE-GER-PER-PAT card, also become non-controllable sources. A park whose PEE-CONFIG-PER operating state is centralizado can be curtailed below its declared generation; a park in any other state is must-run, and so is a park with no state, with a warning. convert decomp writes a non_controllable_source.curtailment_cost of 0.0 to penalties.json, so curtailing a park is free and the cheapest way to absorb a surplus. A park declared on both generation cards, or with more than one operating state, stops the conversion.

For the full list of conversion rules, see the novomodelo-bridge NEWAVE track guide.

Both tracks set simulation.cut_set to "active", so the simulation loads the cuts active when training ended; where a conversion turns on no cut selection every cut stays active and the two sets coincide. When dger.dat turns cut selection on for either pass, convert newave sets the selection of training.cut_selection to the level1 method with check_frequency 1 and lookback_iterations 2, and when dger.dat turns the cut window on, it warns that Novomodelo has no cut window and keeps every cut. convert newave sets modeling.inflow_non_negativity to truncation_with_penalty and convert decomp sets it to penalty, whose unbounded correction keeps the boundary-FCF import’s one-iteration pass feasible when a volume target becomes a hard storage floor. convert decomp trains and simulates under the enumerated selection.

When the source case configures risk-averse optimization (CVaR), novomodelo-bridge converts the alpha and lambda parameters to per-stage risk_measure entries in stages.json. Three modes are supported:

  • Disabled — all stages use "expectation".
  • Constant — all stages use the same CVaR parameters.
  • Temporal — per-stage alpha/lambda values, with fallback to constants when a stage override is zero.

convert newave converts three families of source constraints into Novomodelo’s generic-constraint authoring format: one definition per constraint in generic_constraints.json (a name, an expression, and an optional slack), merged with sequential integer IDs, and per-stage right-hand-side bounds in generic_constraint_bounds.parquet.

  • VminOP — minimum stored-energy constraints, one per REE. Each bounds the useful stored energy, in MWh, of the REE’s participating reservoirs at the end of the stage from below by the curve’s fraction of their maximum useful stored energy. Each reservoir’s weight and maximum are named parameters (@rho_acum_mwh_h<id>, @earm_max_mwh_h<id>) declared in constraints/generic_parameters.json as computed parameters (integrated_accumulated_productivity_mwh_per_hm3, max_stored_energy_mwh), which Novomodelo resolves from the study’s own productivities (see the security-curve example). The slack’s penalty basis is per_unit, a price per MWh of violation.
  • Electric — operational constraints on hydro and thermal generation and line flows, whose demand and constant terms fold into the bounds, with a per_hour slack priced at the penalid.dat electric-constraint penalty, or at 10 times the largest deficit cost when the case declares none. A constraint with a term the case cannot carry is left out, with a warning.
  • AGRINT — exchange-group limits, each a cap on a weighted sum of directional line flows, with no slack.

convert decomp converts the deck’s RE, RHQ, RHV and LIBS electrical constraints with a per_hour slack and its RHE stored-energy constraints with a per_unit slack. An FE term of an RE constraint becomes a term on the CI import or CE export contract with that number in its submarket; a term that matches no single such contract drops its constraint, with a decomp-fe-no-contract warning.

A LIBS restriction keeps its decision terms in the expression and folds its data terms into the bound at each stage and block: the load, the aliases, the available power, a peq_<name>_<subsystem> term (the generation of that subsystem’s PQ small-plant series) and the ger_usit of a dadgnl GNL plant (the generation its GL record commands). A peq_ term that names no PQ series drops its restriction, with a warning, and a GNL term stops the conversion when the restriction is active at a stage in which no GL week starts. A restriction declared only by RESTRICAO-ELETRICA-INEQUACAO-PERIODO-PATAMAR rows applies in the stages and blocks those rows cover, and a limit written as 1E+31 leaves that side unbounded. On both tracks a record name in the electrical-constraint file is read only in full and in upper case, as the manual writes it, and every line that is not a record the conversion reads, other than blank and & comment lines, is listed in a libs-unread-lines warning.

Each soft constraint’s slack declares its penalty_basis: VminOP is per_unit, so its penalty is charged per MWh of violation, and the electric constraints are per_hour. On the DECOMP track, RHE bounds useful stored energy the same way as VminOP, one constraint per record, and is per_unit; the RE, HQ, HV and LIBS constraints are per_hour.


novomodelo-bridge 0.21.0 installs its runtime dependencies with the package. Four pins matter:

  • novomodelo-python==0.21.0: the pinned version is the bridge’s MIN_NOVOMODELO_VERSION (0.21.0), and a packaging test keeps the pin and that constant in lockstep.
  • inewave>=1.17.2: the NEWAVE file reader.
  • idecomp>=1.18.3: the DECOMP file reader.
  • cfinterface>=1.10.4: the file-format library both readers are built on, which the bridge also imports directly.

The bridge also requires pandas 3.1, plotly 7.1, polars 2.1, pyarrow 26, rich 15 and typer 0.27.3 or newer, so installing it into an existing environment upgrades them.

The minimum novomodelo version does not make versions interchangeable: an imported boundary needs an exact version match (see the caution under Converting a DECOMP Deck).