Skip to content

Benchmarking

Open-TYNDP includes a structured benchmarking framework that runs automatically with every workflow execution. It provides systematic, reproducible comparisons of Open-TYNDP model outcomes against official TYNDP 2024 reference data for both the Scenario Building (SB) and Cost-Benefit Analysis (CBA) phases.

Benchmarking results continue to be published with every release on Zenodo (currently only Scenario Building results). View the project website for a detailed explanation of the results.

Warning

Open-TYNDP is under active development and is not yet feature-complete. As of v0.7, Open-TYNDP was successfully benchmarked against TYNDP 2024 (see outcomes). The current development status and the general Limitations are important to understand before using the model.

Benchmarking Methodology

The benchmarking is based on the methodology proposed by Wen et al. (2022). This methodology provides a multi-criteria approach to ensure:

  • the diversity (each indicator has its own added value),
  • the effectiveness (each indicator provides essential and correct information),
  • the robustness (against diverse units and orders of magnitude), and
  • the compatibility (can be used to compare across countries) of the selected set of indicators.

This methodology defines the following accuracy indicators:

  • Missing: Count of carriers / sectors dropped due to missing values.
  • sMPE (Symmetric Mean Percentage Error): Indicates the direction of the deviation, showing if outputs are overall overestimated or underestimated.
  • sMAPE (Symmetric Mean Absolute Percentage Error): Indicates the absolute magnitude of the deviations, avoiding the cancellation of negative and positive errors.
  • sMdAPE (Symmetric Median Absolute Percentage Error): Provides skewness information to complement sMAPE.
  • RMSLE (Root Mean Square Logarithmic Error): Complements the percentage errors by showing logarithmic deviation values.
  • Growth error: Shows the error on the temporal scale. This indicator is not applied to dynamic time series (i.e. hourly generation profiles).

Scenario Building Benchmarking

Metrics

The following metrics from the TYNDP 2024 Scenarios Report are considered relevant for Scenario Building benchmarking:

  • Exogenous inputs:

  • Final energy demand by fuel, EU27 (TWh), (Fig 5, p24 and Fig 51, p63)

  • Electricity demand per sector, EU27 (TWh), (Fig 6, p25 and Fig 52, p63)
  • Methane demand by sector, EU27 (TWh), (Fig 8, p27 and Fig 53, p64)
  • Hydrogen demand by sector, EU27 (TWh), (Fig 10, p28 and Fig 54, p64)

  • Investment and dispatch modelling outputs:

  • Net installed capacity for electricity generation, EU27 (GW), (Fig 25, p39 and Fig 55, p65)

  • Electricity generation, EU27 (TWh), (Fig 26, p39 and Fig 56, p65)
  • Methane supply, EU27 (TWh), (Fig 32, p45 and Fig 57, p66)
  • Hydrogen supply, EU27 (TWh), (Fig 33, p46 and Fig 58, p67)
  • Biomass supply, EU27 (TWh), (Fig 59, p67)
  • Energy imports, EU27 (TWh), (Fig 40, p51 and Fig 60, p68)
  • Hourly generation profile of power generation, (Fig 30, p35)

The data is published in the Scenarios package. In addition to the Scenarios Report data, data from the Market Model Outputs and the Visualisation Platform are also processed and included in the relevant figures.

Hourly time series from TYNDP 2024 are aggregated to match the temporal resolution of Open-TYNDP. Summary tables are computed for both overall and per-carrier results. Spatially resolved benchmarking at bus and country level is available via the benchmarking.spatial.by_bus and benchmarking.spatial.by_country configuration keys.

Workflow

The SB benchmarking workflow is controlled by config/benchmarking.default.yaml.

  1. retrieve_tyndp: Retrieve the TYNDP 2024 Scenarios Report Data Figures package.
  2. clean_tyndp_report_benchmark: Read and process the raw Scenarios Report data into a long-format table.
  3. clean_tyndp_vp_data: Read and process the Visualisation Platform data into a long-format table.
  4. clean_tyndp_output_benchmark: Read and process the Market Model Outputs, including cross-border flows, prices, and country and EU27 level values.
  5. build_statistics: Compute benchmarking statistics from the optimised network for every planning horizon.
  6. make_benchmark: Compute accuracy indicators comparing model outcomes against the Market Model Outputs, Scenarios Report, and Visualisation Platform.
  7. make_benchmarks: Collect outputs from all make_benchmark runs.
  8. plot_benchmark: Generate visualisation outputs.
  9. plot_benchmarks: Collect outputs from all plot_benchmark runs.

The full set of benchmarking output files is stored under results/benchmarks/tyndp-2024/:

  • resources/ — processed benchmarking inputs from both Open-TYNDP and TYNDP 2024.
  • csvs_s_{clusters}_{opts}_{sector_opts}_all_years/ — quantitative tables.
  • graphics_s_{clusters}_{opts}_{sector_opts}_all_years/ — figures.
  • kpis_s_{clusters}_{opts}_{sector_opts}_all_years_by_bus.csv — summary table aggregated from bus level KPIs.
  • kpis_s_{clusters}_{opts}_{sector_opts}_all_years_by_country.csv — summary table aggregated from country level KPIs.
  • kpis_s_{clusters}_{opts}_{sector_opts}_all_years_by_bus.pdf — summary figure aggregated from bus level KPIs.
  • kpis_s_{clusters}_{opts}_{sector_opts}_all_years_by_country.pdf — summary figure aggregated from country level KPIs.

The structure of these outputs can be validated in the published preliminary benchmarking results on Zenodo.

Outputs

Example of indicators extracted from power_generation_cy2009_s_all___all_years.csv by countries for the NT scenario with hourly resolution:

Carrier sMPE sMAPE sMdAPE RMSLE Growth Error Missing countries reference version
Coal + other fossil (incl. biofuels) 0.3 0.76 0.25 8.92 -1.87 0 TYNDP 2024 Market Model Outputs v0.7.1
Hydro (exc. pump storage) 0.01 0.03 0.0 0.06 0.02 0 TYNDP 2024 Market Model Outputs v0.7.1
Hydrogen -0.93 1.19 1.21 6.0 -0.51 1 TYNDP 2024 Market Model Outputs v0.7.1
Methane (incl. biofuels) -0.21 0.34 0.17 0.65 0.16 0 TYNDP 2024 Market Model Outputs v0.7.1
Nuclear 0.08 0.08 0.07 0.08 0 0 TYNDP 2024 Market Model Outputs v0.7.1
Oil (incl. biofuels) 0.83 1.18 0.95 10.68 0.12 0 TYNDP 2024 Market Model Outputs v0.7.1
Other non-res -0.23 0.31 0.06 1.16 0 0 TYNDP 2024 Market Model Outputs v0.7.1
Other res 0 0 0 0 0 0 TYNDP 2024 Market Model Outputs v0.7.1
Solar 0 0.01 0 0.02 0 0 TYNDP 2024 Market Model Outputs v0.7.1
Wind offshore -0.1 0.11 0 6.23 0.01 0 TYNDP 2024 Market Model Outputs v0.7.1
Wind onshore 0 0 0 0 0 0 TYNDP 2024 Market Model Outputs v0.7.1
Demand shedding 1.63 1.8 2.0 13.51 0.03 0 TYNDP 2024 Market Model Outputs v0.7.1
Solar thermal — — — — — 0 TYNDP 2024 Market Model Outputs v0.7.1

Example of figure created for the final energy demand for the NT scenario in 2030 with hourly resolution:

Example of figure including Visualisation Platform data created for the power capacity for the NT scenario in 2030 with hourly resolution:

Example of figure created for the cross-border flows between countries (focusing on incorrect net direction) for the NT scenario in 2030 with hourly resolution:

Example of figure created for the electricity prices by country for the NT scenario in 2030 with hourly resolution:

Example of figure created for the generation profiles for the DE scenario in 2040 with 45SEG:

Example of summary indicators extracted from kpis_s_all__all_years_by_country.csv for the NT scenario with hourly resolution:

Metric sMPE sMAPE sMdAPE RMSLE Growth Error Missing carriers Missing countries reference version
biomass_supply 0.14 0.14 0.11 0.2 0 2 0 TYNDP 2024 Scenarios Report v0.7.1
crossborder_electricity -0.03 0.56 0.3 2.29 0 0 27 TYNDP 2024 Market Model Outputs v0.7.1
crossborder_hydrogen -0.02 0.31 0.13 0.47 0.05 0 1 TYNDP 2024 Market Model Outputs v0.7.1
electricity_demand 0 0 0 0 0 0 0 TYNDP 2024 Market Model Outputs v0.7.1
electricity_demand_shedding_hours 0.41 0.52 0 6.54 0.21 0 0 TYNDP 2024 Market Model Outputs v0.7.1
electricity_price 0.11 0.19 0.07 0.3 0.05 0 0 TYNDP 2024 Market Model Outputs v0.7.1
electricity_price_excl_shed -0.04 0.12 0.07 0.19 -0.02 0 0 TYNDP 2024 Market Model Outputs v0.7.1
energy_imports 0.52 0.52 0.3 1.15 0.01 1 0 TYNDP 2024 Scenarios Report v0.7.1
final_energy_demand 0.11 0.11 0.09 0.15 0.01 0 0 TYNDP 2024 Scenarios Report v0.7.1
generation_profiles — — — — — NA NA — v0.7.1
hydrogen_demand -0.22 0.28 0 2.96 0 0 1 TYNDP 2024 Market Model Outputs v0.7.1
hydrogen_demand_shedding_hours 0.38 0.38 0 5.64 0 0 0 TYNDP 2024 Market Model Outputs v0.7.1
hydrogen_price 0.11 0.16 0.03 0.27 0 0 0 TYNDP 2024 Market Model Outputs v0.7.1
hydrogen_price_excl_shed 0.01 0.06 0.02 0.11 0 0 0 TYNDP 2024 Market Model Outputs v0.7.1
hydrogen_supply -0.15 0.38 0.13 2.05 0.06 1 0 TYNDP 2024 Market Model Outputs v0.7.1
methane_demand -0.1 0.17 0.16 0.21 0 0 0 TYNDP 2024 Scenarios Report v0.7.1
methane_supply 0.1 0.1 0.1 0.1 0 4 0 TYNDP 2024 Scenarios Report v0.7.1
power_capacity 0 0 0 0.01 0 1 0 TYNDP 2024 Market Model Outputs v0.7.1
power_generation 0.28 0.51 0.05 6.99 0 1 1 TYNDP 2024 Market Model Outputs v0.7.1
Total (excl. time series) 0.14 0.31 0.03 1.48 0 13 — — v0.7.1

Example of summary figure created for the NT scenario with hourly resolution:

Naming conventions

To align naming conventions across data sources, Open-TYNDP uses a mapping table defined in tyndp_technology_map.csv. This table maps the different naming conventions between the Open-TYNDP, various input data sources and benchmarking tables. It contains the following columns:

  • pemmdb_carrier: Higher level carrier names as listed in the PEMMDB v2.4 dataset.
  • pemmdb_type: Specific carrier technology type as listed in the PEMMDB v2.4 dataset.
  • pemmdb_hydro_inflows: Carrier names as listed in the PEMMDB v2.4 hydro inflows dataset.
  • pecd_carrier: Carrier names as listed in the PECD v3.1 dataset.
  • investment_dataset_carrier: Carrier names as listed in the TYNDP 2024 investment dataset.
  • tyndp_output_carrier: Carrier names as listed in the TYNDP 2024 Market Model output files.
  • tyndp_report_carrier: Carrier names as listed in the TYNDP 2024 Scenarios Report.
  • tyndp_vp_carrier: Carrier names as listed on the TYNDP 2024 Visualization Platform.
  • open_tyndp_carrier: Open-TYNDP carrier naming convention representing either an energy carrier or a technology, depending on the context.
  • open_tyndp_index: Open-TYNDP technology naming convention found in the component index.
  • open_tyndp_nice_name: Open-TYNDP nice names for better readability and aggregation of technologies for reporting purposes.
  • benchmarking_capacity: Naming convention for the benchmarking framework's power_capacity table.
  • benchmarking_generation: Naming convention for the benchmarking framework's power_generation table.
  • benchmarking_generation_profiles: Naming convention for the benchmarking framework's generation_profiles table.
  • benchmarking_fed: Naming convention for the benchmarking framework's final_energy_demand table.
  • benchmarking_elec_demand: Naming convention for the benchmarking framework's electricity_demand table.
  • benchmarking_ch4_demand: Naming convention for the benchmarking framework's methane_demand table.
  • benchmarking_h2_demand: Naming convention for the benchmarking framework's hydrogen_demand table.
  • benchmarking_ch4_supply: Naming convention for the benchmarking framework's methane_supply table.
  • benchmarking_h2_supply: Naming convention for the benchmarking framework's hydrogen_supply table.
  • benchmarking_biomass_supply: Naming convention for the benchmarking framework's biomass_supply table.
  • benchmarking_energy_imports: Naming convention for the benchmarking framework's energy_imports table.
  • benchmarking_prices: Naming convention for the benchmarking framework's different price tables.
  • benchmarking_shed_hours: Naming convention for the benchmarking framework's electricity_demand_shedding_hours and hydrogen_demand_shedding_hours tables.
  • open_tyndp_type: Open-TYNDP technology type for grouping together detailed technologies if electricity:group_tyndp_conventionals is enabled in the configuration.
  • pypsa_eur_carrier: Corresponding PyPSA-Eur carrier/technology if applicable.
  • comment: Any additional comments.

The complete technology mapping is shown below. Scroll horizontally and vertically to explore the table.

pemmdb_carrier pemmdb_type pemmdb_hydro_inflows pecd_carrier investment_dataset_carrier tyndp_output_carrier tyndp_report_carrier tyndp_vp_carrier cba_storage_technology open_tyndp_carrier open_tyndp_index open_tyndp_nice_name benchmarking_capacity benchmarking_generation benchmarking_generation_profiles benchmarking_fed benchmarking_elec_demand benchmarking_ch4_demand benchmarking_h2_demand benchmarking_ch4_supply benchmarking_h2_supply benchmarking_biomass_supply benchmarking_energy_imports benchmarking_prices benchmarking_shed_hours open_tyndp_type pypsa_eur_carrier comment
Solar Photovoltaic LFSolarPVUtility Solar PV Utility Solar (Photovoltaic) Solar Solar PV Utility solar-pv-utility solar-pv-utility Solar PV (Utility) solar solar solar solar-pv-utility For non Italian nodes Utility and Rooftop share the same PECD profile called LFSolarPV
Solar Rooftop LFSolarPVRooftop Solar PV Rooftop Solar (Photovoltaic) Solar Solar PV Rooftop solar-pv-rooftop solar-pv-rooftop Solar PV (Rooftop) solar solar solar solar-pv-rooftop For non Italian nodes Utility and Rooftop share the same PECD profile called LFSolarPV
Solar PV solar solar solar
DRES Solar PV solar solar solar
Solar Thermal Solar CSP_noStorage Solar (Thermal) Solar Thermal solar-thermal solar-thermal Solar Thermal solar thermal solar thermal solar thermal solar-thermal
Solar Solar Thermal with Storage CSP_withStorage solar-thermal-w-storage solar-thermal-w-storage Solar Thermal solar thermal solar thermal solar thermal solar-thermal-w-storage
Solar Solar Thermal with Storage - Storage CSP_withStorage solar-thermal-w-storage solar-thermal-w-storage-storage Solar Thermal solar thermal solar thermal solar thermal solar-thermal-w-storage-storage
Wind Onshore wind Wind_Onshore Wind Onshore Wind Onshore Wind Onshore Wind Onshore onwind onwind Onshore Wind wind onshore wind onshore onshore wind onwind
DRES Wind On wind onshore wind onshore onshore wind
Wind Offshore wind Wind_Offshore Wind Offshore Wind Offshore Wind Offshore offwind offwind Offshore Wind wind offshore wind offshore offshore wind offwind only used w/o offshore hubs
Wind_Offshore Wind Offshore Wind Offshore Wind Offshore offwind-ac-fb-r offwind-ac-fb-r Offshore Wind (AC) wind offshore wind offshore offshore wind offwind-ac-fb-r only used with offshore hubs
Wind_Offshore Wind Offshore Wind Offshore Wind Offshore offwind-ac-fl-r offwind-ac-fl-r Offshore Wind (AC Float.) wind offshore wind offshore offshore wind offwind-ac-fl-r only used with offshore hubs
Wind_Offshore Wind Offshore Wind Offshore Wind Offshore offwind-dc-fb-r offwind-dc-fb-r Offshore Wind (DC) wind offshore wind offshore offshore wind offwind-dc-fb-r only used with offshore hubs
Wind_Offshore Wind Offshore Wind Offshore Wind Offshore offwind-dc-fl-r offwind-dc-fl-r Offshore Wind (DC Float.) wind offshore wind offshore offshore wind offwind-dc-fl-r only used with offshore hubs
Wind_Offshore Wind Offshore Wind Offshore Wind Offshore offwind-dc-fb-oh offwind-dc-fb-oh Offshore Wind (DC OH) wind offshore wind offshore offshore wind offwind-dc-fb-oh only used with offshore hubs
Wind_Offshore Wind Offshore Wind Offshore Wind Offshore offwind-dc-fl-oh offwind-dc-fl-oh Offshore Wind (DC Float. OH) wind offshore wind offshore offshore wind offwind-dc-fl-oh only used with offshore hubs
Wind_Offshore Wind Offshore Wind Offshore Wind Offshore offwind-h2-fb-oh offwind-h2-fb-oh Offshore Wind (H2 OH) wind offshore wind offshore offshore wind offwind-h2-fb-oh only used with offshore hubs
Wind_Offshore Wind Offshore Wind Offshore Wind Offshore offwind-h2-fl-oh offwind-h2-fl-oh Offshore Wind (H2 Float. OH) wind offshore wind offshore offshore wind offwind-h2-fl-oh only used with offshore hubs
DRES Wind Off Offshore Wind (DRES) wind offshore wind offshore offshore wind
Hydro Run of River - Turbine Run_of_River Run-of-River Hydro and pumped storage Run-of-River hydro-ror hydro-ror Run of River hydro and pumped storage hydro (exc. pump storage) hydro hydro-ror-turbine
Hydro Pondage - Reservoir Pondage hydro-pondage hydro-pondage-reservoir Hydro Pondage (Reservoir) hydro-pondage-reservoir
Hydro Pondage - Turbine Pondage Pondage Hydro and pumped storage Pondage hydro-pondage hydro-pondage Hydro Pondage hydro and pumped storage hydro (exc. pump storage) hydro hydro-pondage-turbine
Hydro Reservoir - Reservoir Reservoir hydro-reservoir hydro-reservoir-reservoir Reservoir & Dam (Reservoir) hydro-reservoir-reservoir
Hydro Reservoir - Turbine Reservoir Reservoir Hydro and pumped storage Reservoir hydro-reservoir hydro-reservoir Reservoir & Dam hydro and pumped storage hydro (exc. pump storage) hydro hydro-reservoir-turbine
Hydro (exc. Pump storage) hydro (exc. pump storage)
Hydro Pump Storage - Reservoir PS_Open HydroPumpedStorage hydro-phs hydro-phs-reservoir Pumped Hydro (Reservoir) hydro-phs-reservoir
Hydro Pump Storage - Turbine PS_Open Pump Storage - Open Loop (turbine) Hydro and pumped storage Pump Storage - Open Loop (turbine) hydro-phs hydro-phs-turbine Pumped Hydro hydro and pumped storage hydro hydro-phs-turbine
PS Open hydro-phs Pumped Hydro Open hydro and pumped storage
Hydro Pump Storage - Pump PS_Open Pump Storage - Open Loop (pump) hydro-phs hydro-phs-pump Pumped Hydro hydro and pumped storage (load) hydro hydro-phs-pump
hydro-phs-inflows Pumped Hydro (Inflows)
Hydro Pure Pump Storage - Reservoir PS_Closed hydro-phs-pure hydro-phs-pure-reservoir Pumped Hydro (Reservoir) hydro-phs-pure-reservoir
Hydro Pure Pump Storage - Turbine PS_Closed Pump Storage - Closed Loop (turbine) Hydro and pumped storage Pump Storage - Closed Loop (turbine) hydro-phs-pure hydro-phs-pure-turbine Pumped Hydro hydro and pumped storage hydro hydro-phs-pure-turbine
PS Closed hydro-phs-pure Pumped Hydro Closed hydro and pumped storage
Hydro Pure Pump Storage - Pump PS_Closed Pump Storage - Closed Loop (pump) hydro-phs-pure hydro-phs-pure-pump Pumped Hydro hydro and pumped storage (load) hydro hydro-phs-pure-pump
Other RES Small Biomass Others renewable Small Scale RES Other RES other-res other-res-biomass Other Renewables other res other res other res other res other-res-biomass
Other RES Geothermal, Marine, Waste and Not Defined Others renewable Small Scale RES Other RES other-res other-res-mix Other Renewables other res other res other res other-res-mix
Other RES Geothermal, Marine, Waste and Not Defined Other RES other res The TYNDP 2024 Scenarios Report maps Other RES capacities as 'Small Scale RES' and Other RES generation as 'Other RES'
Gas Conventional old 1 Gas conventional old 1 Methane Gas conventional gas gas-conv-old1 Gas Conventional methane (incl. biofuels) methane (incl. biofuels) methane&hydrogen power generation gas-conv OCGT Methane demand benchmark
Gas Conventional old 2 Gas conventional old 2 Methane Gas conventional gas gas-conv-old2 Gas Conventional methane (incl. biofuels) methane (incl. biofuels) methane&hydrogen power generation gas-conv OCGT Methane demand benchmark
Gas CCGT old 1 Gas CCGT old 1 Methane Gas CCGT gas gas-ccgt-old1 Gas CCGT methane (incl. biofuels) methane (incl. biofuels) methane&hydrogen power generation gas-ccgt CCGT Methane demand benchmark
Gas CCGT old 2 Gas CCGT old 2 Methane Gas CCGT gas gas-ccgt-old2 Gas CCGT methane (incl. biofuels) methane (incl. biofuels) methane&hydrogen power generation gas-ccgt CCGT Methane demand benchmark
Gas CCGT new Gas CCGT new Methane Gas CCGT gas gas-ccgt-new Gas CCGT methane (incl. biofuels) methane (incl. biofuels) methane&hydrogen power generation gas-ccgt CCGT Methane demand benchmark
Gas CCGT CCS Gas CCGT CCS Methane Gas CCGT CCS gas gas-ccgt-ccs Gas CCGT (CCS) methane (incl. biofuels) methane (incl. biofuels) methane&hydrogen power generation gas-ccgt-ccs Methane demand benchmark
Gas OCGT old Gas OCGT old Methane Gas OCGT gas gas-ocgt-old Gas OCGT methane (incl. biofuels) methane (incl. biofuels) methane&hydrogen power generation gas-ocgt OCGT Methane demand benchmark
Gas OCGT new Gas OCGT new Methane Gas OCGT gas gas-ocgt-new Gas OCGT methane (incl. biofuels) methane (incl. biofuels) methane&hydrogen power generation gas-ocgt OCGT Methane demand benchmark
Gas CCGT present 1 Gas CCGT present 1 Methane Gas CCGT gas gas-ccgt-present1 Gas CCGT methane (incl. biofuels) methane (incl. biofuels) methane&hydrogen power generation gas-ccgt CCGT Methane demand benchmark
Gas CCGT present 2 Gas CCGT present 2 Methane Gas CCGT gas gas-ccgt-present2 Gas CCGT methane (incl. biofuels) methane (incl. biofuels) methane&hydrogen power generation gas-ccgt CCGT Methane demand benchmark
Methane Gas methane (incl. biofuels) methane (incl. biofuels) methane&hydrogen power generation
Nuclear Nuclear Nuclear Nuclear Nuclear Nuclear uranium nuclear Nuclear nuclear nuclear nuclear nuclear nuclear
Hard coal old 1 Hard coal old 1 Coal + Other fossil Hard coal coal coal-old1 Coal coal + other fossil (incl. biofuels) coal + other fossil (incl. biofuels) coal&lignite&oil coal coal
Hard coal old 2 Hard coal old 2 Coal + Other fossil Hard coal coal coal-old2 Coal coal + other fossil (incl. biofuels) coal + other fossil (incl. biofuels) coal&lignite&oil coal coal
Hard coal new Hard coal new Coal + Other fossil Hard coal coal coal-new Coal coal + other fossil (incl. biofuels) coal + other fossil (incl. biofuels) coal&lignite&oil coal coal
Hard coal CCS Hard coal CCS Coal + Other fossil Hard coal coal coal-ccs Coal (CCS) coal + other fossil (incl. biofuels) coal + other fossil (incl. biofuels) coal&lignite&oil coal-ccs
Lignite old 1 Lignite old 1 Coal + Other fossil Lignite lignite lignite-old1 Lignite coal + other fossil (incl. biofuels) coal + other fossil (incl. biofuels) coal&lignite&oil lignite lignite
Lignite old 2 Lignite old 2 Coal + Other fossil Lignite lignite lignite-old2 Lignite coal + other fossil (incl. biofuels) coal + other fossil (incl. biofuels) coal&lignite&oil lignite lignite
Lignite new Lignite new Coal + Other fossil Lignite lignite lignite-new Lignite coal + other fossil (incl. biofuels) coal + other fossil (incl. biofuels) coal&lignite&oil lignite lignite
Lignite CCS Lignite CCS Coal + Other fossil Lignite lignite lignite-ccs Lignite (CCS) coal + other fossil (incl. biofuels) coal + other fossil (incl. biofuels) coal&lignite&oil lignite-ccs
Light oil Light oil Light oil Oil Light oil oil-light oil-light Oil (Light) oil (incl. biofuels) oil (incl. biofuels) coal&lignite&oil oil-light oil PyPSA-Eur does not explicitly differentiate light, heavy and shale oil
Heavy oil old 1 Heavy oil old 1 Oil Heavy oil oil-heavy oil-heavy-old1 Oil (Heavy) oil (incl. biofuels) oil (incl. biofuels) coal&lignite&oil oil-heavy oil PyPSA-Eur does not explicitly differentiate light, heavy and shale oil
Heavy oil old 2 Heavy oil old 2 Oil Heavy oil oil-heavy oil-heavy-old2 Oil (Heavy) oil (incl. biofuels) oil (incl. biofuels) coal&lignite&oil oil-heavy oil PyPSA-Eur does not explicitly differentiate light, heavy and shale oil
Oil shale old Oil shale old Oil Oil shale oil-shale oil-shale-old Oil (Shale) oil (incl. biofuels) oil (incl. biofuels) coal&lignite&oil oil-shale oil PyPSA-Eur does not explicitly differentiate light, heavy and shale oil
Oil shale new Oil shale new Oil Oil shale oil-shale oil-shale-new Oil (Shale) oil (incl. biofuels) oil (incl. biofuels) coal&lignite&oil oil-shale oil PyPSA-Eur does not explicitly differentiate light, heavy and shale oil
Other Non-RES Gas conventional old 1 Others non-renewable CHP and Small Thermal Others non-RES gas other-non-res-gas-conv-old1 Other Non-Renewables other non-res other non-res other non-res power generation other-non-res-gas-conv OCGT Methane demand benchmark
Other Non-RES Gas conventional old 2 Others non-renewable CHP and Small Thermal Others non-RES gas other-non-res-gas-conv-old2 Other Non-Renewables other non-res other non-res other non-res power generation other-non-res-gas-conv OCGT Methane demand benchmark
Other Non-RES Gas ccgt old 1 Others non-renewable CHP and Small Thermal Others non-RES gas other-non-res-gas-ccgt-old1 Other Non-Renewables other non-res other non-res other non-res power generation other-non-res-gas-ccgt CCGT Methane demand benchmark
Other Non-RES Gas ccgt old 2 Others non-renewable CHP and Small Thermal Others non-RES gas other-non-res-gas-ccgt-old2 Other Non-Renewables other non-res other non-res other non-res power generation other-non-res-gas-ccgt CCGT Methane demand benchmark
Other Non-RES Gas ccgt new Others non-renewable CHP and Small Thermal Others non-RES gas other-non-res-gas-ccgt-new Other Non-Renewables other non-res other non-res other non-res power generation other-non-res-gas-ccgt CCGT Methane demand benchmark
Other Non-RES Gas ccgt ccs Others non-renewable CHP and Small Thermal Others non-RES gas other-non-res-gas-ccgt-ccs Other Non-Renewables other non-res other non-res other non-res power generation other-non-res-gas-ccgt-ccs Methane demand benchmark
Other Non-RES Gas ocgt old Others non-renewable CHP and Small Thermal Others non-RES gas other-non-res-gas-ocgt-old Other Non-Renewables other non-res other non-res other non-res power generation other-non-res-gas-ocgt OCGT Methane demand benchmark
Other Non-RES Gas ocgt new Others non-renewable CHP and Small Thermal Others non-RES gas other-non-res-gas-ocgt-new Other Non-Renewables other non-res other non-res other non-res power generation other-non-res-gas-ocgt OCGT Methane demand benchmark
Other Non-RES Gas ccgt present 1 Others non-renewable CHP and Small Thermal Others non-RES gas other-non-res-gas-ccgt-present1 Other Non-Renewables other non-res other non-res other non-res power generation other-non-res-gas-ccgt CCGT Methane demand benchmark
Other Non-RES Gas ccgt present 2 Others non-renewable CHP and Small Thermal Others non-RES gas other-non-res-gas-ccgt-present2 Other Non-Renewables other non-res other non-res other non-res power generation other-non-res-gas-ccgt CCGT Methane demand benchmark
Other Non-RES Hard coal old 1 Others non-renewable CHP and Small Thermal Others non-RES coal other-non-res-coal-old1 Other Non-Renewables other non-res other non-res other non-res other-non-res-coal coal
Other Non-RES Hard coal old 2 Others non-renewable CHP and Small Thermal Others non-RES coal other-non-res-coal-old2 Other Non-Renewables other non-res other non-res other non-res other-non-res-coal coal
Other Non-RES Hard coal new Others non-renewable CHP and Small Thermal Others non-RES coal other-non-res-coal-new Other Non-Renewables other non-res other non-res other non-res other-non-res-coal coal
Other Non-RES Hard coal ccs Others non-renewable CHP and Small Thermal Others non-RES coal other-non-res-coal-ccs Other Non-Renewables other non-res other non-res other non-res other-non-res-coal-ccs
Other Non-RES Lignite old 1 Others non-renewable CHP and Small Thermal Others non-RES lignite other-non-res-lignite-old1 Other Non-Renewables other non-res other non-res other non-res other-non-res-lignite lignite
Other Non-RES Lignite old 2 Others non-renewable CHP and Small Thermal Others non-RES lignite other-non-res-lignite-old2 Other Non-Renewables other non-res other non-res other non-res other-non-res-lignite lignite
Other Non-RES Lignite new Others non-renewable CHP and Small Thermal Others non-RES lignite other-non-res-lignite-new Other Non-Renewables other non-res other non-res other non-res other-non-res-lignite lignite
Other Non-RES Lignite ccs Others non-renewable CHP and Small Thermal Others non-RES lignite other-non-res-lignite-ccs Other Non-Renewables other non-res other non-res other non-res other-non-res-lignite-ccs
Other Non-RES Light oil light oil Others non-renewable CHP and Small Thermal Others non-RES oil-light other-non-res-oil-light Other Non-Renewables other non-res other non-res other non-res other-non-res-oil-light oil PyPSA-Eur does not explicitly differentiate light, heavy and shale oil
Other Non-RES Heavy oil old 1 Others non-renewable CHP and Small Thermal Others non-RES oil-heavy other-non-res-oil-heavy-old1 Other Non-Renewables other non-res other non-res other non-res other-non-res-oil-heavy oil PyPSA-Eur does not explicitly differentiate light, heavy and shale oil
Other Non-RES Heavy oil old 2 Others non-renewable CHP and Small Thermal Others non-RES oil-heavy other-non-res-oil-heavy-old2 Other Non-Renewables other non-res other non-res other non-res other-non-res-oil-heavy oil PyPSA-Eur does not explicitly differentiate light, heavy and shale oil
Other Non-RES Oil shale old Others non-renewable CHP and Small Thermal Others non-RES oil-shale other-non-res-oil-shale-old Other Non-Renewables other non-res other non-res other non-res other-non-res-oil-shale oil PyPSA-Eur does not explicitly differentiate light, heavy and shale oil
Other Non-RES Oil shale new Others non-renewable CHP and Small Thermal Others non-RES oil-shale other-non-res-oil-shale-new Other Non-Renewables other non-res other non-res other non-res other-non-res-oil-shale oil PyPSA-Eur does not explicitly differentiate light, heavy and shale oil
Other non-res other non-res The TYNDP 2024 Scenarios Report maps Other Non-RES capacities as 'CHP and Small Thermal' and Other Non-RES generation as 'Other non-res'
Hydrogen Fuel Cell Hydrogen Fuel Cell Hydrogen Hydrogen FC h2-fuel-cell h2-fuel-cell H2 Fuel Cell hydrogen hydrogen methane&hydrogen power generation h2-fuel-cell Hydrogen demand benchmark
Hydrogen CCGT Hydrogen CCGT Hydrogen Hydrogen CCGT h2-ccgt h2-ccgt H2 CCGT hydrogen hydrogen methane&hydrogen power generation h2-ccgt Hydrogen demand benchmark
Electrolyser Onshore grid connected Electrolyzers Electrolyser (gen.) P2G H2 Electrolysis H2 Electrolysis H2 Electrolysis p2g electrolyser-onshore-grid Hydrogen supply benchmark
Battery Discharge Battery Storage discharge (gen.) Battery Large scale batteries battery battery discharger Battery Storage battery battery battery discharger
Battery Charge Battery Storage charge (load) battery battery charger Battery Storage battery charge (load) battery battery charger
Battery Store ElectrochemicalStorage battery battery Battery Storage battery
CompressedAirEnergyStorage caes caes Compressed Air Energy Storage caes Only used for CBA storage projects (no SB source; not part of the TYNDP scenarios)
battery-prosumer discharger Battery Storage Prosumer
battery-prosumer charger Battery Storage Prosumer
Batteries Prosumer battery battery-prosumer Battery Storage Prosumer battery-prosumer
battery-utility discharger Battery Storage Utility
battery-utility charger Battery Storage Utility
Batteries Utility battery battery-utility Battery Storage Utility battery-utility
Lignite biofuel Biofuels Lignite biofuel coal + other fossil (incl. biofuels) coal + other fossil (incl. biofuels)
Hard Coal biofuel Biofuels Hard coal biofuel coal + other fossil (incl. biofuels) coal + other fossil (incl. biofuels)
Gas biofuel methane (incl. biofuels) methane (incl. biofuels)
Light oil biofuel oil (incl. biofuels) oil (incl. biofuels)
Heavy oil biofuel oil (incl. biofuels) oil (incl. biofuels)
Oil shale biofuel Oil shale biofuel oil (incl. biofuels) oil (incl. biofuels)
Demand Side Response Explicit Demand shedding Demand Side Response Explicit dsr DSR dsr dsr dsr 'Demand Side Response Implicit' is not included in the benchmarking for DSR.
DSR dsr DSR dsr dsr
Unserved energy [GWh] load Demand Shedding demand shedding demand shedding demand shedding loss of load expectation Load/Demand shedding, Demand shedding hours benchmark
Dump energy [GWh] dumped energy dumped energy Curtailment/dump energy
Native Demand (excl. H2 storage charge) [GWhH2] H2 exogenous demand H2 Exogenous Demand exogenous demand Hydrogen demand benchmark
Marginal Cost Yearly Average [€] AC Prices benchmark
Marginal Cost Yearly Average (excl. 3 000 €/MWh) [€] AC Prices benchmark
Marginal Cost Yearly Average [€/MWhH2] H2 Prices benchmark
Marginal Cost Yearly Average (excl. 3 000 €/MWhH2) [€/MWhH2] H2 Prices benchmark
Loss of load expectation [hour] loss of load expectation Electricity demand shedding hours benchmark
Loss of H2 load expectation [hour] loss of load expectation Hydrogen demand shedding hours benchmark
Electricity AC Electricity electricity Final energy demand benchmark
Electricity AC_OH Electricity Offshore Hub electricity Final energy demand benchmark
Electricity low voltage Electricity Low Voltage electricity Final energy demand benchmark
Hydrogen H2 Hydrogen hydrogen Final energy demand benchmark
Methane exogenous gas demand Gas Exogenous Demand methane Final energy demand benchmark
Biofuels solid biomass Solid Biomass biofuels Final energy demand benchmark
Solids coal Coal solids Final energy demand benchmark
Liquids oil Oil liquids Final energy demand benchmark
oil refining Oil Refining
Liquids kerosene for aviation Kerosene for Aviation liquids Final energy demand benchmark
Liquids shipping oil Shipping Oil Demand liquids Final energy demand benchmark
Liquids land transport oil Land Transport Oil Demand liquids Final energy demand benchmark
Native Demand (excl. Pump load & Battery charge) [GWh] Final demand (inc. T&D losses, excl. pump storage ) electricity Electricity Exogenous Demand final demand (inc. t&d losses, excl. pump storage ) Electricity demand benchmark
H2 Residential&Tertiary H2 exogenous demand H2 Exogenous Demand exogenous demand Hydrogen demand benchmark
H2 Transport H2 exogenous demand H2 Exogenous Demand exogenous demand Hydrogen demand benchmark
H2 Industry H2 exogenous demand H2 Exogenous Demand exogenous demand Hydrogen demand benchmark
H2 Agriculture H2 exogenous demand H2 Exogenous Demand exogenous demand Hydrogen demand benchmark
H2 Power Generation power generation Hydrogen demand benchmark
H2 Non-energy use H2 exogenous demand H2 Exogenous Demand exogenous demand Hydrogen demand benchmark
H2 Others H2 exogenous demand H2 Exogenous Demand exogenous demand Hydrogen demand benchmark
H2 e-fuels Fischer-Tropsch Fischer-Tropsch e-fuels Hydrogen demand benchmark
H2 e-fuels Sabatier Sabatier e-fuels Hydrogen demand benchmark
H2 e-fuels methanolisation Methanolisation e-fuels Hydrogen demand benchmark
H2 International maritime bunkers H2 exogenous demand H2 Exogenous Demand exogenous demand Hydrogen demand benchmark
CH4 Residential&Tertiary exogenous gas demand Gas Exogenous Demand exogenous demand Methane demand benchmark
CH4 Transport exogenous gas demand Gas Exogenous Demand exogenous demand Methane demand benchmark
CH4 Industry exogenous gas demand Gas Exogenous Demand exogenous demand Methane demand benchmark
CH4 Agriculture exogenous gas demand Gas Exogenous Demand exogenous demand Methane demand benchmark
CH4 Power Generation power generation Methane demand benchmark
CH4 Non-energy use exogenous gas demand Gas Exogenous Demand exogenous demand Methane demand benchmark
CH4 Others exogenous gas demand Gas Exogenous Demand exogenous demand Methane demand benchmark
CH4 International maritime bunkers exogenous gas demand Gas Exogenous Demand exogenous demand Methane demand benchmark
CH4 SMR smr (grey) and smr with ccs (blue) Methane demand benchmark
Steam methane reformer SMR (Grey) SMR SMR (Grey Hydrogen) smr (grey) and smr with ccs (blue) smr (grey) and smr with ccs (blue) Hydrogen supply benchmark, Methane demand benchmark, TYNDP market model doesn't distinguish between grey and blue SMR in the output files
Steam methane reformer SMR with CCS (Blue) SMR CC SMR (CCS) (Blue Hydrogen) smr (grey) and smr with ccs (blue) smr (grey) and smr with ccs (blue) Hydrogen supply benchmark, Methane demand benchmark, TYNDP market model doesn't distinguish between grey and blue SMR in the output files
Undefined for generation undefined for generation Hydrogen supply benchmark
Low carbon imports H2 import Pipeline H2 Import Pipeline imports (renewable & low carbon) Hydrogen supply benchmark
Renewable imports H2 import Pipeline H2 Import Pipeline imports (renewable & low carbon) Hydrogen supply benchmark
Ammonia imports H2 import LH2 H2 Import LH2 ammonia imports Hydrogen supply benchmark
Biomethane (Domestic) biogas to gas Biogas to Gas biomethane (domestic) Methane supply benchmark
Biomethane (Import) biomethane (import) Methane supply benchmark
Natural gas (Domestic) natural gas (domestic) Methane supply benchmark
Natural gas (Import) gas Gas Primary natural gas (import) Methane supply benchmark
Synthetic methane (Domestic) Sabatier Sabatier synthetic methane (domestic) Methane supply benchmark
Synthetic methane (Import) synthetic methane (import) Methane supply benchmark
Final demand biomass final energy demand Biomass Final Energy Demand final demand Biomass supply benchmark
For electricity for electricity Biomass supply benchmark
For biomethane biogas Biogas for biomethane Biomass supply benchmark
For biofuels biomass to liquid Biomass to Liquid for biofuels Biomass supply benchmark
For biofuels biomass to liquid CC Biomass to Liquid (CCS) for biofuels Biomass supply benchmark
Methane gas Gas Primary methane Energy imports benchmark
Hydrogen H2 import LH2 H2 Import LH2 hydrogen Energy imports benchmark
Hydrogen H2 import Pipeline H2 Import Pipeline hydrogen Energy imports benchmark
Liquids oil primary Oil Primary liquids Energy imports benchmark
Solids coal Coal Primary solids Energy imports benchmark
Solids lignite Lignite Primary solids Energy imports benchmark
Biomass biomass Energy imports benchmark
electricity distribution grid Electricity Distribution Grid
DC Electricity Transmission
DC_OH Offshore Hub Transmission
H2 cavern-storage charger H2 Cavern Storage Bicharger
H2 cavern-storage discharger H2 Cavern Storage Bicharger
H2 cavern-storage H2 Cavern Storage
H2 Hydrogen
H2_OH Hydrogen Offshore Hub

Cost-Benefit Analysis Benchmarking

CBA indicators computed by Open-TYNDP are benchmarked against the official TYNDP 2024 CBA results for all calculated indicators (B1-B4) and all transmission projects. The comparison covers both individual climate year runs and multi-climate year averages.

Metrics

The following CBA indicators are benchmarked:

  • B1: Social Economic Welfare (SEW): Quantifies the change in operational system costs (socio-economic welfare).
  • B2: Social costs of CO2 emissions: Calculates the impact using societal cost assumptions (low, central, high).
  • B3: RES integration costs: Tracks changes in renewable capacity, generation, and avoided curtailment.
  • B4: Non-direct greenhouse emissions: Quantifies pollutants (NOx, SO2, PM, etc.) using fuel consumption multipliers.

Per Climate Year

Indicators are first benchmarked for each individual climate year run (e.g. CY2009, CY2008, CY1995). This allows deviations to be traced to specific weather conditions and isolates the effect of inter-annual climate variability on project benefits. For each climate year, all four indicators (B1–B4) are compared against the TYNDP 2024 reference values for every transmission project.

Averaged Indicators

Following the TYNDP 2024 methodology, indicators are also compared after averaging across the three climate years. A weighted average is calculated using weights for each climate year provided by the TYNDP 2024 CBA Implementation Guidelines: 0.233 for 1995, 0.367 for 2008 and 0.4 for 2009.

This averaged comparison is the primary benchmarking target, as it reflects the same aggregation approach used in the official TYNDP 2024 CBA. Deviations in the averaged indicators provide the most direct measure of how closely Open-TYNDP reproduces the official CBA outcomes at the project level.

Workflow

The CBA benchmarking workflow runs automatically after the CBA solve steps and produces per-indicator, per-project comparison figures and summary tables for both the individual climate year and averaged results. Output files follow the same structure as the SB benchmarking and are stored under results/[run]/cba/.