Skip to content

Scenario Building (SB)

The Scenario Building (SB) workflow is implemented in rules/sb.smk. Rules are organised into stages: Retrieve, Development, Build electricity, Build sector, Postprocess, Benchmark and Explore. In addition, a Collect stage provides convenient target rules that run the workflow up to a specific stage, aggregating each stage's output across all configured wildcards.

Retrieve

Rule retrieve_tyndp_pecd

Downloads the PECD dataset.

Rule retrieve_tyndp_vp_data

Downloads the TYNDP Visualisation Platform data used by the benchmarking framework.

Rule retrieve_tyndp_nuclear_profiles

Downloads per-country nuclear availability profiles.

Rule retrieve_presolved_networks

Downloads pre-solved networks from a previous Open-TYNDP release (preliminary outcomes published on Zenodo) and extracts the solved network for each planning horizon. These can be investigated with PyPSA-Explorer's web interface using the launch_presolved_explorer rule without having to re-run the workflow.

Relevant Settings

data:
    open_tyndp_prelim:
        source:
        version:

Outputs

  • data/open_tyndp_prelim/{source}/{version}/base_s_all___{planning_horizons}.nc

Rule retrieve_countries_centroids

Downloads country centroid geometry data by Copyright (c) 2021 Gavin Rehkemper from https://cdn.jsdelivr.net/gh/gavinr/world-countries-centroids@v1.0.0/dist/countries.geojson.

Relevant Settings

None.

Outputs

  • data/countries_centroids.geojson

Development

Rule prepare_pecd_release

Loads and filters the available raw PECD data for the subset of required climate years as specified in the configuration.

Outputs

PECD prebuilt directory with filtered csv files including only relevant climate year data.

Build electricity

Rule clean_tyndp_electricity_demand

This script is used to clean TYNDP Scenario Building demand data to be used in the PyPSA-Eur workflow. The snapshot year is used as climatic year (cyear). For DE and GA, it must be one of the following years: 1995, 2008 or 2009. For NT, it must be between 1982 and 2019. If the snapshot is not one of these years, then the demand is set to 2009 electricity demand (2009 being considered as the most representative of the three years).

Depending on the scenario, different planning years (pyear) are available. DE and GA are defined for 2030, 2040 and 2050. NT scenario is only defined for 2030 and 2040. All the planning years are read at once.

Rule build_electricity_demand_tyndp

Extends the upstream build_electricity_demand rule with TYNDP-specific load data. Builds per-country load time series from the TYNDP electricity demand prepared by clean_tyndp_electricity_demand.

Rule clean_pecd_data

Loads and cleans the available PECD capacity factor generation time series based on PECD weather data. The script is executed for a given technology and planning horizon. Technologies can be one of:

  • LFSolarPVUtility,
  • LFSolarPVRooftop,
  • Wind_Offshore,
  • Wind_Onshore,
  • CSP_noStorage,
  • CSP_withStorage_7h_dispatched,
  • CSP_withStorage_7h_preDispatch (note: includes cf > 1 for when thermal storage can be used).
Outputs

Cleaned csv file with capacity factor generation time series and regions as columns.

Rule build_renewable_profiles_pecd

Create renewable profiles for each region from PECD and TYNDP datasets, building on PECD climate data. The available generation time series are read for each node across all renewable technologies, including onshore wind, AC-connected offshore wind, DC-connected offshore wind, and solar PV generators.

.. note:: Hydroelectric profiles will be built in script build_hydro_profiles_PECD. Not yet implemented.

Outputs

  • resources/profile_pecd_{clusters}_{technology}.nc with the following structure

    Field Dimensions Description
    profile year, bus, bin, time the per unit hourly availability factors for each bus

Rule build_pemmdb_data

Collects and bundles the available PEMMDB v2.5 capacities and profiles for different PEMMDB technologies from TYNDP data bundle for a given planning horizon.

Outputs

Cleaned CSV file with all NT capacities (p_nom) in long format and NetCDF file containing the must run obligations (p_min_pu) and availability (p_max_pu) for each of the different PEMMDB technologies.

  • resources/pemmdb_capacities_{planning_horizon}.csv in long format

  • resources/pemmdb_profiles_{planning_horizon}.nc with the following structure:

    Field Coordinates Description
    p_min_pu, p_max_pu time, bus, carrier, index_carrier, open_tyndp_type the per unit hourly availability and must-run obligations for each bus and PEMMDB technology

Rule build_tyndp_transmission_projects

Build TYNDP transmission projects from the Grid Investment Dataset.

This script processes the TYNDP 2024 Grid Investment Dataset to extract transmission projects for inclusion in both the electrical and hydrogen reference grid.

Projects are filtered by commissioning year and category, then formatted as network links with geometry attributes derived from bus locations.

Inputs
  • data/tyndp_2024_bundle/Investment Datasets/GRID.xlsx: Grid investment dataset containing electricity and hydrogen transmission projects with capacities and commissioning years.
  • resources/tyndp/build/geojson/buses.geojson: Electrical buses with geometry.
  • resources/tyndp/build/geojson/buses_h2.geojson: Hydrogen buses with geometry.
Outputs
  • resources/tyndp/new_links_{planning_horizons}.csv: Processed electricity transmission projects with bus connections, capacities (p_nom), lengths, and geometry attributes ready for network integration.
  • resources/tyndp/new_links_h2_{planning_horizons}.csv: Processed hydrogen transmission projects with bus connections and capacities (p_nom) attributes ready for network integration.

Rule build_tyndp_trajectories

Loads and cleans the TYNDP capacity trajectories for a given TYNDP scenario.

Outputs

Cleaned CSV file with all TYNDP trajectories (p_nom_min, p_nom_max) in long format.

  • resources/tyndp_trajectories.csv in long format.

Rule clean_tyndp_hydro_inflows

Loads and cleans the available hydro inflow data from TYNDP data bundle for a given

  • climate year,
  • planning horizon,
  • hydro technology.

Input data for TYNDP 2024 comes from PEMMDB v2.5.

Outputs

Cleaned csv file with hourly hydro inflow time series in MW per region.

Rule build_tyndp_hydro_profile

Builds hydroelectric inflow time-series for each country based on PEMMDB v2.5 hydro inflow data from the 2024 TYNDP.

Outputs

  • resources/profile_pemmdb_hydro.nc:

    Field Dimensions Description
    inflow bus, time, year, hydro_tech Inflow to the state of charge (in MW), e.g. due to river inflow in hydro reservoir.

Rule build_electricity_demand_base_tyndp

Extends the upstream build_electricity_demand_base rule with TYNDP-specific load data. Builds the electricity demand for base regions from the TYNDP electricity demand prepared by build_electricity_demand_tyndp.

Build sector

Rule build_tyndp_gas_demand

Builds TYNDP Scenario Building gas demand for Open-TYNDP.

This script processes methane (gas) demand data from TYNDP 2024 Supply Tool, extracting both final energy demand (direct gas consumption) and heat-related gas demand. The data is filtered and interpolated based on the selected scenario (Distributed Energy, Global Ambition, or National Trends) and planning horizon.

Data Availability by Scenario

The input data has different temporal coverage depending on scenario:

National Trends (NT): - Available for 2030 and 2040 - Includes final gas demand (incl. heat demand) from NT+ data collection - Heat demand processed with distribution shares and efficiency factors - Special handling for Italian demand

Distributed Energy (DE) and Global Ambition (GA): - Processing not yet implemented

Processing

Missing years are linearly interpolated between available data points.

For each planning year, the script: 1. Reads final energy demand (direct gas consumption by carrier) 2. Reads heat demand and converts it to primary energy using distribution and efficiency data 3. Combines both demand components 4. Aggregates by country/bus

Inputs
  • data/tyndp_2024_bundle/Supply Tool/20240518-Supply-Tool.xlsm: TYNDP 2024 Supply Tool Excel file containing:
  • NT+ data sheet: Final demand by country
  • Other data and Conversions sheet: Heat distribution and efficiency factors
  • IT sheet: Italian gas production data
Outputs
  • gas_demand_tyndp_{planning_horizons}.csv: Processed gas demand data (MWh) by country/bus for the specified planning horizon

Rule build_tyndp_h2_demand

Builds TYNDP Scenario Building hydrogen demand profiles for Open-TYNDP.

This script processes hydrogen demand data from TYNDP 2024, using the snapshots year as the climatic year (cyear) for demand profiles. The data is filtered and interpolated based on the selected scenario (Distributed Energy, Global Ambition, or National Trends) and planning horizon.

Climatic Year Selection

The snapshots year determines the climatic year for demand profiles:

  • DE and GA scenarios: Must use 1995, 2008, or 2009. If snapshots is not one of these years, 2009 is used as the default (considered most representative).
  • NT scenario: Must be between 1982 and 2019.
Data Availability by Scenario

The input data has different temporal and spatial coverage depending on scenario:

Distributed Energy (DE) and Global Ambition (GA): - Hydrogen zone Z2: Available for 2030, 2040, and 2050 - Hydrogen zone Z1: Available for 2050 (DE and GA) and 2040 (GA only)

National Trends (NT): - Available for 2030 and 2040 only - No split into hydrogen zones

Processing

Missing years are linearly interpolated between available data points.

Inputs
  • data/tyndp_2024_bundle/Demand Profiles: TYNDP 2024 hydrogen demand profiles
Outputs
  • resources/h2_demand_tyndp_{planning_horizons}.csv: Processed hydrogen demand time series for the specified planning horizon

Rule build_tyndp_h2_network

This script loads and cleans the TYNDP H2 reference grid and interzonal connections for a given wildcard planning horizon and TYNDP scenario as defined in the config file. The reference grid contains data for the TYNDP planning year 2030, while depending on the scenario, different planning years (pyear) are available for the interzonal connections. DE and GA are defined for 2030, 2035, 2040, 2045 and 2050. For the NT scenario no interzonal capacities are defined.

Rule clean_tyndp_h2_imports

This script is used to load and clean TYNDP H2 import data.

Rule build_tyndp_h2_imports

Filters TYNDP H2 import potentials, maximum capacity, offer quantity and marginal cost for pipeline and shipping for a specific TYNDP scenario and a given year wildcard. The function saves a csv file with TYNDP H2 import potentials and marginal cost filtered for a specific TYNDP scenario and for a given year.

Rule clean_tyndp_smr

This script cleans and extracts the TYNDP SMR data and saves it in a common pypsa friendly format.

Rule clean_tyndp_h2_storages

This script cleans and extracts the TYNDP H2 Storage data and saves it in a common pypsa friendly format.

Rule build_tyndp_offshore_hubs

This script is used to clean TYNDP Scenario Building offshore hubs data to be used in the PyPSA-Eur workflow. Depending on the scenario, different planning years (pyear) are available. DE and GA are defined for 2030, 2040 and 2050. NT scenario is only defined for 2030 and 2040. All the planning years are read at once.

Rule group_tyndp_conventionals

Groups TYNDP conventional carriers according to a pre-determined mapping.

This script takes PEMMDB capacities and profiles and groups the TYNDP conventional carriers according to a pre-determined mapping. For capacity and energy values, the sum of the grouped components is calculated. For the aggregated efficiency value, the mean is taken. This script will only be executed if activated in the configuration and will be executed once for each planning horizon.

Inputs
  • pemmdb_capacities_{planning_horizon}.csv: Processed PEMMDB capacities for the given planning_horizon.
  • pemmdb_profiles_{planning_horizon}.nc: Processed PEMMDB must-run and availability profiles for the given planning_horizon.
  • data/tyndp_technology_map.csv: TYNDP technology mapping used for the grouping.
Outputs
  • pemmdb_capacities_{planning_horizon}_grouped.csv: Grouped PEMMDB capacities for the given planning_horizon.
  • pemmdb_profiles_{planning_horizon}_grouped.nc: Grouped PEMMDB must-run and availability profiles for the given planning_horizon.

Postprocess

Rule plot_base_hydrogen_network

Creates map of base hydrogen network, added before the optimization.

Rule plot_base_offshore_network

Plot offshore transmission network with existing capacities. If expanded is enabled, the optimal capacities are plotted instead.

Rule plot_offshore_network

Plot offshore transmission network with existing capacities. If expanded is enabled, the optimal capacities are plotted instead.

Benchmark

Rule clean_tyndp_output_benchmark

This script cleans the TYNDP market model output data for benchmarking.

Reads TYNDP market model (MM) MMStandardOutputFile xlsx files from both: - "Yearly Outputs" sheet for power and electricity data - "Yearly H2 Outputs" sheet for hydrogen-specific data

Note: Currently, only NT scenario processing is supported.

Rule clean_tyndp_report_benchmark

This script cleans the raw TYNDP 2024 Scenarios Report Data that will be used for benchmarking.

It reads and processes all the tables defined in the configuration file. The correct indexes and headers are then assigned. The data structure is subsequently converted to a long format. Finally, the units are converted to standard units: MW for power units and MWh for energy units.

Rule clean_tyndp_vp_data

This script cleans the TYNDP 2024 Visualisation Platform data for benchmarking.

It reads and processes the data, applying correct indexes, headers, and naming conventions. The data structure is converted to long format, and units are standardized to MW for power and MWh for energy.

Rule build_statistics

This script computes the benchmark statistics from the optimised network.

Rule make_benchmark

This script computes accuracy indicators for comparing workflow results against reference data from TYNDP 2024.

Benchmarks are computed only for planning years available in the TYNDP 2024 Scenarios data: - NT (National Trends): 2030, 2040 - DE (Distributed Energy) and GA (Global Ambition): 2040, 2050

This module implements a methodology introduced by Wen et al. (2022) for evaluating performance of energy system models using multiple accuracy indicators.

Rule plot_benchmark

Plot benchmark figures.

Collect

Aggregate rules that run the corresponding base rule across all configured wildcards. They do not have dedicated scripts.

Rule clean_pecd_datas

Aggregate clean_pecd_data outputs.

Rule build_renewable_profiles_pecds

Aggregate build_renewable_profiles_pecd outputs.

Rule prepare_benchmarks

Aggregate benchmark inputs before the benchmarking stage.

Rule make_benchmarks

Aggregate make_benchmark outputs.

Rule plot_benchmarks

Aggregate plot_benchmark outputs.

Rule build_pemmdb_and_trajectories

Aggregate build_pemmdb_data and build_tyndp_trajectories outputs.

Rule build_tyndp_h2_demands

Aggregate build_tyndp_h2_demand outputs.

Rule build_tyndp_gas_demands

Aggregate build_tyndp_gas_demand outputs.

Explore

Rule launch_explorer

Launches the PyPSA-Explorer web interface to visualize workflow result networks. The explorer will automatically use the next available port starting from 8050.

To close all running explorer instances when finished, run: snakemake -c1 close_explorers --configfile config/config.tyndp.yaml

Rule launch_presolved_explorer

Mirrors the launch_explorer rule to launch the PyPSA-Explorer web interface with pre-solved SB networks from previous Open-TYNDP release runs (see retrieve_presolved_networks).

Rule close_explorers

Closes all open local instances of launched PyPSA-Explorers and frees up used ports again.