Model coupling scripts#

Coupling setups are created and executed using Python scripts.

Simple example#

Here is a simple example coupling two components:

# simple-coupling.py
import random
from datetime import datetime, timedelta

import finam as fm

# Instantiate components, e.g. models

# Here, we use simplex noise to get a value smoothly varying over time
generator = fm.modules.SimplexNoise(
    time_frequency=0.000001,
    info=fm.Info(time=None, grid=fm.NoGrid()),
)

# A debug printing component
consumer = fm.modules.DebugConsumer(
    inputs={"Value": fm.Info(time=None, grid=fm.NoGrid())},
    start=datetime(2000, 1, 1),
    step=timedelta(days=1),
    log_data="INFO",
)

# Create a `Composition` containing all components
composition = fm.Composition([generator, consumer])

# Initialize the `Composition`
composition.initialize()

# Couple inputs to outputs
generator.outputs["Noise"] >> consumer.inputs["Value"]

# Run the composition until June 2000
composition.run(end_time=datetime(2000, 6, 30))    # doctest: +ELLIPSIS

In the above example, we couple a simplex noise generator component (modules.SimplexNoise) with a consumer component for debug printing (modules.DebugConsumer).

Note:

with package finam installed, simply run the above scripts with:

$ python simple-coupling.py

The typical steps in a script are:

  1. Instantiate components and adapters (see next example)

  2. Create a Composition and initialize it

  3. Connect outputs to inputs using the overloaded >> operator (IOutput.__rshift__())

  4. Run the Composition

Inputs and outputs#

Inputs and outputs of a component can be retrieved via IComponent.inputs and IComponent.outputs properties. Both methods return a Python dict-like, with strings as keys and input or output objects as values, respectively.

An input can be connected to an output using either >> (as in the examples), or the output’s method IOutput.chain(). Both lines here are equivalent:

generator.outputs["Value"] >> plot.inputs["Value"]
generator.outputs["Value"].chain(consumer.inputs["Value"])

As a shortcut, slots can be accessed by the component’s [] operator directly (see Component.__getitem__()):

generator["Value"] >> plot["Value"]

Adapters#

In the above example, both coupled components match in terms of the exchanged data (numeric value) as well as their time step (1 day).

This is not necessarily the case for all coupling setups. To mediate between components, FINAM uses adapters. Those can be used to transform data (regridding, geographic projections, …) or for temporal interpolation or aggregation.

The following examples uses a similar setup like the previous one, but with differing time steps and an adapter:

# adapter-coupling.py
import random
from datetime import datetime, timedelta

import finam as fm

# Instantiate components, e.g. models

# Here, we use simplex noise to get a value smoothly varying over time
generator = fm.modules.SimplexNoise(
    time_frequency=0.000001,
    info=fm.Info(time=None, grid=fm.NoGrid()),
)
# A debug printing component
consumer_1 = fm.modules.DebugConsumer(
    inputs={"Value": fm.Info(time=None, grid=fm.NoGrid())},
    start=datetime(2000, 1, 1),
    step=timedelta(days=1),
    log_data="INFO",
)
# A second debug printing component with a different time step
consumer_2 = fm.modules.DebugConsumer(
    inputs={"Value": fm.Info(time=None, grid=fm.NoGrid())},
    start=datetime(2000, 1, 1),
    step=timedelta(days=2.324732),
    log_data="INFO",
)

# Create a `Composition` containing all components
composition = fm.Composition([generator, consumer_1, consumer_2])

# Initialize the `Composition`
composition.initialize()

# Couple inputs to outputs, without an adapter
(
    generator.outputs["Noise"]
    >> consumer_1.inputs["Value"]
)
# Couple inputs to outputs, with an adapters
(
    generator.outputs["Noise"]
    >> fm.adapters.Scale(scale=10.0)
    >> consumer_2.inputs["Value"]
)

# Run the composition until June 2000
composition.run(end_time=datetime(2000, 6, 30))    # doctest: +ELLIPSIS

Adapter chaining#

As can be seen from the example, components and adapters can be chained using the >> operator (or the IOutput.chain() method).

This is achieved by:

  1. An adapter is an input, and at the same time an output

  2. The chained input is returned by >> and IOutput.chain(). In case the chained input is an adapter (and thus also an output), it can be immediately reused in a further chaining operation

The syntax looks like this:

(
    generator.outputs["Noise"]
    >> AdapterA()
    >> AdapterB()
    >> consumer.inputs["Value"]
)

Or, in the short slot syntax:

(
    generator["Noise"]
    >> AdapterA()
    >> AdapterB()
    >> consumer["Value"]
)

Circular and bi-directional coupling#

FINAM allows for bi-directional and circular coupling.

For acyclic coupling, the FINAM scheduler updates upstream components first to allow downstream components to pull data for the end of their next time step. With circular dependencies, this would result in an infinite loop. The scheduler detects these cases and exits with a respective message.

To resolve circular dependencies, one of the models in the cycle must use data from the past (i.e. delayed). FINAM provides several adapters for this purpose:

The adapters are used on the inputs of the component that is intended to work with delayed data.

For all except adapters.DelayToPush, the adapters must be parametrized with a sensible delay. Some rules of thumb for choosing the delay:

  • For components where one time step is an integral multiple of other one, a delay equal to the larger step should be sufficient.

  • For components with no such time step ratio, the sum of the (two largest) time steps should be sufficient.

Logging#

FINAM provides a comprehensive logging framework built on Pythons standard logging package.

You can configure the base logger when creating the Composition as shown above:

import finam as fm
import logging

comp = fm.Composition(
    [],
    logger_name="FINAM",
    print_log=True,
    log_file=True,
    log_level=logging.INFO,
)

There you have several options:

  • logger_name: (str) Base name of the logger in the output ("FINAM" by default)

  • print_log: (bool) Whether logging should be shown in the terminal output

  • log_file: (None, bool, pathlike) Whether a log-file should be created - None or False: no log file will be written - True: a log file with the name {logger_name}_{time.strftime('%Y-%m-%d_%H-%M-%S')}.log will be created in the current working directory (e.g. FINAM_2022-09-26_12-58-15.log) - <pathlike>: log file will be created under the given path

  • log_level: (int) this will control the level of logging (logging.INFO by default) - only log messages with a level equal or higher than the given logging level will be shown - options are (from most to least verbose): logging.TRACE, logging.DEBUG, logging.PROFILE, logging.INFO, logging.WARNING, logging.ERROR, logging.CRITICAL or any positive integer number

A log file could look like this, when setting the logging level to logging.INFO:

2022-08-26 11:31:28,283 - FINAM - INFO - doing fine
2022-08-26 11:31:28,284 - FINAM - WARNING - Boo

or like this, when setting logging level to logging.DEBUG:

2022-08-26 11:31:28,283 - FINAM - INFO - doing fine
2022-08-26 11:31:28,284 - FINAM - WARNING - Boo
2022-08-26 11:31:28,285 - FINAM - DEBUG - Some debugging message