Release notes#
unpublished#
Breaking changes#
All fields of
Composition
are now private (!273)Input.source
is private,Input.get_source()
becomes propertyInput.source
,Input.set_source
becomes a getter (!273)Output.targets
is private,Output.get_targets()
becomes propertyOutput.targets
(!273)Composition metadata was restructured to hold components and adapters in separate sub-dictionaries (!274)
Time components implement method
_next_time
instead of propertynext_time
(!283)Info
now has properties forgrid
,time
andmask
(!286)all init-args of
Info
are now optional (!286)Info.accepts
has changed signature: renamedignore_none
toincoming_donwstream
(!286)Info.accepts
now only checks:grid
,mask
andunits
(other meta data can differ) (!286)Grid.to_/from_canonical
now allows additional dimensions (!286)data_shape
now a property ofGridBase
(!286)NoGrid
can be initialized withdim
ordata_shape
nowNoGrid.data_shape
can have-1
entries for variable size dimensionsif only
dim
given toNoGrid
, all entries indata_shape
will be-1
Features#
Components and adapters automatically provide default metadata that can be extended by implementations (!274, !276)
Grid class now have attributes providing connectivity information for the contained cells (!275)
cells_connectivity
: connectivity array as used by ESMF and VTKcells_definition
: cell definition as used by PyVista and legacy VTKcells_offset
: location of the start of each cell incells_connectivity
added convenience functions and constants to
grid_tools
(!275)get_cells_matrix
: convertcells_connectivity
orcells_definition
back to the default cells matrix used in the Grid class (can be used to convert VTK-grids into FINAM-grids)INV_VTK_TYPE_MAP
: inverse mapping toVTK_TYPE_MAP
- FINAM cell type to VTK cell typeVTK_CELL_DIM
: parametric dimension for each VTK cell type
Grid class now reusable when having different data locations and better grid type casting (!278)
added
copy
method to grids with optional argumentdeep
(False
by default) to create a copy of a gridadded setter for
data_location
in order to set a new data location (e.g. after copying a grid)added class attribute
valid_locations
in order to check the set data location (esri-grid only supports cells, unstructured-points only support points)added missing casting methods to convert esri to uniform and uniform to rectilinear (when you want to use point data on an esri-grid, you can cast it to uniform first)
added
axes_attributes
also to unstructured grids
Grid method
compatible_with
now has acheck_location
argument to optionally check data location (!280)added
Mask
enum with two options: (!286)Mask.FLEX
for flexible maskingMask.NONE
to explicitly use plain numpy arrays
added
mask
attribute and init-arg toInfo
: can be aMask
value or a valid mask fornumpy.ma.MaskedArray
(!286)data.tools.prepare
now applies masks to data if set inInfo
object (!286)ARegridding
now has aout_mask
arg (!286)RegridNearest
andRegridLinear
now support explicitly masked data (input doesn’t haveMask.FLEX
) (!286)adapters now have an
in_info
property (!286)
Bug fixes#
cells for structured grids in 3D are now created correctly (no negative Volume in VTK/ESMF) (!286)
Documentation#
Minor fixes in documentation examples and links (!272)
Adds a book section on composition, component and adapter metadata (!274)
v0.5.1#
Bug fixes#
Fix unquantified masked arrays loosing mask in
fm.data.prepare()
(#115, !270)
v0.5.0#
Features#
Components and adapters can provide a dictionary of meta data (!259)
Class
Composition
hat a propertymetadata
that collects and returns the meta data from all components and adapters (!259)Automatic conversion between compatible grids (!255)
Adds methods
to_canonical
,from_canonical
andget_transform_to
to grid classes (!255)Adds support for masked grids using
numpy.ma.MaskedArray
(!258, !260)Adds convenience functions for dealing with masked arrays in
data.tools
(!260):is_masked_array
,has_masked_values
,filled
,to_compressed
,from_compressed
,check_data_covers_domain
Documentation#
Adds a book chapter on wrapping existing models for FINAM (!256)
Adds a book section on masked data (!262)
Bug fixes#
No more logging of expected
FinamNoDataError
in inputs during the connect phase (!257)
Other#
FINAM is now available on Conda via conda-forge
v0.4.0#
New scheduling algorithm#
FINAM uses a new scheduling algorithm that allows components to use future data instead of only past/current (!157, !159)
New adapters to resolve circular coupling through the use of delayed data (!187)
It is now possible to set up static couplings that run only once and have no explicit time or stepping (!166)
FINAM can handle different starting times of components by pushing initial data twice (!206): Once for the common starting time, and once for the actual component time
Components are no longer required to push all outputs on every step (!208)
Data and metadata rework#
Outputs check compatibility between metadata of inputs if there is more than one target input (!104)
Add data tools function
compatible_units
to check for convertibility (!105)Components can exchange their starting time through the
Info
object (!111)Info exchange is automated by the
ConnectHelper
by specifying transfer rules at initialization (!154)Info
now requires time in constructor (can beNone
) (!111)Scheduler checks for dead links that don’t work in terms of push/pull combination (!112)
IInput
,IOutput
andIAdapter
have new internally used propertiesneeds_push
andneeds_pull
(!112)to_xarray
now checks the data shape if the data is not flat (!130)Outputs can be flagged
static
for data that is only used during initialization, or that is constant (!166)Inputs can be flagged
static
for constant data (!171)Outputs accept and convert compatible units, not only exactly equal units (!215)
Outputs check that subsequent data pushes don’t share memory (!217)
Exchanged
xarray
data has no time coordinate anymore, only a dimension without values (for performance and usability) (!223)Remove the
xarray
wrapping completely. Use numpy arrays in pintQuantity
(!235)Outputs and adapters can have a
memory_limit
and write data to disk if the limit is exceeded (!238)
Components#
Add
modules.WeightedSum
for aggregation of multiple inputs (!105)Add
modules.SimplexNoise
for generating spatio-temporal noise (!131)Add
modules.TimeTrigger
to forward data from pull-based to push-based components (!131)Add
modules.ScheduleLogger
to visualize scheduling/module updates through ASCII charts (!160)Add
modules.DebugPushConsumer
as a push-based variant of the debug consumer (!165)Add
modules.UserControl
that lets users control FINAM runs from the terminal (!184)modules.DebugConsumer
andmodules.DebugPushConsumer
can use optional callbacks for better debugging (!176)Components can be renamed using the method
with_name()
(!243)
Adapters#
Add
adapters.Histogram
to extract a histogram from grid values (!182)Add
adapters.DelayFixed
,adapters.DelayToPull
andadapters.DelayToPush
to resolve circular coupling through the use of delayed data (!187)Add
adapters.StepTime
for step-wise interpolation (!194)Restructuring of time integration adapters (!194)
adapters.IntegrateTime
renamed toadapters.AvgOverTime
Add
adapters.SumOverTime
for sum/Area under Curve integration
Adapters have a method
finalize()
for cleanup (!226).Adapters can be renamed using the method
with_name()
(!243)
Other#
Remove module
core
, subpackages now underfinam
(!106)Rename
IOutput.source_changed()
tosource_updated
(!107)Rename
LogError
toErrorLogger
(!107)Rename abstract SDK classes: (!107)
AAdapter
is nowAdapter
AComponent
is nowComponent
ATimeComponent
is nowTimeComponent
Changed arguments for
create_connector()
(!111)Removed
required_out_infos
Renamed
required_in_data
topull_data
Added arguments to specify info exchange rules
All error types are in module
errors
now, and re-exported at top level (!116)Overwriting
_validate()
and_finalize()
in components is now mandatory (!156)Input and output slots can be accessed from components directly, e.g.
comp["A"]
instead ofcomp.inputs["A"]
(!147)Inputs and outputs can be marked as
static
for constant data without time information (!166, !171)New helper function
tools.inspect()
to inspect components, adapters and I/O slots (!197)Publish on PyPI, starting with the next release (!198, !200, !201)
Added benchmarks for the most important FINAM functions (see the benchmarks README)
Added profiling for full runs to the CI (!221)
Optimization of data tool functions, with approx. 20-fold speedup of basic push+pull (!222, !223, !224, !228, !229, !237).
Add two more log levels:
TRACE
(most verbose) andPROFILE
(betweenDEBUG
andINFO
) (!240)
v0.4.0-rc.2#
Data and metadata rework#
Add conversion between CRS to regridding adapters, using
pyproj
(!95)Add more data tool functions:
quantify(xdata)
,check_axes_uniformity(axes)
andstrip_data(xdata)
(!96, !100)In outputs, the name of the data is overwritten instead of failing the check (!98)
Adapters can pass through data with time, even if it does not match the pull time (which is quite common) (!98)
Interface#
Add
CallbackOutput
for implementing pull-based components (!102)Connect phase of scheduler can be called separately from run (!99)
No need to set component status in constructor anymore (!100)
Other#
Components are allowed to be in state VALIDATED at the end of a run (i.e. not updated) (!97)
Component connector checks that inputs and outputs referenced in arguments actually exist (!101)
v0.4.0-rc.1#
Data and metadata rework#
Grid specifications for structured and unstructured grids (!74):
RectilinearGrid
,UniformGrid
,EsriGrid
,UnstructuredGrid
andUnstructuredPoints
Use of
xarray.DataArray
for all exchanged data (!74)All exchanged data must have
pint
units (can be “dimensionless”) (!74)Metadata about grid specification, units and other metadata is exchanged before the first data exchange (!77)
Metadata exchange is iterative and bi-directional (!77) Components can depend on metadata from source or target components
Inputs check compatibility of incoming metadata with own requirements (!77)
Inputs and outputs check compatibility of incoming data with metadata (!77)
Automatic conversion of array-like and scalars to
xarray.DataArray
in outputs, with metadata check (!74, !77)
Adapters#
New adapters for linear and nearest-neighbour regridding from and to all available grid types (!77, !87)
Removed adapter
GridCellCallback
(!79)
Components#
Removed
GridView
component (new implementation in finam-plot) (!79)
Usability#
The finam package has a flatter module hierarchy now, so the most important classes are now exported at the top level (!92)
Component developers do not overwrite interface methods like
update()
anymore, but internal methods like_update()
instead (!85)
Other#
More ergonomic input and output creation in components (!80, !82)
Input and output mappings are immutable after initialization (!82)
Brought up test coverage to 98% (!93)
v0.3.0#
Interface#
IOutput
now has a propertyhas_target
to determine whether it is connected (!54)IInput
now has a propertyhas_source
to determine whether it is connected (!63)CallbackComponent
for generic, callback-based data generation, transform or consumption (!55)IComponent.inputs
,IComponent.outputs
,IComponent.status
andITimeComponent.time
are now properties instead of methods (!53)Support for initialization from inputs / iterative connect (!69)
Changed logic and
ComponentStatus
to be set in component methodconnect()
Other#
Logging capability, incl. C-level output capturing (!64, !70, !71)
Grid
now usesMaskedArray
, to improve handling of missing dataVertical/Y grid indices are now flipped, to conform with (typical) NetCDF and ASCII grid format (!51)
Minimal Python version is now 3.7, to ensure consistent
dict
order (!58)Most
assert
s replaced by raising errors (!52)Grid visualization supports color scale limits (!61)
Context manager and helper function to execute code in a certain working directory (!62)
Status checks moved from component methods to composition/scheduler (!65)
Adapters check that requested time it in the range of available data (!66)
Bug fixes#
Fix check for
None
data inGrid
constructor (!50)
v0.2.0#
Changes#
Uses Python’s
datetime
andtimedelta
for all time-related parametersRemoved temporal sum integration adapter
v0.1.0#
initial release of FINAM