Extensions

Version 1.8.8.0 for version 1.8 of the CF conventions.

The cfdm package has been designed to be subclassed, so that the creation of a new implementation of the CF data model, based on cfdm, is straight forward. For example:

Create a new implementation with a new field construct class.
>>> import cfdm
>>> class my_Field(cfdm.Field):
...     def info(self):
...         return 'I am a {!r} instance'.format(
...             self.__class__.__name__)

The interpretation of CF-netCDF files that is encoded within the cfdm.read and cfdm.write functions is also inheritable, so that an extended data model implementation need not recreate the complicated mapping of CF data model constructs to, and from, CF-netCDF elements. This is made possible by the bridge design pattern, that decouples the implementation of the CF data model from the CF-netCDF encoding so that the two can vary independently.

Define an implementation that is the same as cfdm, but which uses the my_Field class to represent field constructs
>>> my_implementation = cfdm.implementation()
>>> my_implementation.set_class('Field', my_Field)
Define new functions that can read into my_Field instances, and write write my_Field instances to datasets on disk.
>>> import functools
>>> my_read = functools.partial(cfdm.read,
...                             _implementation=my_implementation)
>>> my_write = functools.partial(cfdm.write,
...                              _implementation=my_implementation)
Read my_field constructs from ‘file.nc’, the netCDF file used in the tutorial, using the new my_read function.
>>> q, t = my_read('file.nc')
Inspect a field construct read from the dataset, demonstrating that it is a my_Field instance from the new implementation that has the inherited functionality of a cfdm.Field instance.
>>> print(type(q))
<class '__main__.my_Field'>
>>> print(q.info())
I am a 'my_Field' instance
>>> print(repr(q))
<my_Field: specific_humidity(latitude(5), longitude(8)) 1>
>>> print(q.data.array)
[[0.007 0.034 0.003 0.014 0.018 0.037 0.024 0.029]
 [0.023 0.036 0.045 0.062 0.046 0.073 0.006 0.066]
 [0.11  0.131 0.124 0.146 0.087 0.103 0.057 0.011]
 [0.029 0.059 0.039 0.07  0.058 0.072 0.009 0.017]
 [0.006 0.036 0.019 0.035 0.018 0.037 0.034 0.013]]
Write the my_field constructs to a netCDF file using the new my_write function.
>>> my_write([q, t], 'new_file.nc')

Note that, so far, we have only replaced the field construct class in the new implementation, and not any of the metadata constructs or other component classes:

Demonstrate that the metadata construct classes of within the my_Field instance are still cfdm classes.
>>> print(type(q))
<class '__main__.my_Field'>
>>> print(type(q.construct('latitude')))
<class 'cfdm.dimensioncoordinate.DimensionCoordinate'>

If the API of the new implementation is changed such that a given cfdm functionality has a different API in the new implementation, then the new read-from-disk and write-to-disk functions defined above can still be used provided that the new implementation is created from a subclass of cfdm.CFDMImplementation, with the new API being applied in overridden methods.

Create an implementation with a different API.
>>> class my_Field_2(cfdm.Field):
...    def my_coordinates(self):
...        '''Get coordinate constructs with a different API
...        '''
...        c = self.coordinates
...        if not c:
...            return {}
...        return c
...
>>> class my_CFDMImplementation(cfdm.CFDMImplementation):
...    def get_coordinates(self, field):
...        '''Get coordinate constructs from a my_Field_2 instance,
...    using its different API.
...        '''
...        return field.my_coordinates()
...
>>> my_implementation_2 = my_CFDMImplementation(
...    cf_version=cfdm.CF(),
...
...    Field=my_Field_2,
...
...    AuxiliaryCoordinate=cfdm.AuxiliaryCoordinate,
...    CellMeasure=cfdm.CellMeasure,
...    CellMethod=cfdm.CellMethod,
...    CoordinateReference=cfdm.CoordinateReference,
...    DimensionCoordinate=cfdm.DimensionCoordinate,
...    DomainAncillary=cfdm.DomainAncillary,
...    DomainAxis=cfdm.DomainAxis,
...    FieldAncillary=cfdm.FieldAncillary,
...
...    Bounds=cfdm.Bounds,
...    InteriorRing=cfdm.InteriorRing,
...    CoordinateConversion=cfdm.CoordinateConversion,
...    Datum=cfdm.Datum,
...
...    List=cfdm.List,
...    Index=cfdm.Index,
...    Count=cfdm.Count,
...    NodeCountProperties=cfdm.NodeCountProperties,
...    PartNodeCountProperties=cfdm.PartNodeCountProperties,
...
...    Data=cfdm.Data,
...    GatheredArray=cfdm.GatheredArray,
...    NetCDFArray=cfdm.NetCDFArray,
...    RaggedContiguousArray=cfdm.RaggedContiguousArray,
...    RaggedIndexedArray=cfdm.RaggedIndexedArray,
...    RaggedIndexedContiguousArray=cfdm.RaggedIndexedContiguousArray,
... )

As all classes are required for the initialization of the new implementation class, this demonstrates explicitly that, in the absence of subclasses of the other classes, the cfdm classes may be used.

Read the file into ‘my_Field_2’ instances.
>>> my_read_2 = functools.partial(cfdm.read,
...                               _implementation=my_implementation2)
>>> q, t = my_read_2('file.nc')
>>> print(repr(q))
<my_Field_2: specific_humidity(latitude(5), longitude(8)) 1>

Finally, the mapping of CF data model constructs from CF-netCDF elements, and vice versa, may be modified where desired, leaving all other aspects it unchanged

Modify the mapping of netCDF elements to CF data model instances.
>>> class my_NetCDFRead(cfdm.read_write.netcdf.NetCDFRead):
...     def read(self, filename):
...         '''Read my fields from a netCDF file on disk or from
...     an OPeNDAP server location, using my modified mapping
...     from netCDF to the CF data model.
...         '''
...         print("Reading dataset using my modified mapping")
...         return super().read(filename)
Create a new read-from-disk function that uses the modified mapping.
>>> my_netcdf = my_NetCDFRead(my_implementation_2)
>>> def my_read_3(filename, ):
...     '''Read my field constructs from a dataset.
...     '''
...     return my_netcdf.read(filename)
Read the file from disk into ‘my_Field_2’ instances, demonstrating that the modified mapping is being used.
>>> q, t = my_read_3('~/cfdm/docs/source/sample_files/file.nc')
Reading dataset using my modified mapping
>>> print(repr(q))
<my_Field_2: specific_humidity(latitude(5), longitude(8)) 1>

In the same manner, cfdm.read_write.netcdf.NetCDFWrite may be subclassed, and a new write-to-disk function defined, to override aspects of the mapping from CF data model constructs to netCDF elements in a dataset.

Documentation

The cfdm package uses a “docstring rewriter” that allows commonly used parts of class method docstrings to be written once in a central location, and then inserted into each class at import time. In addition, parts of a docstring are modified to reflect the appropriate package and class names. This functionality extends to subclasses of cfdm classes. New docstring substitutions may also be defined for the subclasses.

See cfdm.core.meta.DocstringRewriteMeta for details on how to add to create new docstring substitutions for extensions, and how to modify the substitutions defined in the cfdm package.

A complete example

See cf-python for a complete example of extending the cfdm package in the manner described above.

cf-python adds more flexible inspection, reading and writing; and provides metadata-aware analytical processing capabilities such as regridding and statistical calculations.

It also has a more sophisticated data class that allows for larger-than-memory manipulations and parallel processing.

cf-python strictly extends the cfdm API, so that a cfdm command will always work on its cf-python counterpart.