cfdm.Field.nc_set_hdf5_chunksizes

Field.nc_set_hdf5_chunksizes(chunksizes, ignore=False, constructs=False, **filter_kwargs)[source]

Set the HDF5 chunking strategy.

See also

nc_hdf5_chunksizes, nc_clear_hdf5_chunksizes, cfdm.read, cfdm.write

New in version (cfdm): 1.11.2.0

Parameters
chunksizes: None or str or int or float or dict or a sequence

Set the chunking strategy for writing to a netCDF4 file. One of:

  • None: No HDF5 chunking strategy has been defined. The chunking strategy will be determined at write time by cfdm.write.

  • 'contiguous': The data will be written to the file contiguously, i.e. no chunking.

  • int or float or str: The size in bytes of the HDF5 chunks. A floating point value is rounded down to the nearest integer, and a string represents a quantity of byte units. “Square-like” chunk shapes are preferred, maximising the amount of chunks that are completely filled with data values (see the cfdm.write hdf5_chunks parameter for details). For instance a chunksize of 1024 bytes may be specified with any of 1024, 1024.9, '1024', '1024.9', '1024 B', '1 KiB', '0.0009765625 MiB', etc. Recognised byte units are (case insensitive): B, KiB, MiB, GiB, TiB, PiB, KB, MB, GB, TB, and PB. Spaces in strings are optional.

  • sequence of int or None: The maximum number of array elements in a chunk along each data axis, provided in the same order as the data axes. Values are automatically limited to the full size of their corresponding data axis, but the special values None or -1 may be used to indicate the full axis size. This chunking strategy may get automatically modified by methods that change the data shape (such as insert_dimension).

  • dict: The maximum number of array elements in a chunk along the axes specified by the dictionary keys. Integer values are automatically limited to the full size of their corresponding data axis, and the special values None or -1 may be used to indicate the full axis size. The chunk size for an unspecified axis defaults to an existing chunk size for that axis, if there is one, or else the axis size. This chunking strategy may get automatically modified by methods that change the data shape (such as insert_dimension). Each dictionary key (k) specifies the unique axis that would be identified by f.domain_axis(k, **filter_kwargs), and it is allowed to specify a domain axis that is not spanned by the data array. See domain_axis for details.

constructs: dict or bool, optional

Also apply the HDF5 chunking strategy of the field construct data to the applicable axes of selected metadata constructs. The chunking strategies of unselected metadata constructs are unchanged.

If constructs is a dict then the selected metadata constructs are those that would be returned by f.constructs.filter(**constructs, filter_by_data=True). Note that an empty dictionary will therefore select all metadata constructs that have data. See filter for details.

For constructs being anything other than a dictionary, if it evaluates to True then all metadata constructs that have data are selected, and if it evaluates to False (the default) then no metadata constructs selected.

ignore: bool, optional

If True and chunksizes is a dict then ignore any dictionary keys that do not identify a unique axis of the field construct’s data. If False, the default, then an exception will be raised when such keys are encountered.

filter_kwargs: optional

When chunksizes is a dict, provide additional keyword arguments to domain_axis to customise axis selection criteria.

Returns

None

Examples

>>>
>>> f = cfdm.example_field(0)
>>> print(f)
Field: specific_humidity (ncvar%q)
----------------------------------
Data            : specific_humidity(latitude(5), longitude(8)) 1
Cell methods    : area: mean
Dimension coords: latitude(5) = [-75.0, ..., 75.0] degrees_north
                : longitude(8) = [22.5, ..., 337.5] degrees_east
                : time(1) = [2019-01-01 00:00:00]
>>> f.shape
(5, 8)
>>> print(f.nc_hdf5_chunksizes())
None
>>> f.nc_set_hdf5_chunksizes({'latitude': 1})
>>> f.nc_hdf5_chunksizes()
(1, 8)
>>> f.nc_set_hdf5_chunksizes({'longitude': 7})
>>> f.nc_hdf5_chunksizes()
(1, 7)
>>> f.nc_set_hdf5_chunksizes({'latitude': 4, 'longitude': 2})
>>> f.nc_hdf5_chunksizes()
(4, 2)
>>> f.nc_set_hdf5_chunksizes([1, 7])
>>> f.nc_hdf5_chunksizes()
(1, 7)
>>> f.nc_set_hdf5_chunksizes(64)
>>> f.nc_hdf5_chunksizes()
64
>>> f.nc_set_hdf5_chunksizes('128 B')
>>> f.nc_hdf5_chunksizes()
128
>>> f.nc_set_hdf5_chunksizes('contiguous')
>>> f.nc_hdf5_chunksizes()
'contiguous'
>>> f.nc_set_hdf5_chunksizes(None)
>>> print(f.nc_hdf5_chunksizes())
None
>>>
>>> f.nc_set_hdf5_chunksizes([-1, None])
>>> f.nc_hdf5_chunksizes()
(5, 8)
>>> f.nc_set_hdf5_chunksizes({'latitude': 999})
>>> f.nc_hdf5_chunksizes()
(5, 8)
>>>
>>> f.nc_set_hdf5_chunksizes({'latitude': 4, 'time': 1})
>>> f.nc_hdf5_chunksizes()
(4, 8)
>>> print(f.dimension_coordinate('time').nc_hdf5_chunksizes())
None
>>> print(f.dimension_coordinate('latitude').nc_hdf5_chunksizes())
None
>>> print(f.dimension_coordinate('longitude').nc_hdf5_chunksizes())
None
>>>
>>> f.nc_set_hdf5_chunksizes({'latitude': 4, 'time': 1}, constructs=True)
>>> f.dimension_coordinate('time').nc_hdf5_chunksizes()
(1,)
>>> f.dimension_coordinate('latitude').nc_hdf5_chunksizes()
(4,)
>>> f.dimension_coordinate('longitude').nc_hdf5_chunksizes()
(8,)
>>> f.nc_set_hdf5_chunksizes('contiguous', constructs={'filter_by_axis': ('longitude',)})
>>> f.nc_hdf5_chunksizes()
'contiguous'
 >>> f.dimension_coordinate('time').nc_hdf5_chunksizes()
(1,)
>>> f.dimension_coordinate('latitude').nc_hdf5_chunksizes()
(4,)
>>> f.dimension_coordinate('longitude').nc_hdf5_chunksizes()
'contiguous'
>>>
>>> f.nc_set_hdf5_chunksizes({'height': 19, 'latitude': 3})
Traceback
    ...
ValueError: Can't find unique 'height' axis. Consider setting ignore=True
>>> f.nc_set_hdf5_chunksizes({'height': 19, 'latitude': 3}, ignore=True)
>>> f.nc_hdf5_chunksizes(todict=True)
{'time': 1, 'latitude': 3, 'longitude': 8}