cfdm.AggregatedArray

---

class cfdm.AggregatedArray(filename=None, address=None, dtype=None, mask=True, unpack=True, fragment_array=None, attributes=None, filesystem=None, backend=None, backend_options=None, fragment_filesystem=None, fragment_backend=None, fragment_backend_options=None, source=None, copy=True)

Bases: FileArray

An array stored in a CF aggregation variable.

Added in version (cfdm): 1.12.0.0

Initialisation

Parameters:

filename: str, optional

The name of the aggregation file containing the aggregation variable.

address: str, optional

The name of the aggregation variable for the array.

dtype: numpy.dtype

The data type of the aggregated data array. May be None if the numpy data-type is not known (which can be the case for some string types, for example).

mask: bool, optional

If True (the default) then mask by convention when reading data from disk.

unpack: bool, optional

If True (the default) then unpack by convention when reading data from disk.

fragment_array: dict

A dictionary representation of the fragment array, in either “uri” form:

{'map': <'map' fragment variable data>,
 'uris': <'uris' fragment variable data>,
 'identifiers': <'identifiers' fragment variable data>}

or else in “unique_value” form:

{'map': <'map' fragment variable data>,
 'unique_values': <'unique_values' fragment variable data>}

filesystem: optional

A pre-authenticated filesystem object (for example an fsspec filesystem instance) to use for opening the filename. When the dataset is given as a string, it is treated as a path understood by filesystem and the dataset will be opened by calling filesystem.open(dataset, 'rb'), the result of which will be passed to the backend(s) (see the backend parameter). S3 is a special case for which the leading scheme and authority is automatically removed from the dataset name passed to the filesystem open method; for instance, the dataset 's3://authority/bucket/file.nc' is treated as filesystem.open('bucket/file.nc', 'rb').

If filename is not a string then filesystem is ignored.

If None (the default) then filename, regardless of its type, is passed unchanged to the backends.

Added in version (cfdm): 1.13.1.0

backend: None or (sequence of) str, optional Which library to use for opening a string-like, file-like, or directory-like dataset. An attempt to read the dataset is made by the given backends in the order in which they are provided, stopping after the first successful read. By default backend is None, which is equivalent to providing the ordered sequence of backends:

('pyfive', 'zarr', 'umfive', 'netCDF4', 'netcdf_file', 'h5py', 'xarray')

If the dataset is given as a (subclass of a) backend object, then that backend must be one of the backends identified by the backend parameter.

Performance may be improved by specifying a backend library, as this reduces or removes unsuccessful dataset read attempts, which can be expensive, especially for remote datasets.

The available backends, and the formats they can read, are:

Backend	Library	Dataset formats
'pyfive'	pyfive	netCDF-4
'zarr'	zarr	Zarr, Kerchunk
'umfive'	umfive	PP, fields file
'netCDF4'	netCDF4	netCDF-4, netCDF-3
'netcdf_file'	scipy.io.netcdf_file	netCDF-3
'h5py'	h5py	netCDF-4
'xarray'	xarray	netCDF-4, netCDF-3, Zarr, Kerchunk, GRIB

Note that the xarray library is itself an interface to other backends.

Example:

To only attempt 'netCDF4': 'netCDF4' or ['netCDF4']

Example:

To only attempt 'netCDF4' or 'pyfive', in that order: ('netCDF4', 'pyfive')

Added in version (cfdm): 1.13.3.0

backend_options: None or dict, optional

The options to use with each backend when opening a dataset.

Added in version (cfdm): 1.13.3.0

attributes: dict or None, optional

Provide netCDF attributes for the data as a dictionary of key/value pairs.

If attributes is None, the default, then the attributes will be set from the netCDF variable during the first __getitem__ call.

fragment_filesystem: optional

A pre-authenticated filesystem object (for example an fsspec filesystem instance) to use for opening the fragment.

If None (the default) then a fragment path is passed unchanged to the backends defined by the fragament_backend parameter.

When provided, a fragment path (fragment) is treated as the file-like object filesytem.open(fragment, 'rb') which is passed to the backends (see the fragment_backend parameter).

Added in version (cfdm): 1.13.3.0

fragment_backend: None or (sequence of) str, optional

Which library or libraries to use for reading the fragment. When the fragment is accessed, an attempt to open each fragment dataset is made by the given backends in order, stopping after the first successful read. The available backends are those allowed by xnetcdf.

By default fragment_backend is None, which is equivalent to providing the ordered sequence of the default backends for xnetcdf.

Added in version (cfdm): 1.13.3.0

fragment_backend_options: None or dict, optional

The options to use with each backend when opening a fragment dataset.

Added in version (cfdm): 1.13.3.0

source: optional

Convert source, which can be any type of object, to a AggregatedArray instance.

All other parameters, apart from copy, are ignored and their values are instead inferred from source by assuming that it has the AggregatedArray API. Any parameters that can not be retrieved from source in this way are assumed to have their default value.

Note that if x is also a AggregatedArray instance then cfdm.AggregatedArray(source=x) is equivalent to x.copy().

copy: bool, optional

If True (the default) then deep copy the input parameters prior to initialisation. By default the parameters are not deep copied.

storage_options: Deprecated at version NEXTERSION

Use filesystem instead.

storage_protocol: Deprecated at version NEXTERSION

Use filesystem instead.

Methods

Methods

Units	The Units object containing the units of the array.
array	Return an independent numpy array containing the data.
astype	Cast the data to a specified type.
close	Close the dataset containing the data.
copy	Return a deep copy of the array.
dtype	Data-type of the array.
file_directory	The file directory.
get_address	The name of the file containing the array.
get_attributes	The attributes of the array.
get_backend	The names of the packages for accessing the dataset.
get_backend_options	Backend options when opening a dataset.
get_calendar	The calendar of the array.
get_compression_type	Returns the array's compression type.
get_filename	The name of the file containing the array.
get_filesystem	Return the file system which contains the dataset.
get_fragment_array	Get the aggregation data dictionary.
get_fragment_array_shape	Get the sizes of the fragment dimensions.
get_fragment_backend	The names of the packages for accessing the fragment dataset.
get_fragment_backend_options	Backend options when opening a fragment dataset.
get_fragment_filesystem	Return the file system which contains the fragment dataset.
get_fragment_type	The type of fragments in the fragment array.
get_fragmented_dimensions	The positions of dimensions spanned by two or more fragments.
get_mask	Whether or not to automatically mask the data.
get_missing_values	The missing values of the data.
get_storage_options	Return the file system options.
get_storage_protocol	The file system protocol.
get_units	The units of the array.
get_unpack	Whether or not to automatically unpack the data.
get_variable	Get the open dataset variable object for the data.
has_remote_storage_protocol	Whether or not there is a remote file system protocol.
ndim	Number of array dimensions.
open	Return a dataset file object and address.
replace_directory	Replace the file directory.
replace_filename	Replace the file location.
shape	Shape of the array.
size	Number of elements in the array.
subarray_shapes	Create the subarray shapes.
subarrays	Return descriptors for every subarray.
to_dask_array	Create a dask array with FragmentArray chunks.
to_memory	Bring data on disk into memory.