cf.Data¶

class
cf.
Data
(array=None, units=None, calendar=None, fill_value=None, hardmask=True, chunks='auto', dt=False, source=None, copy=True, dtype=None, mask=None, to_memory=False, init_options=None, _use_array=True)[source]¶ Bases:
cf.data.mixin.deprecations.DataClassDeprecationsMixin
,cf.mixin2.cfanetcdf.CFANetCDF
,cf.mixin2.container.Container
,cfdm.data.data.Data
An Ndimensional data array with units and masked values.
Contains an Ndimensional, indexable and broadcastable array with many similarities to a
numpy
array.Contains the units of the array elements.
Supports masked arrays, regardless of whether or not it was initialised with a masked array.
Stores and operates on data arrays which are larger than the available memory.
Indexing
A data array is indexable in a similar way to numpy array:
>>> d.shape (12, 19, 73, 96) >>> d[...].shape (12, 19, 73, 96) >>> d[slice(0, 9), 10:0:2, :, :].shape (9, 5, 73, 96)
There are three extensions to the numpy indexing functionality:
Size 1 dimensions are never removed by indexing.
An integer index i takes the ith element but does not reduce the rank of the output array by one:
>>> d.shape (12, 19, 73, 96) >>> d[0, ...].shape (1, 19, 73, 96) >>> d[:, 3, slice(10, 0, 2), 95].shape (12, 1, 5, 1)
Size 1 dimensions may be removed with the
squeeze
method.The indices for each axis work independently.
When more than one dimension’s slice is a 1d boolean sequence or 1d sequence of integers, then these indices work independently along each dimension (similar to the way vector subscripts work in Fortran), rather than by their elements:
>>> d.shape (12, 19, 73, 96) >>> d[0, :, [0, 1], [0, 13, 27]].shape (1, 19, 2, 3)
Boolean indices may be any object which exposes the numpy array interface.
>>> d.shape (12, 19, 73, 96) >>> d[..., d[0, 0, 0]>d[0, 0, 0].min()]
Cyclic axes
Initialisation
 Parameters
 array: optional
The array of values. May be any scalar or arraylike object, including another
Data
instance. Parameter example:
array=[34.6]
 Parameter example:
array=[[1, 2], [3, 4]]
 Parameter example:
array=numpy.ma.arange(10).reshape(2, 1, 5)
 units:
str
orUnits
, optional The physical units of the data. if a
Units
object is provided then this an also set the calendar.The units (without the calendar) may also be set after initialisation with the
set_units
method. Parameter example:
units='km hr1'
 Parameter example:
units='days since 20181201'
 calendar:
str
, optional The calendar for reference time units.
The calendar may also be set after initialisation with the
set_calendar
method. Parameter example:
calendar='360_day'
 fill_value: optional
The fill value of the data. By default, or if set to
None
, thenumpy
fill value appropriate to the array’s datatype will be used (seenumpy.ma.default_fill_value
).The fill value may also be set after initialisation with the
set_fill_value
method. Parameter example:
fill_value=999.
 dtype: datatype, optional
The desired datatype for the data. By default the datatype will be inferred form the array parameter.
The datatype may also be set after initialisation with the
dtype
attribute. Parameter example:
dtype=float
 Parameter example:
dtype='float32'
 Parameter example:
dtype=numpy.dtype('i2')
New in version 3.0.4.
 mask: optional
Apply this mask to the data given by the array parameter. By default, or if mask is
None
, no mask is applied. May be any scalar or arraylike object (such as alist
,numpy
array orData
instance) that is broadcastable to the shape of array. Masking will be carried out where the mask elements evaluate toTrue
.This mask will applied in addition to any mask already defined by the array parameter.
New in version 3.0.5.
 source: optional
Convert source, which can be any type of object, to a
Data
instance.All other parameters, apart from copy, are ignored and their values are instead inferred from source by assuming that it has the
Data
API. Any parameters that can not be retrieved from source in this way are assumed to have their default value.Note that if
x
is also aData
instance thencf.Data(source=x)
is equivalent tox.copy()
. hardmask:
bool
, optional If False then the mask is soft. By default the mask is hard.
 dt:
bool
, optional If True then strings (such as
'19901201 12:00'
) given by the array parameter are reinterpreted as datetime objects. By default they are not. copy:
bool
, optional If True (the default) then deep copy the input parameters prior to initialisation. By default the parameters are not deep copied.
 chunks:
int
,tuple
,dict
orstr
, optional Specify the chunking of the underlying dask array.
Any value accepted by the chunks parameter of the
dask.array.from_array
function is allowed.By default,
"auto"
is used to specify the array chunking, which uses a chunk size in bytes defined by thecf.chunksize
function, preferring squarelike chunk shapes. Parameter example:
A blocksize like
1000
. Parameter example:
A blockshape like
(1000, 1000)
. Parameter example:
Explicit sizes of all blocks along all dimensions like
((1000, 1000, 500), (400, 400))
. Parameter example:
A size in bytes, like
"100MiB"
which will choose a uniform blocklike shape, preferring squarelike chunk shapes. Parameter example:
A blocksize of
1
orNone
in a tuple or dictionary indicates the size of the corresponding dimension. Parameter example:
Blocksizes of some or all dimensions mapped to dimension positions, like
{1: 200}
, or{0: 1, 1: (400, 400)}
.
New in version 3.14.0.
 to_memory:
bool
, optional If True then ensure that the original data are in memory, rather than on disk.
If the original data are on disk, then reading data into memory during initialisation will slow down the initialisation process, but can considerably improve downstream performance by avoiding the need for independent reads for every dask chunk, each time the data are computed.
In general, setting to_memory to True is not the same as calling the
persist
of the newly createdData
object, which also decompresses data compressed by convention and computes any data type, mask and datetime modifications.If the input array is a
dask.array.Array
object then to_memory is ignored.New in version 3.14.0.
 init_options:
dict
, optional Provide optional keyword arguments to methods and functions called during the initialisation process. A dictionary key identifies a method or function. The corresponding value is another dictionary whose key/value pairs are the keyword parameter names and values to be applied.
Supported keys are:
'from_array'
: Provide keyword arguments to thedask.array.from_array
function. This is used when initialising data that is not already a dask array and is not compressed by convention.'first_non_missing_value'
: Provide keyword arguments to thecf.data.utils.first_non_missing_value
function. This is used when the input array contains datetime strings or objects, and may affect performance.
 Parameter example:
{'from_array': {'inline_array': True}}
 chunk: deprecated at version 3.14.0
Use the chunks parameter instead.
Examples
>>> d = cf.Data(5) >>> d = cf.Data([1,2,3], units='K') >>> import numpy >>> d = cf.Data(numpy.arange(10).reshape(2,5), ... units=Units('m/s'), fill_value=999) >>> d = cf.Data('fly') >>> d = cf.Data(tuple('fly'))
Inspection¶
Attributes
A numpy array copy of the data. 

A numpy array view of the data array. 

The 

Number of dimensions in the data array. 

Tuple of the data array’s dimension sizes. 

Number of elements in the data array. 

Total number of bytes consumed by the elements of the array. 

Return a string containing a full description of the instance. 

Inspect the object for debugging. 

True if the data is a 0d scalar array. 
Units¶
Delete the units. 

Return the units. 

Whether units have been set. 

Set the units. 

Override the data array units. 

Delete the calendar. 

Return the calendar. 

Whether a calendar has been set. 

Set the calendar. 

Override the calendar of the data array elements. 

Change the calendar of datetime array elements. 
Attributes
The 
Dask¶
Remove unnecessary tasks from the dask graph inplace. 

Returns a dask array of the compressed data. 

Change the chunk structure of the data. 

Return indices that define each dask compute chunk. 

Return a dictionary of the dask graph key/value pairs. 

Convert the data to a 

Get the deterministic name for the data. 

Whether there is a deterministic name for the data. 
Attributes
A numpy view the data. 

The chunk sizes for each dimension. 

The total number of chunks. 

The number of chunks along each dimension. 
Data creation routines¶
Ones and zeros¶
Return a new array of given shape and type, without initialising entries. 

Return a new array of given shape and type, filled with a fill value. 

Returns a new array filled with ones of set shape and type. 

Returns a new array filled with zeros of set shape and type. 

Return an empty masked array with all elements masked. 
From existing data¶
Return a deep copy. 

Convert the input to a 

Reset the data in place from a dictionary serialisation. 

Reset the data in place from a string serialisation. 
Data manipulation routines¶
Changing data shape¶
Flatten specified axes of the data. 

Change the shape of the data without changing its values. 
Transposelike operations¶
Interchange two axes of an array. 

Permute the axes of the data array. 
Changing number of dimensions¶
Expand the shape of the data array in place. 

Change the shape of the data without changing its values. 

Remove size 1 axes from the data array. 
Joining data¶
Join a sequence of data arrays together. 

Concatenates a list of Data objects along the specified axis. 
Adding and removing elements¶
The unique elements of the data. 
Rearranging elements¶
Reverse the direction of axes of the data array. 

Roll array elements along one or more axes. 
Expanding the data¶
Expand the data by adding a halo. 
Binary operations¶
Datetime support¶
Attributes
Change the calendar of datetime array elements. 

Convert reference time data values to have new units. 

An independent numpy array of datetime objects. 

Returns an independent numpy array with datetimes as strings. 

The day of each datetime value. 

Alias for 

The hour of each datetime value. 

The minute of each datetime value. 

The month of each datetime value. 

The second of each datetime value. 

The year of each datetime value. 
Indexing routines¶
Single value selection¶
Return an element of the data array as a standard Python scalar. 

Return the first element of the data as a scalar. 

Return the second element of the data as a scalar. 

Return the last element of the data as a scalar. 
Iterating over data¶
Return a flat iterator over elements of the data array. 

Return an iterator over the Ndimensional indices of the data array. 
Cyclic axes¶
Get or set the cyclic axes. 
Input and output¶
Return a serialisation of the data array. 

Return a JSON string serialisation of the data array. 

Return the data as a scalar or (nested) list. 
Linear algebra¶
Compute the outer product with another data array. 
Logic functions¶
Truth value testing¶
Test whether all data array elements evaluate to True. 

Test whether any data array elements evaluate to True. 
Comparison¶
Whether an array is elementwise equal within a tolerance. 

Return where data are elementwise equal within a tolerance. 

True if two data arrays are logically equal, False otherwise. 
Mask support¶
Apply masking. 

Count the nonmasked elements of the data. 

Count the masked elements of the data. 

Return all nonmasked values in a one dimensional data array. 

Replace masked elements with a fill value. 

Force the mask to hard. 

Mask the array where invalid values occur (NaN or inf). 

Delete the fill value. 

Return the missing data value. 

Whether a fill value has been set. 

Set the missing data value. 

Force the mask to soft. 
Attributes
A binary (0 and 1) mask of the data array. 

Hardness of the mask. 

True if the data array has any masked values. 

The Boolean missing data mask of the data array. 

The data array missing data value. 
Mathematical functions¶
Trigonometric functions¶
Take the trigonometric sine of the data elementwise. 

Take the trigonometric cosine of the data elementwise. 

Take the trigonometric tangent of the data elementwise. 

Take the trigonometric inverse sine of the data elementwise. 

Take the trigonometric inverse cosine of the data element wise. 

Take the trigonometric inverse tangent of the data element wise. 
Hyperbolic functions¶
Take the hyperbolic sine of the data elementwise. 

Take the hyperbolic cosine of the data elementwise. 

Take the hyperbolic tangent of the data elementwise. 

Take the inverse hyperbolic sine of the data elementwise. 

Take the inverse hyperbolic cosine of the data elementwise. 

Take the inverse hyperbolic tangent of the data elementwise. 
Rounding¶
The ceiling of the data, elementwise. 

Return the floor of the data array. 

Round the data to the nearest integer, elementwise. 

Evenly round elements of the data array to the given number of decimals. 

Return the truncated values of the data array. 
Sums, products, differences, powers¶
Return the data cumulatively summed along the given axis. 

Calculate the nth discrete difference along the given axis. 

Calculate the elementwise square. 

Calculate the nonnegative square root. 

Calculate sum values. 
Convolution filters
Return the data convolved along the given axis with the specified filter. 
Exponents and logarithms¶
Take the exponential of the data array. 

Takes the logarithm of the data array. 
Miscellaneous¶
Clip (limit) the values in the data array in place. 

Apply an elementwise array operation to the data array. 
Set routines¶
Making proper sets¶
The unique elements of the data. 
Sorting, searching, and counting¶
Searching¶
Return the indices of the maximum values along an axis. 

Return the indices of the minimum values along an axis. 

Assign array elements depending on a condition. 
Counting¶
Count the nonmasked elements of the data. 

Count the masked elements of the data. 
Statistics¶
Order statistics¶
Alias for 

Calculate maximum absolute values. 

Alias for 

Calculate minimum absolute values. 

Compute percentiles of the data along the specified axes. 

Calculate maximum values. 

Calculate minimum values. 
Averages and variances¶
Calculate mean values. 

Calculate mean absolute values. 

Mean of values defined by the upper tenth of their distribution. 

Calculate median values. 

Calculate midrange values. 

Calculate range values. 

Calculate root mean square (RMS) values. 

Alias for 

Alias for 

Alias for 

Calculate standard deviations. 

Calculate variances. 
Sums¶
Calculate summed values. 

Calculate sum values. 

Calculate sums of squares. 
Histograms¶
Return the indices of the bins to which each value belongs. 
Miscellaneous¶
Calculate sample size values. 

Calculate statistics of the data. 

Calculate sums of weights. 

Calculate sums of squares of weights. 
Error handling¶
Set how floatingpoint errors in the results of arithmetic operations are handled. 
Compression by convention¶
Returns the dimensions that are compressed in the array. 

Returns the compressed dimension’s array position. 

Returns the type of compression applied to the array. 

Return the count variable for a compressed array. 

Return the index variable for a compressed array. 

Return the list variable for a compressed array. 

Return the list variable for a compressed array. 

Return the list variable for a compressed array. 

Return the list variable for a compressed array. 

Uncompress the data. 
Attributes
Returns an independent numpy array of the compressed data. 
Miscellaneous¶
Return the commands that would create the data object. 

Returns the data. 

The names of files containing parts of the data array. 

The names of files containing the original data and metadata. 

Return the underlying array object. 




Attributes
The data as an object identity. 
Performance¶
Clear the HDF5 chunksizes for the data. 

Return the HDF5 chunksizes for the data. 

Set the HDF5 chunksizes for the data. 

Change the chunk structure of the data. 

Close all files referenced by the data array. 

The chunk sizes for each dimension. 

Change the chunk structure of the data. 



Add partition boundaries. 

Return the partition boundaries for each partition matrix dimension. 





True if the data array is partitioned. 

Store the data array on disk. 

Bring data on disk into memory. 

True if the array is retained in memory. 

Return True if the array is small enough to be retained in memory. 

Returns a dictionary of sections of the 

Persist the underlying dask array into memory. 
Attributes
The chunk sizes for each dimension. 

The total number of chunks. 

The number of chunks along each dimension. 
CFA¶
Add a new file location inplace. 

Remove all of the CFAnetCDF file name substitutions. 

Remove the CFAnetCDF aggregation instruction terms. 

Remove a CFAnetCDF file name substitution. 

Return the CFAnetCDF file name substitutions. 

Return the CFAnetCDF aggregation instruction terms. 

The CFA aggregation instruction term status. 

The CFA write status of the data. 

Whether any CFAnetCDF aggregation instruction terms have been set. 

Whether any CFAnetCDF file name substitutions have been set. 

Set the CFAnetCDF aggregation instruction terms. 

Set the CFA aggregation instruction term status. 

Set the CFA write status of the data. 

Set CFAnetCDF file name substitutions. 

Remove a file location inplace. 

The locations of files containing parts of the data. 
Elementwise arithmetic, bit and comparison operations¶
Arithmetic, bit and comparison operations are defined as elementwise
data array operations which yield a new cf.Data
object or, for
augmented assignments, modify the data inplace.
Comparison operators
The rich comparison operator 

The rich comparison operator 

The rich comparison operator 

The rich comparison operator 

The rich comparison operator 

The rich comparison operator 
Truth value of an array
Truth value testing and the builtin operation 
Binary arithmetic operators
The binary arithmetic operation 

The binary arithmetic operation 

The binary arithmetic operation 

The binary arithmetic operation 

The binary arithmetic operation 

The binary arithmetic operation 

The binary arithmetic operations 

The binary arithmetic operation 
Binary arithmetic operators with reflected (swapped) operands
The binary arithmetic operation 

The binary arithmetic operation 

The binary arithmetic operation 

The binary arithmetic operation 

The binary arithmetic operation 

The binary arithmetic operation 

The binary arithmetic operations 

The binary arithmetic operation 
Augmented arithmetic assignments
The augmented arithmetic assignment 

The augmented arithmetic assignment 

The augmented arithmetic assignment 

The augmented arithmetic assignment 

The augmented arithmetic assignment 

The augmented arithmetic assignment 

The augmented arithmetic assignment 

The binary arithmetic operation 
Unary arithmetic operators
The unary arithmetic operation 

The unary arithmetic operation 

The unary arithmetic operation 
Binary bitwise operators
The binary bitwise operation 

The binary bitwise operation 

The binary bitwise operation 

The binary bitwise operation 

The binary bitwise operation 
..rubric:: Binary bitwise operators with reflected (swapped) operands
The binary bitwise operation 

The binary bitwise operation 

The binary bitwise operation 

The binary bitwise operation 

The binary bitwise operation 
Augmented bitwise assignments
The augmented bitwise assignment 

The augmented bitwise assignment 

The augmented bitwise assignment 

The augmented bitwise assignment 

The augmented bitwise assignment 
Unary bitwise operators
The unary bitwise operation 
Special¶
Membership test operator 

Called by the 

Return a subspace of the data defined by indices. 

The builtin function 

Called when an iterator is required. 

Called to implement the builtin function 

Called by the 

Implement indexed assignment. 

Called by the 

The numpy array interface. 

Returns a new reference to self. 

Query interface method for an “is close” condition. 
Deprecated¶
Methods
Partition the data array. 

Deprecated at version 3.0.0, use attribute 

Deprecated at version 3.0.0. 

Return a serialisation of the data array. 

Return a JSON string serialisation of the data array. 

Deprecated at version 3.0.0, use method 

Deprecated at version 3.4.0, consider using method 

Return True if the master array is small enough to be retained in memory. 

Get or set HDF chunk sizes. 

True if the array is retained in memory. 

True if the data array has any masked values. 

Masking of floatingpoint errors in the results of arithmetic operations. 

Mask the array where invalid values occur (NaN or inf). 

Return the partition boundaries for each partition matrix dimension. 





Expects a dictionary of Data objects with ordering information as keys, as output by the section method when called with a Data object. 

Deprecated. 

Set how floatingpoint errors in the results of arithmetic operations are handled. 

Store the data array on disk. 

Bring data on disk into memory. 

Deprecated at version 3.0.0. 
Attributes
True if the data array is partitioned. 