cf.Data¶

class
cf.
Data
(array=None, units=None, calendar=None, fill_value=None, hardmask=True, chunk=True, loadd=None, loads=None, dt=False, source=None, copy=True, dtype=None, mask=None, _use_array=True)[source]¶ Bases:
cf.mixin_container.Container
,cfdm.data.data.Data
An Ndimensional data array with units and masked values.
Contains an Ndimensional, indexable and broadcastable array with many similarities to a
numpy
array.Contains the units of the array elements.
Supports masked arrays, regardless of whether or not it was initialised with a masked array.
Stores and operates on data arrays which are larger than the available memory.
Indexing
A data array is indexable in a similar way to numpy array:
>>> d.shape (12, 19, 73, 96) >>> d[...].shape (12, 19, 73, 96) >>> d[slice(0, 9), 10:0:2, :, :].shape (9, 5, 73, 96)
There are three extensions to the numpy indexing functionality:
Size 1 dimensions are never removed by indexing.
An integer index i takes the ith element but does not reduce the rank of the output array by one:
>>> d.shape (12, 19, 73, 96) >>> d[0, ...].shape (1, 19, 73, 96) >>> d[:, 3, slice(10, 0, 2), 95].shape (12, 1, 5, 1)
Size 1 dimensions may be removed with the
squeeze
method.The indices for each axis work independently.
When more than one dimension’s slice is a 1d boolean sequence or 1d sequence of integers, then these indices work independently along each dimension (similar to the way vector subscripts work in Fortran), rather than by their elements:
>>> d.shape (12, 19, 73, 96) >>> d[0, :, [0, 1], [0, 13, 27]].shape (1, 19, 2, 3)
Boolean indices may be any object which exposes the numpy array interface.
>>> d.shape (12, 19, 73, 96) >>> d[..., d[0, 0, 0]>d[0, 0, 0].min()]
Cyclic axes
Miscellaneous
A
Data
object is picklable.A
Data
object is hashable, but note that, since it is mutable, its hash value is only valid whilst the data array is not changed in place.Initialization
 Parameters
 array: optional
The array of values. May be any scalar or arraylike object, including another
Data
instance. Ignored if the source parameter is set. Parameter example:
array=[34.6]
 Parameter example:
array=[[1, 2], [3, 4]]
 Parameter example:
array=numpy.ma.arange(10).reshape(2, 1, 5)
 units:
str
orUnits
, optional The physical units of the data. if a
Units
object is provided then this an also set the calendar. Ignored if the source parameter is set.The units (without the calendar) may also be set after initialisation with the
set_units
method. Parameter example:
units='km hr1'
 Parameter example:
units='days since 20181201'
 calendar:
str
, optional The calendar for reference time units. Ignored if the source parameter is set.
The calendar may also be set after initialisation with the
set_calendar
method. Parameter example:
calendar='360_day'
 fill_value: optional
The fill value of the data. By default, or if set to
None
, thenumpy
fill value appropriate to the array’s datatype will be used (seenumpy.ma.default_fill_value
). Ignored if the source parameter is set.The fill value may also be set after initialisation with the
set_fill_value
method. Parameter example:
fill_value=999.
 dtype: datatype, optional
The desired datatype for the data. By default the datatype will be inferred form the array parameter.
The datatype may also be set after initialisation with the
dtype
attribute. Parameter example:
dtype=float
 Parameter example:
dtype='float32'
 Parameter example:
dtype=numpy.dtype('i2')
New in version 3.0.4.
 mask: optional
Apply this mask to the data given by the array parameter. By default, or if mask is
None
, no mask is applied. May be any scalar or arraylike object (such as alist
,numpy
array orData
instance) that is broadcastable to the shape of array. Masking will be carried out where the mask elements evaluate toTrue
.This mask will applied in addition to any mask already defined by the array parameter.
New in version 3.0.5.
 source: optional
Initialize the array, units, calendar and fill value from those of source.
 hardmask:
bool
, optional If False then the mask is soft. By default the mask is hard.
 dt:
bool
, optional If True then strings (such as
'19901201 12:00'
) given by the array parameter are reinterpreted as datetime objects. By default they are not. loadd:
dict
, optional Initialise the data from a dictionary serialization of a
cf.Data
object. All other arguments are ignored. See thedumpd
andloadd
methods. loads:
str
, optional Initialise the data array from a string serialization of a
Data
object. All other arguments are ignored. See thedumps
andloads
methods. copy:
bool
, optional If False then do not deep copy input parameters prior to initialization. By default arguments are deep copied.
 chunk:
bool
, optional If False then the data array will be stored in a single partition. By default the data array will be partitioned if it is larger than the chunk size, as returned by the
cf.chunksize
function.
Examples:
>>> d = cf.Data(5) >>> d = cf.Data([1,2,3], units='K') >>> import numpy >>> d = cf.Data(numpy.arange(10).reshape(2,5), ... units=Units('m/s'), fill_value=999) >>> d = cf.Data(tuple('fly'))
Inspection¶
Attributes
A numpy array copy the data array. 

A numpy array view the data array. 

The 

Number of dimensions in the data array. 

Tuple of the data array’s dimension sizes. 

Number of elements in the data array. 

Total number of bytes consumed by the elements of the array. 

Return a string containing a full description of the instance. 

Inspect the object for debugging. 

True if the data array is a 0d scalar array. 
Units¶
Delete the units. 

Return the units. 

Whether units have been set. 

Set the units. 

Override the data array units. 

Delete the calendar. 

Return the calendar. 

Whether a calendar has been set. 

Set the calendar. 

Override the calendar of the data array elements. 

Change the calendar of the data array elements. 
Attributes
The 
Data creation routines¶
Ones and zeros¶
Create a new data array without initializing the elements. 

Returns a new data array of given shape and type, filled with the given value. 

Returns a new array filled with ones of set shape and type. 

Returns a new array filled with zeros of set shape and type. 

Return a new data array of given shape and type with all elements masked. 
From existing data¶
Return a deep copy. 

Convert the input to a 

Reset the data in place from a dictionary serialization. 

Reset the data in place from a string serialization. 
Data manipulation routines¶
Changing data shape¶
Flatten axes of the data. 
Transposelike operations¶
Interchange two axes of an array. 

Permute the axes of the data array. 
Changing number of dimensions¶
Expand the shape of the data array in place. 

Remove size 1 axes from the data array. 
Joining data¶
Join a sequence of data arrays together. 

Concatenates a list of Data objects into a single Data object along the specified access (see cf.Data.concatenate for details). 
Adding and removing elements¶
The unique elements of the array. 
Rearranging elements¶
Reverse the direction of axes of the data array. 

Roll array elements along a given axis. 
Expanding the data¶
Expand the data by adding a halo. 
Binary operations¶
Datetime support¶
Attributes
An independent numpy array of datetime objects. 

Returns an independent numpy array with datetimes as strings. 

The day of each data array element. 

Alias for 

The hour of each data array element. 

The minute of each data array element. 

The month of each data array element. 

The second of each data array element. 

The year of each data array element. 
Indexing routines¶
Single value selection¶
Return an element of the data array as a standard Python scalar. 

Return the first element of the data as a scalar. 

Return the second element of the data as a scalar. 

Return the last element of the data as a scalar. 
Iterating over data¶
Return a flat iterator over elements of the data array. 

Return an iterator over the Ndimensional indices of the data array. 
Cyclic axes¶
Returns or sets the axes of the data array which are cyclic. 
Input and output¶
Return a serialization of the data array. 

Return a JSON string serialization of the data array. 

Return the array as a (possibly nested) list. 
Linear algebra¶
Compute the outer product with another data array. 
Logic functions¶
Truth value testing¶
Test whether all data array elements evaluate to True. 

Test whether any data array elements evaluate to True. 
Comparison¶
Returns True if two broadcastable arrays have equal values, False otherwise. 

Return where data are elementwise equal to other, broadcastable data. 

True if two data arrays are logically equal, False otherwise. 
Mask support¶
Apply masking. 

Count the nonmasked elements of the data. 

Count the masked elements of the data. 

Return all nonmasked values in a one dimensional data array. 

Replace masked elements with the fill value. 

Masking of floatingpoint errors in the results of arithmetic operations. 

Mask the array where invalid values occur (NaN or inf). 

Delete the fill value. 

Return the missing data value. 

Whether a fill value has been set. 

Set the missing data value. 
Attributes
A binary (0 and 1) mask of the data array. 

Whether the mask is hard (True) or soft (False). 

True if the data array has any masked values. 

The boolean missing data mask of the data array. 

The data array missing data value. 
Mathematical functions¶
Trigonometric functions¶
Take the trigonometric sine of the data elementwise. 

Take the trigonometric cosine of the data elementwise. 

Take the trigonometric tangent of the data elementwise. 

Take the trigonometric inverse sine of the data elementwise. 

Take the trigonometric inverse cosine of the data element wise. 

Take the trigonometric inverse tangent of the data element wise. 
Hyperbolic functions¶
Take the hyperbolic sine of the data elementwise. 

Take the hyperbolic cosine of the data elementwise. 

Take the hyperbolic tangent of the data elementwise. 

Take the inverse hyperbolic sine of the data elementwise. 

Take the inverse hyperbolic cosine of the data elementwise. 

Take the inverse hyperbolic tangent of the data elementwise. 
Rounding¶
The ceiling of the data, elementwise. 

Return the floor of the data array. 

Round the data to the nearest integer, elementwise. 

Evenly round elements of the data array to the given number of decimals. 

Return the truncated values of the data array. 
Sums, products, differences¶
Return the data cumulatively summed along the given axis. 

Calculate the nth discrete difference along the given axis. 

Collapse axes with their sum. 
Convolution filters
Return the data convolved along the given axis with the specified filter. 
Exponents and logarithms¶
Take the exponential of the data array. 

Takes the logarithm of the data array. 
Miscellaneous¶
Clip (limit) the values in the data array in place. 

Apply an elementwise array operation to the data array. 
Set routines¶
Making proper sets¶
The unique elements of the array. 
Sorting, searching, and counting¶
Searching¶
Return the indices of the maximum values along an axis. 

Assign to data elements depending on a condition. 
Counting¶
Count the nonmasked elements of the data. 

Count the masked elements of the data. 
Statistics¶
Order statistics¶
Collapse axes with their maximum. 

Collapse axes with their maximum absolute value. 

Collapse axes with their minimum. 

Collapse axes with their minimum absolute value. 

Compute percentiles of the data along the specified axes. 

Alias for 

Alias for 
Averages and variances¶
Collapse axes with their mean. 

Collapse axes with their mean absolute value. 

Compute the mean the of upper decile. 

Compute the median of the values. 

Collapse axes with the unweighted average of their maximum and minimum values. 

Collapse axes with the absolute difference between their maximum and minimum values. 

Collapse axes with their root mean square. 

Alias for 

Alias for 

Collapse axes by calculating their standard deviation. 

Collapse axes with their weighted variance. 
Sums¶
Collapse axes with their integral. 

Collapse axes with their sum. 

Collapse axes with the sum of the squares of the values. 
Histograms¶
Return the indices of the bins to which each value belongs. 
Miscellaneous¶
Collapses axes with their sample size. 

Calculate statistics of the data. 

Collapse axes with the sum of weights. 

Collapse axes with the sum of squares of weights. 
Error handling¶
Set how floatingpoint errors in the results of arithmetic operations are handled. 
Compression by convention¶
Returns the dimensions that are compressed in the array. 

Returns the compressed dimension’s array position. 

Returns the type of compression applied to the array. 

Return the count variable for a compressed array. 

Return the index variable for a compressed array. 

Return the list variable for a compressed array. 

Uncompress the underlying data. 
Attributes
Returns an independent numpy array of the compressed data. 
Miscellaneous¶
Return the commands that would create the data object. 

Return the names of files containing parts of the data array. 

Returns the data. 

Return the underlying array object. 
Attributes
The data as an object identity. 
Performance¶
Clear the HDF5 chunksizes for the data. 

Return the HDF5 chunksizes for the data. 

Set the HDF5 chunksizes for the data. 

Close all files referenced by the data array. 

Partition the data array. 

Add partition boundaries. 

Return the partition boundaries for each partition matrix dimension. 

Return parameters for opening and closing array partitions. 

Storage for the partitions matrix. 

True if the data array is partitioned. 

Store the data array on disk. 

Store each partition’s data in memory in place if the master array is smaller than the chunk size. 

True if the array is retained in memory. 

Return True if the master array is small enough to be retained in memory. 

Return True if the master array is small enough to be retained in memory. 

Returns a dictionary of sections of the Data object. 

Expects a dictionary of Data objects with ordering information as keys, as output by the section method when called with a Data object. 

TODO. 
Elementwise arithmetic, bit and comparison operations¶
Arithmetic, bit and comparison operations are defined as elementwise
data array operations which yield a new cf.Data
object or, for
augmented assignments, modify the data inplace.
Comparison operators
The rich comparison operator 

The rich comparison operator 

The rich comparison operator 

The rich comparison operator 

The rich comparison operator 

The rich comparison operator 
Truth value of an array
Truth value testing and the builtin operation 
Binary arithmetic operators
The binary arithmetic operation 

The binary arithmetic operation 

The binary arithmetic operation 

The binary arithmetic operation 

The binary arithmetic operation 

The binary arithmetic operation 

The binary arithmetic operations 

The binary arithmetic operation 
Binary arithmetic operators with reflected (swapped) operands
The binary arithmetic operation 

The binary arithmetic operation 

The binary arithmetic operation 

The binary arithmetic operation 

The binary arithmetic operation 

The binary arithmetic operation 

The binary arithmetic operations 

The binary arithmetic operation 
Augmented arithmetic assignments
The augmented arithmetic assignment 

The augmented arithmetic assignment 

The augmented arithmetic assignment 

The augmented arithmetic assignment 

The augmented arithmetic assignment 

The augmented arithmetic assignment 

The augmented arithmetic assignment 

The binary arithmetic operation 
Unary arithmetic operators
The unary arithmetic operation 

The unary arithmetic operation 

The unary arithmetic operation 
Binary bitwise operators
The binary bitwise operation 

The binary bitwise operation 

The binary bitwise operation 

The binary bitwise operation 

The binary bitwise operation 
..rubric:: Binary bitwise operators with reflected (swapped) operands
The binary bitwise operation 

The binary bitwise operation 

The binary bitwise operation 

The binary bitwise operation 

The binary bitwise operation 
Augmented bitwise assignments
The augmented bitwise assignment 

The augmented bitwise assignment 

The augmented bitwise assignment 

The augmented bitwise assignment 

The augmented bitwise assignment 
Unary bitwise operators
The unary bitwise operation 
Special¶
The numpy array interface. 

Membership test operator 

Returns a new reference to self. 

Called by the 

Return a subspace of the data defined by indices. 

The builtin function 

Called when an iterator is required. 

The builtin function 

Called by the 

Implement indexed assignment. 

Called by the 

Implements the “member of set” condition. 

Implements the “within a range” condition. 

Implements the “without a range” condition. 
Deprecated¶
Methods
Deprecated at version 3.0.0, use attribute 

Deprecated at version 3.0.0. 

Deprecated at version 3.0.0, use method 

Deprecated at version 3.4.0, use method 

cf.Data.save_to_disk is dead. 

Deprecated at version 3.0.0, use 