cf.active_storage_max_requests

cf.active_storage_max_requests(*arg)[source]

Concurrent active storage server requests per dask chunk.

This is the maximum number of concurrent requests per dask chunk that are sent to the active storage server by an Active instance. The default is 100. The optimum number may be estimated by \(N = N_{max} / min(N_{PE}, N_{chunk})\), where

  • \(N_{max}\) is the maximum number of requests that the active storage server can process concurrently before its performance is degraded;

  • \(N_{PE}\) the number of available PEs on the local client;

  • \(N_{chunk}\) the number of dask chunks being reduced with active storage.

This formula only applies to cases where all dask chunks for the collapse operation are utilising active storage. If some are not then \(N\) will likely be underestimated.

Added in version 3.16.3.

See also

active_storage, active_storage_url, configuration

Parameters:
arg: int or Constant, optional

Provide a value that will apply to all subsequent operations.

Returns:
Constant

The value prior to the change, or the current value if no new value was specified.

Examples

>>> print(cf.active_storage_max_requests())
100
>>> with cf.active_storage_max_requests(25):
...     print(cf.active_storage_max_requests())
...
25
>>> print(cf.active_storage_max_requests())
None
>>> print(cf.active_storage_max_requests(12)
None
>>> cf.active_storage_max_requests()
12