CUDA Array Interface (Version 2)¶
The cuda array interface is created for interoperability between different implementation of GPU array-like objects in various projects. The idea is borrowed from the numpy array interface.
Currently, we only define the Python-side interface. In the future, we may add a C-side interface for efficient exchange of the information in compiled code.
Python Interface Specification¶
Experimental feature. Specification may change.
__cuda_array_interface__ attribute returns a dictionary (
that must contain the following entries:
A tuple of
long) representing the size of each dimension.
The type string. This has the same definition as
typestrin the numpy array interface.
The data is a 2-tuple. The first element is the data pointer as a Python
long). The data must be device-accessible. For zero-size arrays, use
0here. The second element is the read-only flag as a Python
Because the user of the interface may or may not be in the same context, the most common case is to use
CU_POINTER_ATTRIBUTE_DEVICE_POINTERin the CUDA driver API (or the equivalent CUDA Runtime API) to retrieve a device pointer that is usable in the currently active context.
An integer for the version of the interface being exported. The current version is 2.
The following are optional entries:
If strides is not given, or it is
None, the array is in C-contiguous layout. Otherwise, a tuple of
long) is explicitly given for representing the number of bytes to skip to access the next element at each dimension.
This is for describing more complicated types. This follows the same specification as in the numpy array interface.
Noneor object exposing the
Nonethen all values in data are valid. All elements of the mask array should be interpreted only as true or not true indicating which elements of this array are valid. This has the same definition as
maskin the numpy array interface.
Numba does not currently support working with masked CUDA arrays and will raise a
NotImplementedErrorexception if one is passed to a GPU function.
Obtaining the value of the
__cuda_array_interface__ property of any object
has no effect on the lifetime of the object from which it was created. In
particular, note that the interface has no slot for the owner of the data.
It is therefore imperative for a consumer to retain a reference to the object owning the data for as long as they make use of the data.
Lifetime management in Numba¶
Numba provides two mechanisms for creating device arrays. Which to use depends on whether the created device array should maintain the life of the object from which it is created:
as_cuda_array: This creates a device array that holds a reference to the owning object. As long as a reference to the device array is held, its underlying data will also be kept alive, even if all other references to the original owning object have been dropped.
from_cuda_array_interface: This creates a device array with no reference to the owning object by default. The owning object, or some other object to be considered the owner can be passed in the
The interfaces of these functions are:
Create a DeviceNDArray from any object that implements the cuda array interface.
A view of the underlying GPU buffer is created. No copying of the data is done. The resulting DeviceNDArray will acquire a reference from obj.
Create a DeviceNDArray from a cuda-array-interface description. The owner is the owner of the underlying memory. The resulting DeviceNDArray will acquire a reference from it.
Additional information about the data pointer can be retrieved using
cudaPointerGetAttributes. Such information
- the CUDA context that owns the pointer;
- is the pointer host-accessible?
- is the pointer a managed memory?
Differences with CUDA Array Interface (Version 0)¶
Version 0 of the CUDA Array Interface did not have the optional mask attribute to support masked arrays.
Differences with CUDA Array Interface (Version 1)¶
Versions 0 and 1 of the CUDA Array Interface neither clarified the strides attribute for C-contiguous arrays nor specified the treatment for zero-size arrays.