oneAPI Deep Neural Network Library (oneDNN)  1.95.0
Performance library for Deep Learning
Classes | Public Types | Public Member Functions | Static Public Member Functions | List of all members
dnnl::memory Struct Reference

Memory object. More...

#include <dnnl.hpp>

Inheritance diagram for dnnl::memory:
Inheritance graph
[legend]
Collaboration diagram for dnnl::memory:
Collaboration graph
[legend]

Classes

struct  desc
 A memory descriptor. More...
 

Public Types

enum  data_type
 Data type specification. More...
 
enum  format_kind
 Memory format kind. More...
 
enum  format_tag
 Memory format tag specification. More...
 
typedef dnnl_dim_t dim
 Integer type for representing dimension sizes and indices.
 
typedef std::vector< dimdims
 Vector of dimensions. More...
 

Public Member Functions

 memory ()=default
 Default constructor. More...
 
 memory (const desc &md, const engine &engine, void *ahandle)
 Constructs a memory object. More...
 
template<typename T , int ndims = 1>
 memory (const desc &md, const engine &aengine, cl::sycl::buffer< T, ndims > &buf)
 Constructs a memory object from a SYCL buffer. More...
 
 memory (const desc &md, const engine &aengine)
 Constructs a memory object. More...
 
desc get_desc () const
 Returns the associated memory descriptor.
 
engine get_engine () const
 Returns the associated engine.
 
void * get_data_handle () const
 Returns the underlying memory buffer. More...
 
void set_data_handle (void *handle, const stream &astream) const
 Sets the underlying memory buffer. More...
 
void set_data_handle (void *handle) const
 Sets the underlying memory buffer. More...
 
template<typename T = void>
T * map_data () const
 Maps a memory object and returns a host-side pointer to a memory buffer with a copy of its contents. More...
 
void unmap_data (void *mapped_ptr) const
 Unmaps a memory object and writes back any changes made to the previously mapped memory buffer. More...
 
cl_mem get_ocl_mem_object () const
 Returns the OpenCL memory object associated with the memory.
 
void set_ocl_mem_object (cl_mem mem_object)
 Sets the OpenCL memory object mem_object associated with the memory. More...
 
template<typename T , int ndims = 1>
cl::sycl::buffer< T, ndims > get_sycl_buffer (size_t *offset=nullptr) const
 Returns the underlying SYCL buffer object. More...
 
template<typename T , int ndims>
void set_sycl_buffer (cl::sycl::buffer< T, ndims > &buf)
 Sets the underlying buffer to the given SYCL buffer. More...
 
- Public Member Functions inherited from dnnl::handle< dnnl_memory_t >
bool operator== (const handle< dnnl_memory_t, handle_traits< dnnl_memory_t > > &other) const
 Equality operator. More...
 
bool operator!= (const handle &other) const
 Inequality operator. More...
 
 handle ()=default
 Constructs an empty handle object. More...
 
 handle (const handle< dnnl_memory_t, handle_traits< dnnl_memory_t > > &)=default
 Copy constructor.
 
 handle (handle< dnnl_memory_t, handle_traits< dnnl_memory_t > > &&)=default
 Move constructor.
 
 handle (dnnl_memory_t t, bool weak=false)
 Constructs a handle wrapper object from a C API handle. More...
 
handle< dnnl_memory_t, handle_traits< dnnl_memory_t > > & operator= (const handle< dnnl_memory_t, handle_traits< dnnl_memory_t > > &)=default
 Assignment operator.
 
handle< dnnl_memory_t, handle_traits< dnnl_memory_t > > & operator= (handle< dnnl_memory_t, handle_traits< dnnl_memory_t > > &&)=default
 Move assignment operator.
 
void reset (dnnl_memory_t t, bool weak=false)
 Resets the handle wrapper objects to wrap a new C API handle. More...
 
dnnl_memory_t get (bool allow_empty=false) const
 Returns the underlying C API handle. More...
 
 operator dnnl_memory_t () const
 Converts a handle to the underlying C API handle type. More...
 
 operator bool () const
 Checks whether the object is not empty. More...
 

Static Public Member Functions

template<typename T >
static void validate_dims (const std::vector< T > &v, int min_size=0)
 Helper function that validates that an std::vector of dimensions can be safely converted to the C API array dnnl_dims_t. More...
 

Detailed Description

Memory object.

A memory object encapsulates a handle to a memory buffer allocated on a specific aengine, tensor dimensions, data type, and memory format, which is the way tensor indices map to offsets in linear memory space. Memory objects are passed to primitives during execution.

Examples
batch_normalization.cpp, binary.cpp, cnn_inference_f32.cpp, cnn_inference_int8.cpp, cnn_training_bf16.cpp, cnn_training_f32.cpp, concat.cpp, convolution.cpp, cpu_matmul_quantization.cpp, cpu_rnn_inference_f32.cpp, cpu_rnn_inference_int8.cpp, cpu_sgemm_and_matmul.cpp, cross_engine_reorder.cpp, eltwise.cpp, getting_started.cpp, gpu_opencl_interop.cpp, inference_int8_matmul.cpp, inner_product.cpp, layer_normalization.cpp, logsoftmax.cpp, lrn.cpp, lstm.cpp, matmul.cpp, memory_format_propagation.cpp, performance_profiling.cpp, pooling.cpp, reorder.cpp, resampling.cpp, rnn_training_f32.cpp, shuffle.cpp, softmax.cpp, sum.cpp, and sycl_interop.cpp.

Member Typedef Documentation

◆ dims

typedef std::vector<dim> dnnl::memory::dims

Vector of dimensions.

Implementations are free to force a limit on the vector's length.

Examples
cpu_sgemm_and_matmul.cpp.

Member Enumeration Documentation

◆ data_type

Data type specification.

Enumerator
undef 

Undefined data type (used for empty memory descriptors).

f16 

16-bit/half-precision floating point.

bf16 

non-standard 16-bit floating point with 7-bit mantissa.

f32 

32-bit/single-precision floating point.

s32 

32-bit signed integer.

s8 

8-bit signed integer.

u8 

8-bit unsigned integer.

Examples
batch_normalization.cpp, binary.cpp, cnn_inference_f32.cpp, cnn_inference_int8.cpp, cnn_training_bf16.cpp, cnn_training_f32.cpp, concat.cpp, convolution.cpp, cpu_matmul_quantization.cpp, eltwise.cpp, inner_product.cpp, layer_normalization.cpp, logsoftmax.cpp, lrn.cpp, lstm.cpp, matmul.cpp, pooling.cpp, reorder.cpp, resampling.cpp, rnn_training_f32.cpp, shuffle.cpp, softmax.cpp, and sum.cpp.

◆ format_kind

Memory format kind.

Enumerator
undef 

Undefined memory format kind, used for empty memory descriptors.

any 

Unspecified format kind.

The primitive selects a format automatically.

blocked 

A tensor in a generic format described by the stride and blocking values in each dimension.

See dnnl_blocking_desc_t for more information.

wino 

Weights format used in 8-bit Winograd convolution.

packed 

Packed weights format used in RNN.

◆ format_tag

Memory format tag specification.

Memory format tags can be further divided into two categories:

  • Domain-agnostic names, i.e. names that do not depend on the tensor usage in the specific primitive. These names use letters from a to l to denote logical dimensions and form the order in which the dimensions are laid in memory. For example, dnnl::memory::format_tag::ab is used to denote a 2D tensor where the second logical dimension (denoted as b) is the innermost, i.e. has stride = 1, and the first logical dimension (a) is laid out in memory with stride equal to the size of the second dimension. On the other hand, dnnl::memory::format_tag::ba is the transposed version of the same tensor: the outermost dimension (a) becomes the innermost one.
  • Domain-specific names, i.e. names that make sense only in the context of a certain domain, such as CNN. These names are aliases to the corresponding domain-agnostic tags and used mostly for convenience. For example, dnnl::memory::format_tag::nc is used to denote 2D CNN activations tensor memory format, where the channels dimension is the innermost one and the batch dimension is the outermost one. Moreover, dnnl::memory::format_tag::nc is an alias for dnnl::memory::format_tag::ab, because for CNN primitives the logical dimensions of activations tensors come in order: batch, channels, spatial. In other words, batch corresponds to the first logical dimension (a), and channels correspond to the second one (b).

The following domain-specific notation applies to memory format tags:

  • 'n' denotes the mini-batch dimension
  • 'c' denotes a channels dimension
  • When there are multiple channel dimensions (for example, in convolution weights tensor), 'i' and 'o' denote dimensions of input and output channels
  • 'g' denotes a groups dimension for convolution weights
  • 'd', 'h', and 'w' denote spatial depth, height, and width respectively

See dnnl_format_tag_t for a detailed description.

Enumerator
undef 

Undefined memory format tag.

any 

Placeholder memory format tag.

Used to instruct the primitive to select a format automatically.

plain 1D tensor

ab 

plain 2D tensor

ba 

permuted 2D tensor

abc 

plain 3D tensor

acb 

permuted 3D tensor

bac 

permuted 3D tensor

bca 

permuted 3D tensor

cba 

permuted 3D tensor

abcd 

plain 4D tensor

abdc 

permuted 4D tensor

acdb 

permuted 4D tensor

bacd 

permuted 4D tensor

bcda 

permuted 4D tensor

cdba 

permuted 4D tensor

dcab 

permuted 4D tensor

abcde 

plain 5D tensor

abdec 

permuted 5D tensor

acbde 

permuted 5D tensor

acdeb 

permuted 5D tensor

bacde 

permuted 5D tensor

bcdea 

permuted 5D tensor

cdeba 

permuted 5D tensor

decab 

permuted 5D tensor

abcdef 

plain 6D tensor

acbdef 

plain 6D tensor

defcab 

plain 6D tensor

1D tensor; an alias for dnnl::memory::format_tag::a

nc 

2D CNN activations tensor; an alias for dnnl::memory::format_tag::ab

cn 

2D CNN activations tensor; an alias for dnnl::memory::format_tag::ba

tn 

2D RNN statistics tensor; an alias for dnnl::memory::format_tag::ab

nt 

2D RNN statistics tensor; an alias for dnnl::memory::format_tag::ba

ncw 

3D CNN activations tensor; an alias for dnnl::memory::format_tag::abc

nwc 

3D CNN activations tensor; an alias for dnnl::memory::format_tag::acb

nchw 

4D CNN activations tensor; an alias for dnnl::memory::format_tag::abcd

nhwc 

4D CNN activations tensor; an alias for dnnl::memory::format_tag::acdb

chwn 

4D CNN activations tensor; an alias for dnnl::memory::format_tag::bcda

ncdhw 

5D CNN activations tensor; an alias for dnnl::memory::format_tag::abcde

ndhwc 

5D CNN activations tensor; an alias for dnnl::memory::format_tag::acdeb

oi 

2D CNN weights tensor; an alias for dnnl::memory::format_tag::ab

io 

2D CNN weights tensor; an alias for dnnl::memory::format_tag::ba

oiw 

3D CNN weights tensor; an alias for dnnl::memory::format_tag::abc

owi 

3D CNN weights tensor; an alias for dnnl::memory::format_tag::acb

wio 

3D CNN weights tensor; an alias for dnnl::memory::format_tag::cba

iwo 

3D CNN weights tensor; an alias for dnnl::memory::format_tag::bca

oihw 

4D CNN weights tensor; an alias for dnnl::memory::format_tag::abcd

hwio 

4D CNN weights tensor; an alias for dnnl::memory::format_tag::cdba

ohwi 

4D CNN weights tensor; an alias for dnnl::memory::format_tag::acdb

ihwo 

4D CNN weights tensor; an alias for dnnl::memory::format_tag::bcda

iohw 

4D CNN weights tensor; an alias for dnnl::memory::format_tag::bacd

oidhw 

5D CNN weights tensor; an alias for dnnl::memory::format_tag::abcde

dhwio 

5D CNN weights tensor; an alias for dnnl::memory::format_tag::cdeba

odhwi 

5D CNN weights tensor; an alias for dnnl::memory::format_tag::acdeb

iodhw 

5D CNN weights tensor; an alias for dnnl::memory::format_tag::bacde

idhwo 

5D CNN weights tensor; an alias for dnnl::memory::format_tag::bcdea

goiw 

4D CNN weights tensor with groups; an alias for dnnl::memory::format_tag::abcd

wigo 

4D CNN weights tensor with groups; an alias for dnnl::memory::format_tag::dcab

goihw 

5D CNN weights tensor with groups; an alias for dnnl::memory::format_tag::abcde

hwigo 

5D CNN weights tensor with groups; an alias for dnnl::memory::format_tag::decab

giohw 

5D CNN weights tensor with groups; an alias for dnnl::memory::format_tag::acbde

goidhw 

6D CNN weights tensor with groups; an alias for dnnl::memory::format_tag::abcdef

giodhw 

6D CNN weights tensor with groups; an alias for dnnl::memory::format_tag::abcdef

dhwigo 

6D CNN weights tensor with groups; an alias for dnnl::memory::format_tag::defcab

tnc 

3D RNN data tensor in the format (seq_length, batch, input channels).

ntc 

3D RNN data tensor in the format (batch, seq_length, input channels).

ldnc 

4D RNN states tensor in the format (num_layers, num_directions, batch, state channels).

ldigo 

5D RNN weights tensor in the format (num_layers, num_directions, input_channels, num_gates, output_channels).

  • For LSTM cells, the gates order is input, forget, candidate and output gate.
  • For GRU cells, the gates order is update, reset and output gate.
ldgoi 

5D RNN weights tensor in the format (num_layers, num_directions, num_gates, output_channels, input_channels).

  • For LSTM cells, the gates order is input, forget, candidate and output gate.
  • For GRU cells, the gates order is update, reset and output gate.
ldio 

4D LSTM projection tensor in the format (num_layers, num_directions, num_channels_in_hidden_state, num_channels_in_recurrent_projection).

ldoi 

4D LSTM projection tensor in the format (num_layers, num_directions, num_channels_in_recurrent_projection, num_channels_in_hidden_state).

ldgo 

4D RNN bias tensor in the format (num_layers, num_directions, num_gates, output_channels).

  • For LSTM cells, the gates order is input, forget, candidate and output gate.
  • For GRU cells, the gates order is update, reset and output gate.
aBcd8b 

4D tensor blocked by 2nd dimension with block size 8

ABcd8b8a 

4D tensor blocked by 1st and 2nd dimension with block size 8

Examples
batch_normalization.cpp, binary.cpp, cnn_inference_f32.cpp, cnn_inference_int8.cpp, cnn_training_bf16.cpp, cnn_training_f32.cpp, concat.cpp, convolution.cpp, eltwise.cpp, inner_product.cpp, layer_normalization.cpp, logsoftmax.cpp, lrn.cpp, lstm.cpp, matmul.cpp, pooling.cpp, reorder.cpp, resampling.cpp, rnn_training_f32.cpp, shuffle.cpp, softmax.cpp, and sum.cpp.

Constructor & Destructor Documentation

◆ memory() [1/4]

dnnl::memory::memory ( )
default

Default constructor.

Constructs an empty memory object, which can be used to indicate absence of a parameter.

◆ memory() [2/4]

dnnl::memory::memory ( const desc md,
const engine engine,
void *  ahandle 
)
inline

Constructs a memory object.

Parameters
mdMemory descriptor.
engineEngine.
ahandlehandle.

◆ memory() [3/4]

template<typename T , int ndims = 1>
dnnl::memory::memory ( const desc md,
const engine aengine,
cl::sycl::buffer< T, ndims > &  buf 
)
inline

Constructs a memory object from a SYCL buffer.

Parameters
mdMemory descriptor.
aengineEngine to store the data on.
bufA SYCL buffer.

◆ memory() [4/4]

dnnl::memory::memory ( const desc md,
const engine aengine 
)
inline

Constructs a memory object.

The underlying storage for the memory will be allocated by the library.

Parameters
mdMemory descriptor.
aengineEngine to store the data on.

Member Function Documentation

◆ validate_dims()

template<typename T >
static void dnnl::memory::validate_dims ( const std::vector< T > &  v,
int  min_size = 0 
)
inlinestatic

Helper function that validates that an std::vector of dimensions can be safely converted to the C API array dnnl_dims_t.

Throws if validation fails.

Parameters
vVector of dimensions.
min_sizeMinimum expected size of the vector.

◆ get_data_handle()

void* dnnl::memory::get_data_handle ( ) const
inline

Returns the underlying memory buffer.

On the CPU engine, or when using USM, this is a pointer to the allocated memory.

◆ set_data_handle() [1/2]

void dnnl::memory::set_data_handle ( void *  handle,
const stream astream 
) const
inline

Sets the underlying memory buffer.

This function may write zero values to the memory specified by the handle if the memory object has a zero padding area. This may be time consuming and happens each time this function is called. The operation is always blocking and the stream parameter is a hint.

Note
The zero padding is required by memory objects created with blocked memory format tags like dnnl_aBcd8b when any of the dimensions is not a multiple of the corresponding block size. For "plain" formats like dnnl::memory::format_tag::nchw or dnnl::memory::format_tag::nhwc zero padding area needs to be set up explicitly when creating the corresponding memory descriptors. See Understanding Memory Formats for more details.
Even when the memory object is used to hold values that stay constant during the execution of the program (pre-packed weights during inference, for example), the function will still write zeroes to the padding area if it exists. Hence, the handle parameter cannot and does not have a const qualifier.
Parameters
handleMemory buffer to use as the underlying storage. On the CPU aengine or when USM is used, the data handle is a pointer to the actual data. For OpenCL it is a cl_mem.
astreamStream to use to execute padding in.

◆ set_data_handle() [2/2]

void dnnl::memory::set_data_handle ( void *  handle) const
inline

Sets the underlying memory buffer.

See documentation for dnnl::memory::set_data_handle(void *, const stream &) const for more information.

Parameters
handleMemory buffer to use as the underlying storage. For the CPU aengine, the data handle is a pointer to the actual data. For OpenCL it is a cl_mem.

◆ map_data()

template<typename T = void>
T* dnnl::memory::map_data ( ) const
inline

Maps a memory object and returns a host-side pointer to a memory buffer with a copy of its contents.

Mapping enables read/write directly from/to the memory contents for engines that do not support direct memory access.

Mapping is an exclusive operation - a memory object cannot be used in other operations until it is unmapped via memory::unmap_data() call.

Note
Any primitives working with the memory should be completed before the memory is mapped. Use stream::wait() to synchronize the corresponding execution stream.
The map_data and unmap_data functions are provided mainly for debug and testing purposes and their performance may be suboptimal.
Template Parameters
TData type to return a pointer to.
Returns
Pointer to the mapped memory.

◆ unmap_data()

void dnnl::memory::unmap_data ( void *  mapped_ptr) const
inline

Unmaps a memory object and writes back any changes made to the previously mapped memory buffer.

The pointer to the mapped buffer must be obtained via the dnnl::memory::map_data() call.

Note
The map_data and unmap_data functions are provided mainly for debug and testing purposes and their performance may be suboptimal.
Parameters
mapped_ptrA pointer previously returned by map_data().

◆ set_ocl_mem_object()

void dnnl::memory::set_ocl_mem_object ( cl_mem  mem_object)
inline

Sets the OpenCL memory object mem_object associated with the memory.

For behavioral details see memory::set_data_handle().

Parameters
mem_objectOpenCL cl_mem object to use as the underlying storage. It must have at least get_desc().get_size() bytes allocated.

◆ get_sycl_buffer()

template<typename T , int ndims = 1>
cl::sycl::buffer<T, ndims> dnnl::memory::get_sycl_buffer ( size_t *  offset = nullptr) const
inline

Returns the underlying SYCL buffer object.

Template Parameters
TType of the requested buffer.
ndimsNumber of dimensions of the requested buffer.
Parameters
offsetOffset within the returned buffer at which the memory object's data starts. Only meaningful for 1D buffers.

◆ set_sycl_buffer()

template<typename T , int ndims>
void dnnl::memory::set_sycl_buffer ( cl::sycl::buffer< T, ndims > &  buf)
inline

Sets the underlying buffer to the given SYCL buffer.

Template Parameters
TType of the buffer.
ndimsNumber of dimensions of the buffer.
Parameters
bufSYCL buffer.

The documentation for this struct was generated from the following file: