File buffer_collectives.h¶
Defines
-
GA_COMM_ID_BYTES¶
sizeof(gpucommCliqueId)
Typedefs
-
typedef struct _gpucomm gpucomm¶
Enums
Functions
-
int gpucomm_new(gpucomm **comm, gpucontext *ctx, gpucommCliqueId comm_id, int ndev, int rank)¶
Create a new gpu communicator instance.
This must be called in parallel by all participants in the same world. The call will block until all participants have joined in. The world is defined by a shared comm_id.
- Parameters
comm – pointer to get a new gpu communicator
ctx – gpu context in which
comm
will be used (contains device information)comm_id – id unique to communicators consisting a world
ndev – number of communicators/devices participating in the world
rank – user-defined rank, from 0 to
ndev
-1. Must be unique for the world.
- Returns
error code or GA_NO_ERROR if success
-
void gpucomm_free(gpucomm *comm)¶
Destroy a gpu communicator instance.
- Parameters
comm – gpu communicator to be destroyed
-
const char *gpucomm_error(gpucontext *ctx)¶
Returns nice error message concerning GA_COMM_ERROR.
- Parameters
ctx – gpu context in which communicator was used
- Returns
useful backend error message
-
gpucontext *gpucomm_context(gpucomm *comm)¶
Returns gpu context in which
comm
is used.- Parameters
comm – gpu communicator
- Returns
gpu context
-
int gpucomm_gen_clique_id(gpucontext *ctx, gpucommCliqueId *comm_id)¶
Creates a unique
comm_id
.The id is guarenteed to be unique in the same host, but not necessarily across hosts.
- Parameters
ctx – gpu context
comm_id – pointer to instance containing id
- Returns
error code or GA_NO_ERROR if success
-
int gpucomm_get_count(gpucomm *comm, int *devcount)¶
Returns total number of devices participating in
comm
’s world.- Parameters
comm – gpu communicator
devcount – pointer to store the number of devices
- Returns
error code or GA_NO_ERROR if success
-
int gpucomm_get_rank(gpucomm *comm, int *rank)¶
Returns the rank of
comm
inside its world.- Parameters
comm – gpu communicator
rank – pointer to store the rank
- Returns
error code or GA_NO_ERROR if success
-
int gpucomm_reduce(gpudata *src, size_t offsrc, gpudata *dest, size_t offdest, size_t count, int typecode, int opcode, int root, gpucomm *comm)¶
Reduce collective operation for ranks in a communicator world [buffer level].
Note
Non root ranks can call this, using a NULL
dest
. In this case,offdest
will not be used.Note
Must be called separately for each rank in
comm
.- Parameters
src – data in device’s buffer to be reduced
offsrc – memory offset after which data is saved in buffer
src
dest – data in device’s buffer to collect result
offdest – memory offset after which data will be saved in buffer
dest
count – number of elements to be reduced in each array
typecode – elements’ data type
opcode – reduce operation code
root – rank in
comm
which will collect resultcomm – gpu communicator
- Returns
error code or GA_NO_ERROR if success
-
int gpucomm_all_reduce(gpudata *src, size_t offsrc, gpudata *dest, size_t offdest, size_t count, int typecode, int opcode, gpucomm *comm)¶
AllReduce collective operation for ranks in a communicator world [buffer level].
Reduces data pointed by
src
using op operation and leaves identical copies of result in data pointed bydest
on each rank ofcomm
.Note
Must be called separately for each rank in
comm
.- Parameters
src – data in device’s buffer to be reduced
offsrc – memory offset after which data is saved in buffer
src
dest – data in device’s buffer to collect result
offdest – memory offset after which data will be saved in buffer
dest
count – number of elements to be reduced in each array
typecode – elements’ data type
opcode – reduce operation code (see gpucomm_reduce_ops)
comm – gpu communicator
- Returns
error code or GA_NO_ERROR if success
-
int gpucomm_reduce_scatter(gpudata *src, size_t offsrc, gpudata *dest, size_t offdest, size_t count, int typecode, int opcode, gpucomm *comm)¶
ReduceScatter collective operation for ranks in a communicator world [buffer level].
Reduces data pointed by
src
usingopcode
operation and leaves reduced result scattered over data pointed bydest
in the user-defined rank order incomm
.Note
Must be called separately for each rank in
comm
.- Parameters
src – data in device’s buffer to be reduced
offsrc – memory offset after which data is saved in buffer
src
dest – data in device’s buffer to collect scattered result
offdest – memory offset after which data will be saved in buffer
dest
count – number of elements to be contained in result
dest
typecode – elements’ data type
opcode – reduce operation code (see gpucomm_reduce_ops)
comm – gpu communicator
- Returns
error code or GA_NO_ERROR if success
-
int gpucomm_broadcast(gpudata *array, size_t offset, size_t count, int typecode, int root, gpucomm *comm)¶
Broadcast collective operation for ranks in a communicator world [buffer level].
Copies data pointed by
array
to all ranks incomm
.Note
Must be called separately for each rank in
comm
.- Parameters
array – data in device’s buffer to get copied or be received
offset – memory offset after which data in
array
begincount – number of elements to be contained in
array
typecode – elements’ data type
root – rank in
comm
which broadcasts its arraycomm – gpu communicator
- Returns
error code or GA_NO_ERROR if success
-
int gpucomm_all_gather(gpudata *src, size_t offsrc, gpudata *dest, size_t offdest, size_t count, int typecode, gpucomm *comm)¶
AllGather collective operation for ranks in a communicator world.
Each rank receives all data pointed by
src
of every rank in the user-defined rank order incomm
.Note
Must be called separately for each rank in
comm
.- Parameters
src – data in device’s buffer to be gathered
offsrc – memory offset after which data in
src
begindest – data in device’s buffer to gather from all ranks
offdest – memory offset after which data in
dest
begincount – number of elements to be gathered from each rank in
src
typecode – elements’ data type
comm – gpu communicator
- Returns
error code or GA_NO_ERROR if success
-
struct gpucommCliqueId¶
- #include <buffer_collectives.h>
Dummy struct to define byte-array’s length through a type
Public Members
-
char internal[GA_COMM_ID_BYTES]¶
-
char internal[GA_COMM_ID_BYTES]¶