API Description

Overview

The DTG library provides a C API for creating and managing dynamic task graphs in MPI applications. Tasks are represented by the dtg_task_t structure, and users implement callback functions for task initialization, execution, and adaptation.

Data Structures

dtg_task_t

The task structure containing information about a task.

typedef struct {
    char task_id[256];
    char **argv;
    int argc;
    MPI_Group task_group;
    MPI_Comm task_comm;
    int task_rank;
    int task_size;
    int dynamic_process;
    MPI_Group task_group_add;
    MPI_Group task_group_sub;
    void * user_data;
    void * library_data;
} dtg_task_t;

task_id: Unique identifier for the task.
argv: Argument vector provided to the task.
argc: Number of arguments in argv.
task_group: MPI group associated with the task.
task_comm: MPI communicator for the task.
task_rank: Rank of the process within the task communicator.
task_size: Size of the task communicator.
dynamic_process: Flag indicating if the process is dynamic.
task_group_add: MPI group used for task expansion.
task_group_sub: MPI group used for task shrinking.
user_data: Pointer to user-defined data.
library_data: Pointer to library-internal data.

dtg_task_coupling_t

Information about a task coupling and its associated communicators.

typedef struct {
    char *name;
    char ** tasks;
    size_t n_tasks;
    size_t *sizes;
    MPI_Comm comm;
    MPI_Comm master_comm;
} dtg_task_coupling_t;

name: Name of the coupling.
tasks: Array of task identifiers in the coupling.
n_tasks: Number of tasks in the coupling.
sizes: Array of sizes for each task in the coupling.
comm: MPI communicator for the coupling.
master_comm: MPI communicator for the coupling master.

JSON Task Description

DTG task graphs may be described using JSON. The top-level object contains a graph entry with:

nodes: An array of task node descriptions.
edges: An array of directed edges connecting task nodes.
couplings: An object describing task couplings.

Example task JSON:

{
  "graph": {
    "nodes": [
      {
        "id": "test0",
        "title": "Sort File A",
        "lib": "/opt/hpc/build/benchmarks_dtg_asan/applications/build/libmpdata3d.so",
        "args": "test2 --size_l 64 --size_m 32 --size_n 16 --threads 1 --timestamps ./timestamps.csv --inhibitor 1 --iterations 10 --monitoring 0",
        "func": "task_func",
        "init": "init_func",
        "finalize": "finalize_func",
        "expand_send": "expand_send",
        "expand_recv": "expand_recv",
        "shrink_send": "shrink_send",
        "shrink_recv": "shrink_recv",
        "user_data": "user_data_ptr"
      }
    ],
    "edges": [],
    "couplings": {}
  }
}

Field descriptions:

id: Unique node identifier.
title: Human-readable task title.
lib: Path to the shared library containing the task implementation.
args: Command-line arguments passed to the task.
func: Name of the task execution function.
init: Name of the task initialization function.
finalize: Name of the task finalization function.
expand_send: Name of the expand send callback.
expand_recv: Name of the expand receive callback.
shrink_send: Name of the shrink send callback.
shrink_recv: Name of the shrink receive callback.
user_data: Optional opaque string passed through task metadata for user code.
edges: Task dependencies or communication edges.
couplings: Descriptions of task couplings; keys are coupling names and values are lists of tasks.

Functions

dtg_task_set_col

Set a key-value pair associated with a task.

int dtg_task_set_col(dtg_task_t *task, char *key, char *value);

Parameters: - task: Pointer to the task structure. - key: Key string. - value: Value string.

Returns: 0 on success, non-zero on error.

dtg_task_get_val

Get a value associated with a key from a task.

int dtg_task_get_val(dtg_task_t *task, char *key, char *buf, int size, int *flag);

Parameters: - task: Pointer to the task structure. - key: Key string. - buf: Buffer to store the value. - size: Size of the buffer. - flag: Pointer to an integer flag (output).

Returns: 0 on success, non-zero on error.

dtg_task_adapt

Adapt the task, potentially changing its size or terminating it.

int dtg_task_adapt(dtg_task_t *task, int *term, int *resized);

Parameters: - task: Pointer to the task structure. - term: Pointer to an integer flag indicating termination (output). - resized: Pointer to an integer flag indicating resizing (output).

Returns: 0 on success, non-zero on error.

dtg_task_get_coupling

Retrieve coupling information for a task.

dtg_task_coupling_t * dtg_task_get_coupling(dtg_task_t *task, char *coupling_name);

Parameters: - task: Pointer to the task structure. - coupling_name: Name of the coupling.

Returns: Pointer to dtg_task_coupling_t on success, or NULL on error.

User-Defined Functions

The library expects users to provide callback functions for various task lifecycle events.

typedef int (* dtg_user_init_func_t ) (dtg_task_t *task);
typedef int (* dtg_user_fini_func_t ) (dtg_task_t *task);
typedef int (* dtg_user_task_func_t ) (dtg_task_t * task);
typedef int (*dtg_user_expand_send_func_t) (dtg_task_t * task);
typedef int (*dtg_user_expand_recv_func_t) (dtg_task_t * task);
typedef int (*dtg_user_shrink_send_func_t) (dtg_task_t * task);
typedef int (*dtg_user_shrink_recv_func_t) (dtg_task_t * task);

dtg_user_init_func_t: Called when a task is initialized.
dtg_user_fini_func_t: Called when a task is finalized.
dtg_user_task_func_t: Called to execute the task.
dtg_user_expand_send_func_t: Called when expanding the task (sending data).
dtg_user_expand_recv_func_t: Called when expanding the task (receiving data).
dtg_user_shrink_send_func_t: Called when shrinking the task (sending data).
dtg_user_shrink_recv_func_t: Called when shrinking the task (receiving data).