Docker Cluster

The Docker Cluster project provides a local, distributed environment for development and functional testing of the DynRes Software Stack. It creates a docker-swarm of containers connected via a common docker network.

Getting Started

The following steps are required to set up the Docker Cluster project on your system

Setting up Docker Swarm

Follow the Docker installation instructions for your Linux distribution. For example, Ubuntu: https://docs.docker.com/engine/install/ubuntu/

It is recommended to use your own regular user, and not root. Add your user to the docker group: ` usermod -aG docker $USER `

Initialize the swarm cluster:

` docker swarm init `

Setting up Docker Cluster

Clone the Docker Cluster repository including its submodules:

` git clone --recursive https://gitlab.inria.fr/dynres/dyn-procs/docker-cluster.git `

Build the docker image:

` ./docker-build.sh [OPTIONS] `

Note

If you just want to get started with development, running the command without options will mostly likely be sufficient. The resulting Docker image will be named docker-cluster:default-latest-e-i-v-l-develop

Options:

```

–docker_file=[name]: The docker file to build. You can create your own Dockerfile by copying and adapting Dockerfile.default (default=Dockerfile.default)
–image_version=[name]: Package versions to build. See export directory for available versions. (default=latest (pulls most recent package versions))
–export_version=[name]: Name to assign to exported version If not None, this will record the version installed during the build and export them to export/[name].txt You can later use this file with the –image_version option (default=None)
–mode=[develop/release]: When using develop mode, the build and install directories will be mounted into the docker containers at /opt/hpc/[build/install]. (default=develop)
–env_install=[env_name]: Release Mode only: DynPkgs Environment to install (default=None)
–env_load=[env_name]: Release Mode only: DynPkgs Environment to load (default=None)
–env_version=[env_version]: Release Mode only: DynPkgs Environment version (default=None)

```

Using the Docker Cluster

Start, drop into and shutdown the cluster.

Starting the Docker Cluster

Starts N containers named n1, …, nN

` ./start.sh [OPTIONS] `

Options:

```

-n | –num NUM: Number of nodes to start on this host (recommended: 9)
-i | –image NAME: Name of the container image (Default: docker-cluster:default-latest-e-i-v-l-develop)

```

Dropping into the Docker Cluster

Note

Use the DynPkgs package manager to manage the DynRes Software Stack inside of the Docker Cluster

Drop into a container of the running cluster:

` ./drop-in.sh [OPTIONS] `

Options:

```

–user=[username]: Drop in as user [user] (default=mpiuser)”
–host=[hostname]: Drop into host [hostname] (default=n1)”

```

Note

mpiuser (default) is in the sudo group. It is usually not necessary/recommended to drop in as root

Installing Software

The Docker Cluster comes with a preinstalled instance of the DynPkgs package manager to install software packages of the DynRes software stack.

General software can installed using dnf install, pip3 install or manual installation.

Shutting down the Docker Cluster

Stop and remove all conatainers of the docker cluster. Note that only data in /opt/hpc/build and /opt/hpc/install will persist (when using develop mode).

` ./tmp/shutdown-cluster.sh `