1. Installation

This section describes how to build and install Charliecloud. For some distributions, this can be done using your package manager; otherwise, both normal users and admins can build and install it manually.

Warning

If you are installing on a Cray and have not applied the patch for Cray case #188073, you must use the cray branch to avoid crashing nodes during job completion. This is a Cray bug that Charliecloud happens to tickle. Non-Cray build boxes and others at the same site can still use the master branch.

Note

These are general installation instructions. If you’d like specific, step-by-step directions for CentOS 7, section VirtualBox appliance has these for a VirtualBox virtual machine.

1.1. Prequisites

Charliecloud is a simple system with limited prerequisites. If your system meets these prerequisites but Charliecloud doesn’t work, please report that as a bug.

1.1.1. Run time

Systems used for running images in the standard unprivileged mode need:

  • Recent Linux kernel with CONFIG_USER_NS=y. We recommend version 4.4 or higher.
  • C compiler and standard library
  • POSIX shell and utilities

Some distributions need configuration changes to enable user namespaces. For example, Debian Stretch needs sysctl kernel.unprivileged_userns_clone=1, and RHEL and CentOS 7.4 need both a kernel command line option and a sysctl (that put you into “technology preview”).

Note

An experimental setuid mode is also provided that does not need user namespaces. This should run on most currently supported Linux distributions.

1.1.2. Build time

Systems used for building images need the run-time prerequisites, plus:

  • Bash 4.1+

and optionally:

  • Docker 17.03+
  • internet access or Docker configured for a local Docker hub
  • root access using sudo

Older versions of Docker may work but are untested. We know that 1.7.1 does not work.

1.1.3. Test suite

In order to run the test suite on a run or build system (you can test each mode independently), you also need:

  • Bash 4.1+
  • Python 2.6+
  • Bats 0.4.0
  • wget

Note that without Docker on the build system, some of the test suite will be skipped.

Bats can be installed at the system level or embedded in the Charliecloud source code. If it’s in both places, the latter is used.

To embed Bats, either:

  • Download Charliecloud using git clone --recursive, which will check out Bats as a submodule in test/bats.
  • Unpack the Bats zip file or tarball in test/bats.

To check an embedded Bats:

$ test/bats/bin/bats --version
Bats 0.4.0

1.2. Package manager install

Charliecloud is available in some distribution package repositories, and packages can be built for additional distributions. (Note, however, that system-wide installation is not required — Charliecloud works fine when built by any user and run from one’s home directory or similar.)

This section describes how to obtain packages for the distributions we know about, and where to go for support on them.

If you’d like to build one of the packages, or if you’re a package maintainer, see packaging/README and packaging/*/README for additional documentation.

Pull requests and other collaboration to improve the packaging situation are particularly welcome!

1.2.1. Debian

Charliecloud has been proposed for inclusion in Debian; see issue 95.

Distribution versions proposed for Buster and Stretch backports
Maintainers Lucas Nussbaum (lucas@debian.org) and Peter Wienemann (wienemann@physik.uni-bonn.de)
Bug reports to Charliecloud’s GitHub issue tracker
Packaging source code in Charliecloud: packaging/debian

1.2.2. Gentoo

A native package for Gentoo is available.

Package name sys-cluster/charliecloud
Maintainer Oliver Freyermuth (o.freyermuth@googlemail.com)
Bug reports to Gentoo Bugzilla
Packaging source code Gentoo ebuild repository

To install:

$ emerge sys-cluster/charliecloud

If may necessary to accept keywords first, e.g.:

$ echo "=sys-cluster/charliecloud-0.2.3_pre20171121 ~amd64" >> /etc/portage/package.accept_keywords

A live ebuild is also available and can be keyworded via:

$ echo "~sys-cluster/charliecloud-9999 \*\*" >> /etc/portage/package.accept_keywords

1.2.3. RPM-based distributions

An RPM .spec file is provided in the Charliecloud source code. We are actively seeking distribution packagers to adapt this into official packages!

Repositories none yet
Maintainer Oliver Freyermuth (o.freyermuth@googlemail.com)
Bug reports to Charliecloud’s GitHub issue tracker
Packaging source code in Charliecloud: packaging/redhat

1.3. Manual build and install

1.3.1. Download

See our GitHub project: https://github.com/hpc/charliecloud

The recommended download method is git clone --recursive.

1.3.2. Build

To build in the standard, unprivileged mode (recommended):

$ make

To build in setuid mode (for testing if your kernel doesn’t support the user namespace):

$ make SETUID=yes

To build the documentation, see doc-src/README.

Warning

Do not build as root. This is unsupported and may introduce security problems.

1.3.3. Install (optional)

You can run Charliecloud from the source directory, and it’s recommended you at least run the test suite before installation to establish that your system will work.

To install (FHS-compliant):

$ make install PREFIX=/foo/bar

Note that PREFIX is required. It does not default to /usr/local like many packages.

1.4. Docker tips

Docker is a convenient way to build Charliecloud images. While installing Docker is beyond the scope of this documentation, here are a few tips.

1.4.1. Understand the security implications of Docker

Because Docker (a) makes installing random crap from the internet really easy and (b) is easy to deploy insecurely, you should take care. Some of the implications are below. This list should not be considered comprehensive nor a substitute for appropriate expertise; adhere to your moral and institutional responsibilities.

1.4.1.1. docker equals root

Anyone who can run the docker command or interact with the Docker daemon can trivially escalate to root. This is considered a feature.

For this reason, don’t create the docker group, as this will allow passwordless, unlogged escalation for anyone in the group.

1.4.1.2. Images can contain bad stuff

Standard hygiene for “installing stuff from the internet” applies. Only work with images you trust. The official Docker Hub repositories can help.

1.4.1.3. Containers run as root

By default, Docker runs container processes as root. In addition to being poor hygiene, this can be an escalation path, e.g. if you bind-mount host directories.

1.4.1.4. Docker alters your network configuration

To see what it did:

$ ifconfig    # note docker0 interface
$ brctl show  # note docker0 bridge
$ route -n

1.4.1.5. Docker installs services

If you don’t want the service starting automatically at boot, e.g.:

$ systemctl is-enabled docker
enabled
$ systemctl disable docker
$ systemctl is-enabled docker
disabled

1.4.2. Configuring for a proxy

By default, Docker does not work if you have a proxy, and it fails in two different ways.

The first problem is that Docker itself must be told to use a proxy. This manifests as:

$ sudo docker run hello-world
Unable to find image 'hello-world:latest' locally
Pulling repository hello-world
Get https://index.docker.io/v1/repositories/library/hello-world/images: dial tcp 54.152.161.54:443: connection refused

If you have a systemd system, the Docker documentation explains how to configure this. If you don’t have a systemd system, then /etc/default/docker might be the place to go?

The second problem is that Docker containers need to know about the proxy as well. This manifests as images failing to build because they can’t download stuff from the internet.

The fix is to set the proxy variables in your environment, e.g.:

export HTTP_PROXY=http://proxy.example.com:8088
export http_proxy=$HTTP_PROXY
export HTTPS_PROXY=$HTTP_PROXY
export https_proxy=$HTTP_PROXY
export ALL_PROXY=$HTTP_PROXY
export all_proxy=$HTTP_PROXY
export NO_PROXY='localhost,127.0.0.1,.example.com'
export no_proxy=$NO_PROXY

You also need to teach sudo to retain them. Add the following to /etc/sudoers:

Defaults env_keep+="HTTP_PROXY http_proxy HTTPS_PROXY https_proxy ALL_PROXY all_proxy NO_PROXY no_proxy"

Because different programs use different subsets of these variables, and to avoid a situation where some things work and others don’t, the Charliecloud test suite (see below) includes a test that fails if some but not all of the above variables are set.

1.5. Running the tests

Charliecloud comes with a fairly comprehensive Bats test suite, in test. Go there:

$ cd test

To check location and version of Bats used by the tests:

$ make where-bats
which bats
/usr/bin/bats
bats --version
Bats 0.4.0

Just like for normal use, the Charliecloud test suite is split into build and run phases, and there is an additional phase that runs the examples’ test suites. These phases can be tested independently on different systems.

Testing is coordinated by make. The test targets run one or more test suites. If any test suite has a failure, testing stops with an error message.

The tests need three work directories with several gigabytes of free space, in order to store image tarballs, unpacked image directories, and permission test fixtures. These are configured with environment variables:

$ export CH_TEST_TARDIR=/var/tmp/tarballs
$ export CH_TEST_IMGDIR=/var/tmp/images
$ export CH_TEST_PERMDIRS='/var/tmp /tmp'

CH_TEST_PERMDIRS can be set to skip in order to skip the file permissions tests.

(Strictly speaking, the build phase needs only the first, and the example test phase does not need the last one. However, for simplicity, the tests will demand all three for all phases.)

Note

Bats will wait until all descendant processes finish before exiting, so if you get into a failure mode where a test suite doesn’t clean up all its processes, Bats will hang.

1.5.1. Build

In this phase, image building and associated functionality is tested.

$ make test-build
bats build.bats build_auto.bats build_post.bats
 ✓ create tarball directory if needed
 ✓ documentations build
 ✓ executables seem sane
[...]
 ✓ ch-build obspy
 ✓ ch-docker2tar obspy
 ✓ docker pull dockerpull
 ✓ ch-docker2tar dockerpull
 ✓ nothing unexpected in tarball directory

41 tests, 0 failures

Note that with an empty Docker cache, this test can be quite lengthy, half an hour or more, because it builds all the examples as well as several basic Dockerfiles for common Linux distributions and tools (in test). With a full cache, expect more like 1–2 minutes.

Note

The easiest way to update the Docker images used in this test is to simply delete all Docker containers and images, and let them be rebuilt:

$ sudo docker rm $(sudo docker ps -aq)
$ sudo docker rmi -f $(sudo docker images -q)

1.5.2. Run

The run tests require the contents of $CH_TEST_TARDIR produced by a successful, complete build test. Copy this directory to the run system.

Additionally, the user running the tests needs to be a member of at least 2 groups.

File permission enforcement is tested against specially constructed fixture directories. These should include every meaningful mounted filesystem, and they cannot be shared between different users. To create them:

$ for d in $CH_TEST_PERMDIRS; do sudo ./make-perms-test $d $USER nobody; done

To skip this test (e.g., if you don’t have root), set $CH_TEST_PERMDIRS to skip.

To run the tests:

$ make test-run

1.5.3. Examples

Some of the examples include test suites of their own. This Charliecloud runs those test suites, using a Slurm allocation if one is available or a single node (localhost) if not.

These require that the run tests have been completed successfully.

Note that this test can take quite a while, and that single tests from the Charliecloud perspective include entire test suites from the example’s perspective, so be patient.

To run the tests:

$ make test-test

1.5.4. Quick and multiple-phase tests

We also provide the following additional test targets:

  • test-quick: key subset of build and run phases (nice for development)
  • test: build and run phases
  • test-all: all three phases

We recommend that a build box pass all phases so it can be used to run containers for testing and development.