Docker is an excellent containerisation system ideally suited to production servers.  It allows you to do one small thing but do it well.  For example, breaking a large blog up into individually maintained containers for a web-server, a database and (say) a wordpress instance. However due to inherent security woes, Docker doesn’t play nicely with multi-tenanted machines, the kind which are the bread and butter for researchers and HPC users.  That’s where Singularity steps in.   

Consider a system where Docker is installed.  This system has two users, Amanda and Bob.  Bob’s minding his own secretive business making files owned solely by him:

Amanda has nothing to do with Bob, however, Amanda is part of the docker group:

Amanda can pull down a docker image and due to the docker daemon running as root, Amanda (who isn’t a root user) can run commands as root.

Amanda.. you nosy sod.  You’ve mounted someone else’s home directory and can read, write and erase files not owned by you.

Singularity allows untrusted users to do untrusted things.

Singularity containers run as the user who started them.  These users never end up with any more rights than they had to begin with.  If they’re trusted to be on the system in the first place, they can’t be any more destructive or disruptive than if they didn’t have access to singularity.

Packaging your software

How many times have you struggled to get someone’s code to work and then given up proclaiming it to be bad/rubbish/unfriendly, just because the designer made assumptions about how your system would be set up on your system vs their system?  Everybody has that one specific system version of python, right?  With the 2.11.123.x version of the library the designer’s using?  Everyone has their home directory in /automount/homes/$USER, right?

Singularity containers provide a great way of distributing software.  They contain not just the code, but more importantly everything that code relies on.  This includes the entire development environment, its libraries and its environment variables.  It gives you the flexibility to encapsulate an OS just by pulling down the appropriate version.  For example if you’ve got something that runs on Ubuntu – Fine.  RHEL – No problem.  They all run under singularity.  More or less any flavour of linux can be containerised as containers hook into the underlying kernel, the utilities are just set dressing.  This has the limitation that windows containers don’t work under linux (and vice versa (at least, not without black magic)) but allows for an enormous amount of flexibility.

TL;DR

• Singularity is a container technology
• Singularity makes it appropriate for untrusted users to do untrusted things
• Singularity is nearly as fast as bare metal
• A Singularity image is a self-contained executable package of a
  piece of software and everything needed to run it, including
  dependencies and environment variables
• Singularity co-exists with the docker ecosystem

Getting started with singularity

Giant copy paste session to get going with singularity on an Ubuntu machine:

sudo apt-get update && sudo apt-get install -y \
build-essential \
uuid-dev \
libgpgme-dev \
squashfs-tools \
uuid-dev \
libseccomp-dev \
wget \
pkg-config \
git \
cryptsetup-bin \
uuid-dev 

export VERSION=1.13.6 OS=linux ARCH=amd64 && \
wget https://dl.google.com/go/go$VERSION.$OS-$ARCH.tar.gz && \
sudo tar -C /usr/local -xzvf go$VERSION.$OS-$ARCH.tar.gz && \
rm go$VERSION.$OS-$ARCH.tar.gz 

The more recent versions of singularity are written in go.  Go get go.  Change the version to match whatever’s the latest and greatest.

echo 'export GOPATH=${HOME}/go' >> ~/.bashrc && \
echo 'export PATH=/usr/local/go/bin:${PATH}:${GOPATH}/bin' >> ~/.bashrc && \
source ~/.bashrc