Singularity

bilby_pipe is a lightweight code for setting up parameter estmation for gravitational wave signals on the LIGO Data Grid clusters. The actual heavy lifting of running parameter is done by bilby. We provide a singularity container for bilby_pipe which has all the prerequisite software (e.g.., bilby, lalsuite, gwpy, and all supported bilby samplers.

A container can be thought of as a lightweight virtual machine which hosts the software. Using containers decreases the start-up difficulty and also aids in ensuring that results are reproducible since we can know the versions of all software used in the container.

singularity is installed on all LIGO Data Grid (LDG) clusters. To install it on your local machine, see the installation instructions.

Obtaining a singularity image

All the singularity containers are hosted on singularity-hub. Whilst working on one of the LDG clusters, you can “pull” a container:

$ singularity pull shub://lscsoft/bilby_pipe:dev

Here, shub:// indicates to singularity that you want to pull from singularity-hub, then the remaining path is the URI. The part of the URI following the colon is the “tag”. We intend to maintain two types of tags:

Release tags, e.g. :0.0.1, for each release of bilby_pipe a singularity image will be provided. The bilby version will be set by the latest bilby-PyPI version at the time of the release.
A development tag :tag which will be manually built from the master branch of bilby_pipe and bilby. Unfortunately, it is not possible yet to have this update automatically with all changes to master, although we will work on adding this in future.

Using a singularity image

After pulling the image, one should see something like this

$ singularity pull shub://lscsoft/bilby_pipe:dev
Progress |===================================| 100.0%
Done. Container is at: /home/gregory.ashton/lscsoft-bilby_pipe-master-dev.simg

The final line provides a path to where the container was downloaded too. The container is an executable version of bilby_pipe. To see what it can do, run

$ ./lscsoft-bilby_pipe-master-dev.simg --help
14:31 bilby_pipe INFO    : Running bilby_pipe version: 0.0.1: (CLEAN) 81118ee 2019-01-04 16:39:08 +1100
usage:
bilby_pipe is a command line tools for taking user input (as command line
arguments or an ini file) and creating DAG files for submitting bilby parameter
estimation jobs.

...

(note the use of ./ before the container path, this “executes” the container).

The container is equivalent to calling bilby_pipe. I.e., all of the commands given in Using bilby_pipe can equivalently be run instead by executing the container image and providing the neccersery command line arguments.

Note

You may wish to define an alias in your .bashrc such as

alias bilby_pipe='./PATH/TO/CONTAINER.simg

Use an absolute path to ensure bilby_pipe can be called from anywhere.

Building a custom container

Singularity images can be built in a number of ways. For full instructions, please refer to the official documentation.

The cleanest way to generate an image is using a container recipe. For an example, see the bilby_pipe build recipes (files starting with Singularity.

Unfortunately, singularity requires root access to build from a recipe. As such, it is not possible to build recipes on the LIGO Data Grid clusters. Instead, one can install a local copy of singularity. Alternatively one can use singularity-hub to set up automatic building of containers from a connected github account.