Server Setup

This page documents the installation and configuration of the rucio server and master database at:

With operating system:

    Scientific Linux release 7.6 (Nitrogen)

Install yum dependencies

Note: a Dockerfile with the recipe to install the server dependencies can be found here.


yum update

Enable additional repositories:

rpm -ivh 
rpm -ivh
rpm -Uvh

Update repository metadata

yum clean all
yum makecache

Yum-based dependencies and useful tools:

yum install -y git \
    vim \
    bzip2 \
    kernel-devel \
    gcc \
    make \
    perl \
    python-devel \
    python-psycopg2 \
    osg-pki-tools \
    fetch-crl \
    memcached \
    postgresql10-server \
    postgresql10 \
    gfal2 \
    gfal2-python \
    gfal2-util \
    gfal2-all \
    globus-proxy-utils \
    voms-clients-cpp \
    httpd-devel \
    openssl-devel \
    mod_wsgi \
    mod_auth_kerb \

pip dependencies

Do not install pip from RPM! Instead, get it from:

curl -O

python2-pip demands to install python-ipaddress which later conflicts with the pip installation of rucio itself.

Install all possible pip-packages required by rucio:

curl -O
pip install -r requirements.readthedocs.txt supervisor

Finally, install ligo branch of rucio:

pip install git+

Set up logging directories:

mkdir /var/log/rucio /var/log/rucio/trace
chmod -R 777 /var/log/rucio

Make RESTful scripts executable:

chmod +x /usr/lib/python2.7/site-packages/rucio/web/rest/webpy/v1/*.py

Additional directory structures and contents:

ln -sf /usr/rucio /opt/rucio
mkdir /opt/rucio/lib
ln -sf /usr/lib/python2.7/site-packages/rucio /opt/rucio/lib/rucio

Clone the source repository for some supplementary files:

git clone

Copy the wsgi aliases from the source repository into place:

cp ~james.clark/rucio/etc/web/aliases-py27.conf /opt/rucio/etc/web

And edit /opt/rucio/etc/web/aliases-py27.conf to point to the correct locations. E.g.,:


Should be:


Service Configuration

Enable/start services:

systemctl enable memcached
systemctl start memcached
systemctl enable fetch-crl-boot
systemctl start fetch-crl-boot
systemctl enable fetch-crl-cron
systemctl start fetch-crl-cron

Apache Configuration

This step assumes the host is called with X509 certificate bundle:

ls /etc/httpd/x509-certs/

Download the Apache configuration into place:

curl -o /etc/httpd/conf.d/rucio.conf

Alternatively: download rucio.conf manually and copy to /etc/httpd/conf.d/rucio.conf

Make sure the following lines are commented out in /etc/httpd/conf/httpd.conf

#<Directory />
#    AllowOverride none
#    Require all denied

Set SELinux to permissive:

setenforce permissive

Start up apache:

systemctl enable httpd
systemctl start httpd

PostgreSQL Configuration


  1. Initialise database

  2. Open database authentication method for initial database creation

  3. Create user and database for rucio

  4. Finalise database access authentication

Initialise data directory:

/usr/pgsql-10/bin/postgresql-10-setup initdb

Edit /var/lib/pgsql/10/data/pg_hba.conf to modify database authentication method for initial database creation. Change /var/lib/pgsql/10/data/pg_hba.conf from:

local   all       all            peer


local   all       all            trust

Can be done by:

curl -o /var/lib/pgsql/10/data/pg_hba.conf

Start database:

systemctl restart postgresql-10

Create user and database for rucio:

psql -U postgres

postgres=# create role rucio password 'rucio' superuser inherit login;
postgres=# create database rucio owner rucio encoding 'utf8';
postgres=# grant all on database rucio to rucio;

Set authentication method to md5 and add a new host:

  • change method for everything to md5

  • Add new line: host  all  all  md5

E.g., download /var/lib/pgsql/10/data/pg_hba.conf in place:

curl -o /var/lib/pgsql/10/data/pg_hba.conf

Set addresses and port to listen to in /var/lib/pgsql/10/data/postgresql.conf:

  • listen_addresses = '*'

  • port = 5432 Equivalently:

curl -o /var/lib/pgsql/10/data/postgresql.conf

Now enable and start or restart the postgreql server:

systemctl enable postgresql-10
systemctl start postgresql-10

Rucio Configuration

Install rucio configuration files from gwrucio repository:

curl -o /opt/rucio/etc/rucio.cfg
curl -o /opt/rucio/alembic.ini

Important Note: when first configuring the rucio server, ensure that the rucio.cfg is set to use userpass authentication with the original ddmlab user:

auth_type = userpass
username = ddmlab
password = secret

Test the RESTful interface

curl -k -X GET
{"version": "1.13.1-2549-g78ef266-dev1555615471"}

Initialise the Rucio database

Manually copy the SQLAlchemy versions directory from the source repository:

cp -r ~james.clark/rucio/lib/rucio/db/sqla/migrate_repo/versions /opt/rucio/lib/rucio/db/sqla/migrate_repo/

Initialise the rucio database:

[root@rucio rucio]# cd /opt/rucio
[root@rucio rucio]# python tools/
INFO  [alembic.runtime.migration] Context impl PostgresqlImpl.
INFO  [alembic.runtime.migration] Will assume transactional DDL.
INFO  [alembic.runtime.migration] Running stamp_revision  -> 3345511706b8

Add Access Control List

Add a grid access control list file in /opt/rucio/etc/gacl:

curl -o /opt/rucio/etc/gacl

Root Account Setup

Full details of account setup can be found in rucio administration

These instructions assume an IGTF X509 host certificate has been installed on the server.

Creating the database adds a root account with default username/password. Add an X509 identity and then change that password ASAP. Add the host certificate X509 identity to the root account:

openssl x509 -in /etc/grid-security/hostcert.pem -noout -subject
subject= /DC=org/DC=incommon/C=US/ST=CA/L=Pasadena/O=California Institute of Technology/OU=Laser Interferometer Gravitational-Wave Observatory/
[root@rucio rucio]# rucio-admin identity add --account root --type X509 --id "/DC=org/DC=incommon/C=US/ST=CA/L=Pasadena/O=California Institute of Technology/OU=Laser Interferometer Gravitational-Wave Observatory/" --email
Added new identity to account: /DC=org/DC=incommon/C=US/ST=CA/L=Pasadena/O=California Institute of Technology/OU=Laser Interferometer Gravitational-Wave Observatory/

Configure the client on the server to use the server’s host x509 certificate. Edit /opt/rucio/etc/rucio.cfg:

;auth_type = userpass
username = ddmlab
password = secret
auth_type = x509
client_cert = /etc/grid-security/hostcert.pem
client_key = /etc/grid-security/hostkey.pem

Delete default identities

rucio-admin identity delete --account root --type GSS --id "ddmlab@CERN.CH"
rucio-admin identity delete --account root --type X509 --id "/C=CH/ST=Geneva/O=CERN/OU=PH-ADP-CO/CN=DDMLAB Client Certificate/"
rucio-admin identity delete --account root --type USERPASS --id "ddmlab"
rucio-admin identity delete --account root --type SSH --id "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAq5LySllrQFpPL614sulXQ7wnIr1aGhGtl8b+HCB/0FhMSMTHwSjX78UbfqEorZV16rXrWPgUpvcbp2hqctw6eCbxwqcgu3uGWaeS5A0iWRw7oXUh6ydnVy89zGzX1FJFFDZ+AgiZ3ytp55tg1bjqqhK1OSC0pJxdNe878TRVVo5MLI0S/rZY2UovCSGFaQG2iLj14wz/YqI7NFMUuJFR4e6xmNsOP7fCZ4bGMsmnhR0GmY0dWYTupNiP5WdYXAfKExlnvFLTlDI5Mgh4Z11NraQ8pv4YE1woolYpqOc/IMMBBXFniTT4tC7cgikxWb9ZmFe+r4t6yCDpX4IL8L5GOQ== ddmlab"

You may also add an X509 identity to the root account for remote administration. E.g.:

rucio-admin identity add --account root --type X509 \
    --id "/DC=org/DC=XXXXXXX/C=XX/O=XXXX/CN=Albert Einstein" \

In your local client configuration, change to x509_proxy authentication in the ${RUCIO_HOME}/etc/rucio.cfg

Daemon Configuration (SupervisorD)

Rucio daemons are controlled with supervisord. These directions assume supervisor has been installed via pip (yum installs supervisor dependencies which conflict with other pip packages).

Copy the supervisor configuration file into place:

curl -o /etc/supervisord.conf

Edit inet server and username/password (if you want the webui). Set an appropriate username/password in::

[inet_http_server]         ; inet (TCP) server disabled by default        ; (ip_address:port specifier, *:port for all iface)
username=user              ; (default is no username (open server))
password=123               ; (default is no password (open server))

Create log and run directories:

mkdir /var/log/supervisor
mkdir /var/run/supervisor

Add the rucio daemon configuration file:

mkdir /etc/supervisord.d
curl -o /etc/supervisord.d/rucio-daemons.ini

Check supervisord runs ok by launching in foreground:

supervisord --nodaemon

Starting supervisord at boot

The pip-installed supervisor does not provide init scripts to run supervisor under systemd. However, the supervisor documentation refers to a github repository where init scripts are provided. Download and install the CentOS script:

curl -o  /usr/lib/systemd/system/supervisord.service

We should now be able to enable and start supervisord through systemctl

systemctl enable supervisord
Created symlink from /etc/systemd/system/ to /usr/lib/systemd/system/supervisord.service.
systemctl start supervisord

Proxy Delegation For FTS

Third party transfers are performed via an FTS server. To authenticate 3rd party transfers:

  • Map the rucio server host identity to an account at each end point.

  • Create an X509 proxy certificate from the rucio server host certificate.

  • Delegate the proxy to the FTS server for authentication at each end point.

Note: the host certificate used here must be from a CA recognised by all sites in the rucio distribution network. We have found that LIGO robot certificates are not recognised by EGI sites and we need to use an IGTF-issued certificate, with a subject which follows the IGTF specifications (e.g., no postcode!).

Valid proxy creation and delegation is performed via a cronjob which executes a simple script with voms-proxy-init and fts-rest-delegate.

The script can be found here and downloaded / installed with

    curl -o /opt/rucio/tools/fts_proxy

Edit the crontab to refresh the proxy and delegation at 6am and 6pm:

0 6,18 * * * /bin/bash /opt/rucio/tools/fts_proxy > /dev/null 2>&1

Support for EGI Sites

To authenticate with EGI sites, we need to register our certificate with the Virgo VOMs:

Create a pkcs12 certificate bundle for import into a browser:

openssl pkcs12 -export -out rucio-bundle.pfx -inkey /etc/grid-security/hostkey.pem -in /etc/grid-security/hostcert.pem

Download and import to a browser and navigate to: request to join the Virgo VO.

Finally, add the following files to /etc/vomses:

mkdir /etc/vomses
curl -o /etc/vomses/
curl -o /etc/vomses/