Rucio Storage Elements (RSEs)

Data managed by rucio resides in rucio storage elements (RSEs). An RSE is defined by a series of settings for how rucio behaves, Attributes which help determine transfer and replication policies and protocols which determine the protocols and servers available for transfers at each RSE.

For example, the current LIGO-CIT RSE is defined:

$ rucio-admin rse info LIGO-CIT
  third_party_copy_protocol: 1
  rse_type: DISK
  domain: [u'lan', u'wan']
  availability_delete: True
  delete_protocol: 1
  rse: LIGO-CIT
  deterministic: True
  write_protocol: 1
  read_protocol: 1
  staging_area: False
  credentials: None
  availability_write: True
  lfn2pfn_algorithm: ligo
  availability_read: True
  volatile: False
  verify_checksum: True
  id: 98a8a2a1621e453aa164fce640b123b8
  HDFS: False
  ALL: True
  LIGO_LAB: True
  site: CIT
  LIGO-CIT: True
  ARCHIVE: False
  istape: False
    extended_attributes: None
    prefix: /mnt/rucio
    domains: {u'wan': {u'read': 1, u'write': 1, u'third_party_copy': 1, u'delete': 1}, u'lan': {u'read': 0, u'write': 0, u'delete': 0}}
    scheme: gsiftp
    port: 2811
    impl: rucio.rse.protocols.gfal.Default
    files: 3
    used: 530367545
    rse: LIGO-CIT
    updated_at: 2018-12-10 21:56:56
    free: None
    source: rucio
    total: 530367545

Create RSE

Create the RSE and define a copying protocol:

$ rucio-admin rse add LIGO-WA-ARCHIVE
Added new deterministic RSE: LIGO-WA-ARCHIVE
$ rucio-admin rse add-protocol\
    --prefix /archive/frames \
    --domain-json '{"wan": {"read": 1, "write": 0, "delete": 0, "third_party_copy": 1}}' \
    --scheme gsiftp \
    --port 2811 \
    --impl rucio.rse.protocols.gfal.Default \

Note: since this is still a testing phase, we have deliberately set this to a read-only RSE. In this particular case, we also need to set the availability_delete setting. At the time, there is no CLI tool for this, so descend into python and use the API directly:

>>> from rucio.client.rseclient import RSEClient
>>> client = RSEClient()
>>> client.update_rse("LIGO-WA-ARCHIVE", {'availability_write': False, 'availability_delete': False})

Define disk quota at this RSE for the root account

$ rucio-admin -a root account set-limits root LIGO-WA-ARCHIVE 10000000000000
Set account limit for account root on RSE LIGO-WA-ARCHIVE: 10.000 TB


We now configure attributes of the RSE to manage transfer and replication policy. First, define the FTS server to use:

$ rucio-admin rse set-attribute --rse LIGO-WA-ARCHIVE --key fts --value
Added new RSE attribute for LIGO-WA-ARCHIVE: fts- 
$ rucio-admin rse set-attribute --rse LIGO-WA-ARCHIVE --key fts_testing --value
Added new RSE attribute for LIGO-WA-ARCHIVE: fts_testing- 

In this example, we’re using the public CERN FTS server. An in-house FTS server should be deployed for production use.

Next, define some attributes useful for implementing replication policies:

$ rucio-admin rse set-attribute --rse LIGO-WA-ARCHIVE --key site --value LHO
Added new RSE attribute for LIGO-WA-ARCHIVE: site-LHO
$ rucio-admin rse set-attribute --rse LIGO-WA-ARCHIVE --key ALL --value 1
Added new RSE attribute for LIGO-WA-ARCHIVE: ALL-1 
$ rucio-admin rse set-attribute --rse LIGO-WA-ARCHIVE --key LIGO_LAB --value 1
Added new RSE attribute for LIGO-WA-ARCHIVE: LIGO_LAB-1 
$ rucio-admin rse set-attribute --rse LIGO-WA-ARCHIVE --key HDFS --value 0
Added new RSE attribute for LIGO-WA-ARCHIVE: HDFS-0 
$ rucio-admin rse set-attribute --rse LIGO-WA-ARCHIVE --key ARCHIVE --value 1
Added new RSE attribute for LIGO-WA-ARCHIVE: ARCHIVE-1 
$ rucio-admin rse set-attribute --rse LIGO-WA-ARCHIVE --key istape --value False
Added new RSE attribute for LIGO-WA-ARCHIVE: istape-False

Note: there is a pecularity (a bug: issue 1455) where RSEs must have attribute istape=False to permit downloads. This is in contrast with the standard boolean values of 0 and 1 used in all other RSE attributes.


In rucio, preferred transfer topologies can be achieved through a notion of distance and rankings between RSEs. Rucio will prefer to transfer data over shorter distances. For the time being, however, we will set the distances all to unity. To set the distance between LIGO-WA-ARCHIVE and LIGO-CIT-ARCHIVE then:

$ rucio-admin rse add-distance --distance 1 --ranking 1 LIGO-WA-ARCHIVE LIGO-CIT-ARCHIVE
Set distance from LIGO-WA-ARCHIVE to LIGO-CIT-ARCHIVE to 1 with ranking 1
$ rucio-admin rse add-distance --distance 1 --ranking 1 LIGO-CIT-ARCHIVE LIGO-WA-ARCHIVE
Set distance from LIGO-CIT-ARCHIVE to LIGO-WA-ARCHIVE to 1 with ranking 1

Note: distances must be defined between RSEs or they will have no connection at all.

That’s it. LIGO-WA-ARCHIVE should be ready to use.