Rucio Storage Elements (RSEs)¶

Data managed by rucio resides in rucio storage elements (RSEs). An RSE is defined by a series of settings for how rucio behaves, Attributes which help determine transfer and replication policies and protocols which determine the protocols and servers available for transfers at each RSE.

For example, the current LIGO-CIT RSE is defined:

$ rucio-admin rse info LIGO-CIT
Settings:
=========
  third_party_copy_protocol: 1
  rse_type: DISK
  domain: [u'lan', u'wan']
  availability_delete: True
  delete_protocol: 1
  rse: LIGO-CIT
  deterministic: True
  write_protocol: 1
  read_protocol: 1
  staging_area: False
  credentials: None
  availability_write: True
  lfn2pfn_algorithm: ligo
  availability_read: True
  volatile: False
  verify_checksum: True
  id: 98a8a2a1621e453aa164fce640b123b8
Attributes:
===========
  HDFS: False
  ALL: True
  LIGO_LAB: True
  site: CIT
  fts_testing: https://fts3-pilot.cern.ch:8446
  fts: https://fts3-pilot.cern.ch:8446
  LIGO-CIT: True
  ARCHIVE: False
  istape: False
Protocols:
==========
  gsiftp
    extended_attributes: None
    hostname: ldas-pcdev6.ligo.caltech.edu
    prefix: /mnt/rucio
    domains: {u'wan': {u'read': 1, u'write': 1, u'third_party_copy': 1, u'delete': 1}, u'lan': {u'read': 0, u'write': 0, u'delete': 0}}
    scheme: gsiftp
    port: 2811
    impl: rucio.rse.protocols.gfal.Default
Usage:
======
  rucio
    files: 3
    used: 530367545
    rse: LIGO-CIT
    updated_at: 2018-12-10 21:56:56
    free: None
    source: rucio
    total: 530367545

Create RSE¶

Create the RSE and define a copying protocol:

$ rucio-admin rse add LIGO-WA-ARCHIVE
Added new deterministic RSE: LIGO-WA-ARCHIVE
$ rucio-admin rse add-protocol\
    --prefix /archive/frames \
    --hostname ldas-pcdev6.ligo-wa.caltech.edu\
    --domain-json '{"wan": {"read": 1, "write": 0, "delete": 0, "third_party_copy": 1}}' \
    --scheme gsiftp \
    --port 2811 \
    --impl rucio.rse.protocols.gfal.Default \
    LIGO-WA-ARCHIVE

Note: since this is still a testing phase, we have deliberately set this to a read-only RSE. In this particular case, we also need to set the availability_delete setting. At the time, there is no CLI tool for this, so descend into python and use the API directly:

>>> from rucio.client.rseclient import RSEClient
>>> client = RSEClient()
>>> client.update_rse("LIGO-WA-ARCHIVE", {'availability_write': False, 'availability_delete': False})
True

Define disk quota at this RSE for the root account

$ rucio-admin -a root account set-limits root LIGO-WA-ARCHIVE 10000000000000
Set account limit for account root on RSE LIGO-WA-ARCHIVE: 10.000 TB

Attributes¶

We now configure attributes of the RSE to manage transfer and replication policy. First, define the FTS server to use:

$ rucio-admin rse set-attribute --rse LIGO-WA-ARCHIVE --key fts --value https://fts3-pilot.cern.ch:8446
Added new RSE attribute for LIGO-WA-ARCHIVE: fts-https://fts3-pilot.cern.ch:8446 
$ rucio-admin rse set-attribute --rse LIGO-WA-ARCHIVE --key fts_testing --value https://fts3-pilot.cern.ch:8446
Added new RSE attribute for LIGO-WA-ARCHIVE: fts_testing-https://fts3-pilot.cern.ch:8446 

In this example, we’re using the public CERN FTS server. An in-house FTS server should be deployed for production use.

Next, define some attributes useful for implementing replication policies:

$ rucio-admin rse set-attribute --rse LIGO-WA-ARCHIVE --key site --value LHO
Added new RSE attribute for LIGO-WA-ARCHIVE: site-LHO
$ rucio-admin rse set-attribute --rse LIGO-WA-ARCHIVE --key ALL --value 1
Added new RSE attribute for LIGO-WA-ARCHIVE: ALL-1 
$ rucio-admin rse set-attribute --rse LIGO-WA-ARCHIVE --key LIGO_LAB --value 1
Added new RSE attribute for LIGO-WA-ARCHIVE: LIGO_LAB-1 
$ rucio-admin rse set-attribute --rse LIGO-WA-ARCHIVE --key HDFS --value 0
Added new RSE attribute for LIGO-WA-ARCHIVE: HDFS-0 
$ rucio-admin rse set-attribute --rse LIGO-WA-ARCHIVE --key ARCHIVE --value 1
Added new RSE attribute for LIGO-WA-ARCHIVE: ARCHIVE-1 
$ rucio-admin rse set-attribute --rse LIGO-WA-ARCHIVE --key istape --value False
Added new RSE attribute for LIGO-WA-ARCHIVE: istape-False

Note: there is a pecularity (a bug: issue 1455) where RSEs must have attribute istape=False to permit downloads. This is in contrast with the standard boolean values of 0 and 1 used in all other RSE attributes.

Distances¶

In rucio, preferred transfer topologies can be achieved through a notion of distance and rankings between RSEs. Rucio will prefer to transfer data over shorter distances. For the time being, however, we will set the distances all to unity. To set the distance between LIGO-WA-ARCHIVE and LIGO-CIT-ARCHIVE then:

$ rucio-admin rse add-distance --distance 1 --ranking 1 LIGO-WA-ARCHIVE LIGO-CIT-ARCHIVE
Set distance from LIGO-WA-ARCHIVE to LIGO-CIT-ARCHIVE to 1 with ranking 1
$ rucio-admin rse add-distance --distance 1 --ranking 1 LIGO-CIT-ARCHIVE LIGO-WA-ARCHIVE
Set distance from LIGO-CIT-ARCHIVE to LIGO-WA-ARCHIVE to 1 with ranking 1

Note: distances must be defined between RSEs or they will have no connection at all.

That’s it. LIGO-WA-ARCHIVE should be ready to use.