Removing Data

Rule Expiration

The cleanest way to manage data removal is to use the replication rules. There are two options:

  1. Delete the rule: there is a grace period (of some hours) before the removal of the rule and the data it managees is effected.

  2. Update the lifetime of the rule to -1. Changes are effected at the next round of reaper operations - usually in minutes or less.

Updating the lifetime of a rule:

$ rucio update-rule 49a17f8b2b1d4aed81d4d9bab3459034 --lifetime -1
Updated Rule

Check the judge-evaluator log:

2019-08-15 08:18:38,420 21979   INFO    rule_cleaner[0/0]: Deleting rule 49a17f8b2b1d4aed81d4d9bab3459034 with expression UNL
2019-08-15 08:18:38,433 21979   DEBUG   Deleting lock ER8:H-H1_HOFT_C00-1123872768-4096.gwf for rule 49a17f8b2b1d4aed81d4d9bab3459034

Similarly, expect messages like the following in the reaper log:

2019-08-15 08:34:36,648       5365    INFO    Reaper 2-1: Deletion SUCCESS of postO2:H-H1_HOFT_C00-1232400384-4096.gwf as gsi on UNL in 3.13224601746 seconds

Erasing DIDs


The nuclear option is to go into the database and eliminate all trace of DIDs manually and, if necessary, delete from disk.

First, log into the SQL server and enter mysql:

$ kubectl exec -it $(kubectl get pods --namespace ligo-rucio -l "app=mysql" -o jsonpath="{.items[0]}") -- /bin/bash
bash-4.2# mysql -u rucio -p

Delete All DIDs

Suppose something has gone awry in file registration / upload in some testing instance: you may want to start from scratch without having to rebuild the entire database. Switch to the rucio db and delete all DIDs and associated tables:

mysql> use rucio;
Database changed

mysql> delete from replicas;
Query OK, 5 rows affected (0.26 sec)

mysql> delete from collection_replicas;
Query OK, 1 row affected (0.09 sec)

mysql> delete from contents;
Query OK, 5 rows affected (0.07 sec)

mysql> delete from dids;
Query OK, 6 rows affected (0.07 sec)