.. _production:

How to Install and Run the DQR in production
====================================================================================================

Installation
----------------------------------------------------------------------------------------------------

The DQR can be installed with the following::

    git clone https://git.ligo.org/detchar/data-quality-report.git
    cd data-quality-report
    pip install . --prefix /path/to/install

*Note*, this will require all dependencies to be installable under pip, which may not be the case.
Instead, it is suggested that users manage dependencies separately (e.g.: rely on system-wide installations of things like `lalsuite`) and install the DQR via::

    pip install . --prefix /path/to/install --no-deps

Make sure to then update your environment appropriately, which will likely involve just the following::

    export PATH=/path/to/install/bin:$PATH
    export PYTHONPATH=/path/to/install/lib/python2.7/site-packages/:$PYTHONPATH

although the `PYTHONPATH` might vary depending on your Python version.

Also note that several configuration files for individual tasks will be installed as `package_data` alongside the source code.
This does *not* include `dqr.ini` and `dqr_listener.ini`, the configuration files used with `dqr-create` and `lvalert_listen`, respectively.
We describe how to manage those below.

Production Management
----------------------------------------------------------------------------------------------------

In production, maintainers will need to manage two configuration files

  * `dqr.ini`: tells the DQR which tasks to include in the html document and which to run under its DAG.
  * `dqr_listener.ini`: tells the `lvalert_listen` instance where to find the DQR executable and `dqr.ini`, etc. Also allows users to specify different executables or config files for different types of events (group, pipeline, search tuples).

We refer developers to the `GraceDb tutorial on managing LVAlert listeners <https://gracedb.ligo.org/documentation/responding_to_lvalert.html>`_ for a description of how to set up the lvalert listener.
Developers should refer to :ref:`configuring` for information about configuring `dqr.ini`.
Maintainers will likely modify these config files slightly by hand to account for different run-time environments and locations.

Both these steps can be tested offline, before the system is put into production.
We describe several solutions for that in :ref:`testing`.

Once you've set up your listener and configuration, you will need to protect it from power cycles and random failures in some way. 
This can often be done with a simple `cron` daemon that attempts to restart the listener periodically.
One solution might be to use `autorestart <https://github.com/reedessick/autorestart>`_, a simple library to mangae processes under `cron` with convenient logging.
However, this should likely be deprecated in favor of `monit`, `systemd`, or long-term processes managed directly by `condor`.

You may also want to monitor your listener via `lvalert-heartbeat <https://git.ligo.org/lscsoft/lvalert-heartbeat>`_ and Nagios. 
Please refer to the documentation within `lvalert-heartbeat <https://git.ligo.org/lscsoft/lvalert-heartbeat/blob/master/README.md>`_ for more details.