Reporters

Reporters are a way for iDQ to record a variety of data products in a consistent fashion so that various asynchronous jobs can discover and access them easily. These include models, datasets, calibration maps, and timeseries that contain data products like p(glitch), log-likelihood, etc.

Each Reporter provides access to data by instantiating one with a location where data can be read from, and the data range in which to access data, along with protocol-specific keyword arguments. Some reporters are specific to the type of data they store and retrieve, i.e. GPSTimesReporter. Others are fairly generic and can be used for a variety of data products, i.e. PickleReporter.

API

class idq.io.reporters.Reporter(rootdir, start, end, **kwargs)

an object that handles I/O of models, calibration_maps this thing knows about file-systems and directory structures

the specifics about where things will be stored, etc should be set on creation

NOTE: this is just a parent class defining the API. Extensions of this

class must define their own functionality.

Upon instantiation, we require some knowledge of where these things will be written (rootdir) as well as some knowledge of what time range the span (start, end)

glob(nickname, start, end)

return a generator over all existing data associated with nickname within [start, end) NOTE: this should work without modifying the object!

classmethod read(path)

return the object from the path given

report(nickname, obj, preferred=False)

write a serialized version of the object into a path determined by kwargs

retrieve(nickname, preferred=False)

the inverse of report returns the object that was serialized

if preferred, looks for the most recent object, which does not necessarily correspond to where this thing would write serialized output supported for convenient retrieval

Generic

class idq.io.reporters.DiskReporter(*args, **kwargs)

a sub-class of Reporter that reads/writes to the local disk (by pickling objects) this defines a directory structure that it expects and works within that paradigm

class idq.io.reporters.hdf5.HDF5Reporter(*args, **kwargs)

a variant of DiskReporter that expects objects to fit into a tabular data format and writes that to disk as hdf5

class idq.io.reporters.pkl.PickleReporter(*args, **kwargs)

pickle objects instead of just writing strings into plaintxt

Models

class idq.io.reporters.pkl.PickleReporter(*args, **kwargs)

pickle objects instead of just writing strings into plaintxt

Datasets

class idq.io.reporters.hdf5.DatasetReporter(*args, **kwargs)

store high-level attributes from a dataset

Calibration Maps

class idq.io.reporters.hdf5.CalibrationMapReporter(*args, **kwargs)

store high-level information from a calibration map based on HDF5Reporter but extends this to use a more complicated structure under the hood

GPS Times

class idq.io.reporters.hdf5.GPSTimesReporter(*args, **kwargs)

a variant of HDF5 reporter that expects to only deal with lists of gps times

Segment Lists

class idq.io.reporters.ligolw.LIGOLWSegmentReporter(*args, **kwargs)

a variant of DiskReporter that expects to only deal with ligolw segments in XML form.

class idq.io.reporters.hdf5.HDF5SegmentReporter(*args, **kwargs)

a variant of HDF5 reporter that expects to only deal with segment lists

Timeseries

class idq.io.reporters.gwf.GWFSeriesReporter(*args, **kwargs)

store series informaiton in gwf files

NOTE: GWFSeriesReporter currently only supports a single contiguous segment and will raise an exception if it finds more than one. This is most likely the most common use cause (e.g.: streaming timeseries ignoring segdb), so it’s probably fine

class idq.io.reporters.hdf5.HDF5SeriesReporter(*args, **kwargs)

store series information in hdf5 files