bilby_pipe.job_creation.node

Classes

Node

Base Node object, handles creation of arguments, executables, etc

Functions

`_is_htcondor_scitoken_local_issuer`()	Test whether the machine being used is configured to use a local issuer
`_log_output_error_submit_lines`(logdir, prefix)	Returns the filepaths for condor log, output, and error options

Module Contents

class bilby_pipe.job_creation.node.Node(inputs, retry=None)[source]

Bases: object

Base Node object, handles creation of arguments, executables, etc

run_node_on_osg = False[source]

inputs[source]

_universe = 'vanilla'[source]

request_disk[source]

notification[source]

retry = None[source]

verbose = 0[source]

condor_job_priority[source]

disable_hdf5_locking[source]

extra_lines[source]

requirements[source]

property universe[source]

process_node()[source]

_get_executable_path(exe_name)[source]

setup_arguments(parallel_program=None, add_command_line_args=True, add_ini=True, add_unknown_args=True)[source]

abstract property log_directory[source]

create_pycondor_job()[source]

add_accounting()[source]: Add the accounting-group and accounting-group-user extra lines

static _checkpoint_submit_lines()[source]

static _condor_file_transfer_lines(inputs, outputs)[source]

static _relative_topdir(path, reference)[source]: Returns the top-level directory name of a path relative to a reference

_osg_submit_options(executable, has_ligo_frames=False)[source]

Returns the extra submit lines and requirements to enable running a job on the Open Science Grid

Returns:

lineslist: the list of extra submit lines to include
requirementsstr: the extra requirements line to include

property slurm_walltime[source]: Default wall-time for base-name

property environment[source]

property transfer_container[source]: Whether a singularity container should be transferred to the job

static extract_paths_from_dict(input)[source]

job_needs_authentication(input_files)[source]

static _file_needs_authentication(fname)[source]

Check if a file needs authentication to be accessed, currently the only repositories that need authentication are ligo.osgstorage.org and *.storage.igwn.org.

Parameters:

fname: str: The file name to check

property scitoken_lines[source]: Additional lines needed for the submit file to enable access to proprietary files/services. Note that we do not support scoped tokens. This is determined by the method used to issue the scitokens. For more details see here.

property authenticated_file_prefix[source]: Return the prefix to add to files that need authentication. This is determined by the method used to issue the scitokens. For more details see here <https://computing.docs.ligo.org/guide/htcondor/credentials>.

property scitoken_issuer[source]: Return the issuer to use for scitokens. This is determined by the --scitoken-issuer argument or the version HTCondor running on the current machine. For more details see here.

bilby_pipe.job_creation.node._is_htcondor_scitoken_local_issuer()[source]: Test whether the machine being used is configured to use a local issuer or not. See here for where this logic comes from.

bilby_pipe.job_creation.node._log_output_error_submit_lines(logdir, prefix)[source]

Returns the filepaths for condor log, output, and error options

Parameters:

logdirstr: the target directory for the files
prefixstr: the prefix for the files

Returns:

log, output, errorlist of str: the list of three file paths to be passed to pycondor.Job

Examples

>>> Dag._log_output_error_submit_lines("test", "job")
['log = test/job.log',
 'output = test/job.out',
 'error = test/job.err']