JupyterHub for IGWN¶

JupyterHub provides browser-based access to IGWN computing environments through Jupyter notebooks and terminals. This service offers authenticated, multi-user notebook servers with automatic access to IGWN software stacks, CVMFS repositories, and your home directory.

Quick Start¶

1. Access JupyterHub: https://jupyter.ligo.caltech.edu

2. Authenticate with your LIGO or KAGRA credentials via CILogon

3. Select an environment (profile) when starting your server

4. Start working - your notebooks run in a containerized environment with full IGWN software access

What is JupyterHub?¶

JupyterHub is a multi-user server that provides:

• Jupyter Notebooks: Interactive Python notebooks for data analysis and visualization
• JupyterLab Interface: Modern web-based IDE with file browser, terminals, and notebooks
• Terminal Access: Full shell access with your preferred shell (bash, csh, tcsh, zsh)
• IGWN Software: Direct access to IGWN conda environments via CVMFS
• Persistent Storage: Your work is saved in your home directory
• Resource Allocation: Configurable CPU and memory based on your analysis needs

Available Environments (Profiles)¶

When you start your JupyterHub server, you choose an environment optimized for your workflow:

IGWN Environment (Default)¶

Best for: General gravitational wave analysis, data exploration, standard workflows

Features: - Access to multiple IGWN conda environments from CVMFS - Available Python versions: 3.9, 3.10, 3.11 (see IGWN software environments) - Dynamically discovers new IGWN software releases - Full suite of IGWN analysis tools (GWpy, PyCBC, LALSuite, etc.)

Kernels: - IGWN py39: Python 3.9 with IGWN software stack - IGWN py310: Python 3.10 with IGWN software stack - IGWN py311: Python 3.11 with IGWN software stack

PyCBC Environment¶

Best for: Compact binary coalescence analysis, PyCBC-specific workflows

Features: - Optimized PyCBC installation with all dependencies - Pre-configured for CBC searches and parameter estimation - Compatible waveform generators

Kernels: - PyCBC Python Environment: Python with full PyCBC stack

Switching Between Profiles¶

If you need to change environments:

1. Save your work - ensure all notebooks are saved
2. Stop your server: File → Hub Control Panel → Stop My Server
3. Wait for confirmation - server status shows "stopped"
4. Start new server: Click "Start My Server" and select different profile
5. Access your files - all files remain in your home directory

Note: You can only run one server (profile) at a time. Switching profiles requires stopping and restarting your server, but your files are preserved.

Using JupyterHub¶

Starting Your Server¶

1. Navigate to https://jupyter.ligo.caltech.edu and authenticate
2. On the spawn page, select your desired environment:
3. IGWN Environment: For general analysis (recommended for most users)
4. PyCBC Environment: For PyCBC-specific workflows
5. Click Start to launch your server

Your server typically starts within 30-60 seconds.

Working with Notebooks¶

Creating a New Notebook¶

1. In JupyterLab, click File → New → Notebook
2. Select your kernel:
3. IGWN Environment: Choose IGWN py311 (recommended), IGWN py310, or IGWN py39
4. PyCBC Environment: Use PyCBC Python Environment
5. Start writing Python code in cells
6. Execute cells with Shift+Enter

Example: Basic Data Access¶

# Load IGWN libraries
from gwpy.timeseries import TimeSeries

# Fetch data from GWOSC
data = TimeSeries.fetch_open_data('H1', 1126259446, 1126259478)

# Plot
plot = data.plot()
plot.show()

Using Terminals¶

JupyterHub provides full terminal access with your LDAP-configured shell:

1. Open a terminal: File → New → Terminal
2. Your shell: Automatically uses your preferred shell from LDAP (bash, csh, tcsh, or zsh)
3. Software access: IGWN conda environments available via CVMFS

Accessing IGWN Software¶

IGWN software is automatically available in both notebooks and terminals:

In Notebooks: - Select an IGWN kernel (py39, py310, or py311) when creating a notebook - All IGWN packages are immediately available

In Terminals: - Conda is automatically initialized when you open a terminal - The default igwn environment is activated automatically - No manual setup required!

# Conda is already initialized - just start using it
conda --version

# List available IGWN environments
conda info --envs

# Switch to a different environment
conda activate igwn-py311

Customizing Your Conda Environment¶

You can control which environment activates automatically:

Set Preferred Environment:

# Create ~/.conda_version with your preferred environment
echo "igwn-py311" > ~/.conda_version

Next time you open a terminal, igwn-py311 will be activated automatically.

Disable Auto-Activation:

# Create opt-out file
touch ~/.noigwn

Restart your server for changes to take effect (File → Hub Control Panel → Stop My Server).

Pre-Configured Environment Variables¶

JupyterHub automatically sets environment variables for data access and computation:

Data Access: - LIGO_DATAFIND_SERVER - LDR server for gravitational wave data - GWDATAFIND_SERVER - GW datafind server

Computation: - XLA_PYTHON_CLIENT_PREALLOCATE - JAX GPU memory configuration

These variables are available in both terminals and notebooks, matching baremetal system behavior. No configuration needed!

File Management¶

Home Directory¶

Your home directory (/home/username) is: - Automatically mounted in all environments - Shared across all profiles - Persistent across server restarts - Backed by the same NFS storage as LIGO cluster systems

CVMFS Access¶

All environments have read-only access to CVMFS: - /cvmfs/software.igwn.org - IGWN software distributions - /cvmfs/oasis.opensciencegrid.org - OSG software - Other CVMFS repositories as configured

CVMFS is automatically mounted, just as on LIGO cluster systems.

Uploading Files¶

1. Via JupyterLab UI: Click upload button in file browser
2. From Git: Clone repositories directly in terminals

Downloading Files¶

1. Right-click file in JupyterLab file browser → Download
2. Multiple files: Select files, right-click → Download

Authentication¶

LIGO Users¶

1. Select LIGO as your identity provider in CILogon
2. Authenticate with your LIGO.ORG credentials
3. Your LDAP identity (UID, GID, groups) is automatically mapped

KAGRA Users¶

1. Select KAGRA as your identity provider in CILogon
2. Authenticate with your KAGRA credentials
3. Your KAGRA LDAP identity is automatically mapped

Multi-Organization Users¶

If you have accounts with both LIGO and KAGRA: - Select the appropriate identity provider for the collaboration you're working with - The system automatically maps to your correct UID based on your chosen IdP - Your home directory location may differ based on organization

Best Practices¶

Notebook Management¶

1. Save frequently - use File → Save or Ctrl+S
2. Use descriptive names - analysis-O3-burst.ipynb not Untitled-1.ipynb
3. Organize in folders - keep related notebooks together
4. Clear output before saving - Cell → All Output → Clear (for version control)

Resource Management¶

1. Close unused notebooks - shut down kernels you're not using
2. Restart kernels periodically - prevents memory leaks: Kernel → Restart Kernel
3. Monitor resource usage - check terminal output for memory warnings

Version Control¶

Use Git for important analysis code:

# In terminal
git clone https://git.ligo.org/your-group/your-repo.git
cd your-repo
# Work on your notebooks
git add analysis.ipynb
git commit -m "Add burst analysis"
git push

Data Analysis Tips¶

1. Process data in chunks - avoid loading entire datasets into memory
2. Use IGWN py311 - most recent Python version with latest features
3. Profile your code - identify bottlenecks before optimizing

Troubleshooting¶

Server Won't Start¶

Problem: Server fails to start or times out

Solutions: - Refresh the page and try again - Try selecting a different profile - Contact support if issue persists

Kernel Dies or Becomes Unresponsive¶

Problem: Kernel crashes or stops responding

Common causes: - Out of memory (exceeded resource limits) - Infinite loop in code - Incompatible library versions

Solutions: - Restart kernel: Kernel → Restart Kernel - Clear all output: Cell → All Output → Clear - Restart server if kernel repeatedly crashes - Optimize code to use less memory - Request higher resource limits from administrators

Package Not Found¶

Problem: ImportError or ModuleNotFoundError for a package

Solutions:

For IGWN packages:

# Check if you're using the right kernel
import sys
print(sys.prefix)
# Should show a /cvmfs/software.igwn.org path

# List installed packages
!conda list | grep package-name

For missing packages:

# Temporary user installation (persists in home directory)
!pip install --user package-name

Note: User-installed packages are temporary workarounds. For permanent additions, request package inclusion through IGWN software working group.

Can't Access Data Files¶

Problem: Data files are not accessible or show permission errors

Check: 1. Verify file path: ls -la /path/to/file 2. Check permissions: files should be readable by your UID 3. Verify CVMFS is mounted: ls /cvmfs/software.igwn.org

Wrong Shell in Terminal¶

Problem: Terminal opens with wrong shell (e.g., bash instead of tcsh)

Cause: LDAP loginShell attribute not set or not synced

Workaround:

# Temporarily switch shells
exec tcsh

Permanent fix: Contact LIGO computing support to update your LDAP loginShell attribute

Connection Lost / Server Unreachable¶

Problem: Can't connect to JupyterHub or connection drops

Check: 1. Verify network connectivity 2. Check service status page (if available) 3. Try incognito/private browsing mode 4. Clear browser cache 5. Try different browser

Resource Limits¶

Default Allocations¶

Each profile starts with: - CPU: 0.5 cores guaranteed (scalable based on availability) - Memory: 2 GB guaranteed (scalable based on availability)

Requesting Higher Limits¶

If your analysis requires more resources:

1. Document your requirements: CPU cores, memory needed, expected runtime
2. Contact your site administrators: Platform team can configure per-user or per-profile limits
3. Consider batch computing: For very large-scale analysis, use HTCondor on LIGO cluster systems

Monitoring Resource Usage¶

In a notebook cell:

# Check memory usage
import psutil
process = psutil.Process()
print(f"Memory: {process.memory_info().rss / 1024**3:.2f} GB")

# Check CPU usage
import os
print(f"CPU cores available: {os.cpu_count()}")

In terminal:

# Check current resource usage
top

# Check memory limits
free -h

Advanced Usage¶

Installing Additional Packages¶

Temporary User Installation¶

# Install in user's home directory (persists across restarts)
!pip install --user package-name

# Verify installation
import package_name

Using Custom Conda Environments¶

# In terminal - create custom environment in home directory
conda create -n myenv python=3.11
conda activate myenv
conda install gwpy pycbc

# Register as Jupyter kernel
python -m ipykernel install --user --name=myenv --display-name="My Custom Environment"

Note: Custom environments persist in your home directory but must be manually maintained.

Running Long-Duration Analyses¶

For analyses that take hours or days:

1. Use batch computing (HTCondor on LIGO cluster systems) instead of JupyterHub for production runs
2. Develop and test in JupyterHub notebooks
3. Convert to scripts for batch execution on clusters
4. Monitor completion and fetch results back to JupyterHub for visualization

Note: HTCondor is not currently available within the JupyterHub environment but is planned for future releases.

Jupyter Extensions and Customization¶

JupyterLab supports extensions, but installation requires administrator access.

Request extensions: Contact administrators with: - Extension name and purpose - Use case / benefit to users - Link to extension documentation

Currently available extensions: [List TBD based on deployment]

Differences from Other IGWN Computing Resources¶

vs. LIGO Clusters (SSH Access)¶

When to use JupyterHub: - Interactive data exploration - Notebook-based analysis - Prototyping and development - Visualization and plotting - Teaching and collaboration

When to use LIGO Clusters: - Large-scale production runs - Batch job submission (HTCondor) - Long-duration analyses - High-memory or high-CPU workloads - GPU computing

vs. Local Installation¶

Advantages of JupyterHub: - No local installation required - Automatic access to IGWN software (no conda environment management) - Access your data and home directory from anywhere - Pre-configured computing environment - Collaboration-provided resources

Advantages of Local: - Works offline - Full control over environment - No resource limits - HTCondor and GPU access (if available locally)

Getting Help¶

Documentation¶

• This guide: General JupyterHub usage for IGWN
• IGWN Software Guide: https://computing.docs.ligo.org/guide/software/
• IGWN Software Environments: https://computing.docs.ligo.org/guide/software/environments/
• JupyterLab Documentation: https://jupyterlab.readthedocs.io/
• IGWN Python Documentation: https://gwpy.readthedocs.io/

Support Channels¶

1. Site-specific support: Contact your local computing support team
2. IGWN Computing: LIGO.ORG Slack #computing channel
3. Bug reports: Report via site administrators
4. Feature requests: Discuss in IGWN computing working group

Common Support Requests¶

When requesting help, include: - What you were trying to do - What actually happened (error messages, screenshots) - Which environment/profile you're using - Which kernel you selected - Steps to reproduce the issue

Maintenance and Updates¶

Service Announcements¶

JupyterHub maintenance windows and updates are announced via: - LIGO.ORG email announcements - Service status pages - Slack #computing channel

Software Updates¶

IGWN Software: Updated automatically via CVMFS (no action required)

Container Images: Updated by administrators on a regular schedule - New releases include updated base software - Server restart required to use new images

Checking your version:

# In a notebook
import sys
print(f"Python: {sys.version}")
print(f"Installation: {sys.prefix}")

# Check specific package versions
import gwpy
print(f"GWpy: {gwpy.__version__}")

Frequently Asked Questions¶

Q: Can I run HTCondor jobs from JupyterHub? A: Not currently. HTCondor is not available in the JupyterHub Kubernetes environment, though it is planned for future releases. Use traditional LIGO cluster access for batch computing.

Q: How long can my server run? A: Servers can run as long as needed. Plan for interruptions during scheduled maintenance windows.

Q: Can multiple people share a notebook? A: Not in real-time. Use Git for collaboration, or consider JupyterHub extensions like jupyterlab-git for better workflow integration.

Q: Is my data backed up? A: Your home directory uses the same NFS storage as LIGO cluster systems with standard backup policies. Always maintain backups of critical analysis results.

Q: Can I run GPU workloads? A: GPU support is not currently available on jupyter.ligo.caltech.edu but is under investigation. Contact administrators for updates on GPU availability.

Q: What's the difference between IGWN py39, py310, and py311? A: Different Python versions - py311 (Python 3.11) is newest and recommended, py310 (Python 3.10) is stable, py39 (Python 3.9) is for compatibility with older code. See IGWN software environments for details.

Q: Can I access proprietary LIGO data? A: Yes, authentication ensures proper access control. Data access follows standard LIGO data policies based on your credentials and collaboration membership.

Q: How do I switch from IGWN to PyCBC environment? A: Save your work, then go to File → Hub Control Panel → Stop My Server. Once stopped, click "Start My Server" and select the PyCBC profile. All your files will still be in your home directory.

For technical details on the JupyterHub deployment, container images, and infrastructure, contact your site computing team or LIGO computing support.

Last updated: October 1, 2025