JupyterHub
- 1 PC² JupyterHub
- 1.1 Access
- 1.2 Quick Start
- 1.3 Server Options
- 1.3.1 Presets
- 1.3.2 Simple
- 1.3.3 Advanced (Slurm)
- 1.4 Loading additional Jupyter kernels
- 1.5 Apptainer (Singularity) Container
- 1.6 Remote Desktop (Graphical Environment via Xpra)
- 1.7 How-To
- 1.7.1 Creating presets
- 1.7.1.1 Spawner
- 1.7.1.2 Preset scopes
- 1.7.1.3 Default URL
- 1.7.1.4 Notebook directory
- 1.7.1.5 Apptainer container
- 1.7.1.6 Environment variables
- 1.7.1.7 Modules
- 1.7.1.8 Slurm job flags
- 1.7.2 Create custom IPython kernel inside custom conda environment
- 1.7.3 Create my own Apptainer/Singularity container
- 1.7.3.1 Container package requirements
- 1.7.3.2 Example Apptainer/Singularity recipe
- 1.7.3.2.1 Base recipe
- 1.7.3.3 Container Location
- 1.7.4 Access remote JupyterHub server with the local Visual Studio Code instance
- 1.7.5 View Slurm job logs
- 1.7.6 PC² Support
- 1.7.1 Creating presets
- 1.8 Troubleshooting
- 1.8.1 JupyterLab
PC² JupyterHub
The JupyterHub service is available for Noctua 1 and Noctua 2.
Access
The JupyterHub can be reached at the following address:
Noctua 2: https://jh.pc2.uni-paderborn.de
Noctua 1: https://jh.noctua1.pc2.uni-paderborn.de
The JupyterHub can be accessed via VPN or on-site at the University of Paderborn.
Quick Start
Spawn host/resources | Start |
---|---|
Jupyter Session on Noctua 2 | |
Jupyter Session on Noctua 1 | |
Jupyter Notebook on Noctua 2 (Inside Slurm job, 1h runtime, normal partition) | |
Jupyter Notebook on Noctua 2 (Inside Slurm job, 1h runtime, gpu partition) | |
Jupyter Notebook on Noctua 1 (Inside Slurm job, 1h runtime, normal partition) | |
Jupyter Notebook on Noctua 1 (Inside Slurm job, 1h runtime, gpu partition - 1x A40) |
Server Options
Presets
You have the possibility of creating a preset with predefined start options for yourself or your project group.
Click here to list your presets: https://jh.pc2.uni-paderborn.de/services/presets/
Simple
Preset enviroments with predefined values how to start the Jupyter Notebook.
Default and self-created apptainer containers can be used.
Advanced (Slurm)
An advanced view with setting options how a Slurm job should be started on a HPC cluster.
Loading additional Jupyter kernels
You can load additional Jupyter kernel using Lmod (module). Following kernel are currently available:
If you need new kernel versions or even other programming languages then you are welcome to contact pc2-support!
Apptainer (Singularity) Container
In JupyterHub it is possible to launch Jupyter Notebook instances inside a Singularity container. This has the advantage of being able to use your own built environment. When starting a container, any directories can be mounted inside the container environment.
We provide a set of default Singularity containers:
Container name | Kernels available | Installed software |
---|---|---|
jupyter_scientific_python | Python |
|
jupyter_datascience | Julia, Python, R |
|
To learn more about Singularity, see here: Singularity-Introduction
If you want to build your own Singularity container for JupyterHub, see here: JupyterHub | Create my own Singularity container
Remote Desktop (Graphical Environment via Xpra)
To create a remote desktop environment, you can click on "Desktop Environment" in the JupyterLab interface:
When you click on the tile ‘Xpra Desktop’, a remote desktop environment is set up in the background. Graphical applications (e.g. loaded via modules) can be started from the started graphical terminal.
How-To
Creating presets
To save time when configuring your Jupyter environenment you have the possibilty to create preset environments for yourself or your compute time group(s).
Created presets can be selected when starting a new Jupyter instance:
Create presets here: https://jh.pc2.uni-paderborn.de/services/presets/ (or JupyterHub home → services → presets)
Spawner
Local spawner (on JupyterHub)
Spawning the Jupyter notebook environment on the JupyterHub host. Slurm job flags not needed.
Slurm tools, Modules, Remote desktop environment are available.
Noctua 2 (via Slurm)
Spawning the Jupyter environment inside a Slurm job (on a compute/gpu/fpga node) on Noctua 2. Note: You need to specifiy Slurm job flags.
Preset scopes
Select who can use your preset. You or one of your compute time projects.
Default URL
The URL to which JupyterHub redirects when the server is started.
Example:
/lab
-> Spawning JupyterLab environment
/xprahtml5
-> Spawning Remote desktop environment
Notebook directory
The working directory. Used for JupyterLab, the remote desktop environment and the classic Jupyter view.
Apptainer container
Your self-built Apptainer/Singularity container. Have a look here for creating your own container: JupyterHub | Create my own Apptainer/Singularity container
Environment variables
Extra environment variables.
Format:
MY_ENV_VAR=”Hello World”
FOO=BAR
Modules
Extra Lmod modules to load on start time. All system modules and Jupyter specific kernels are available.
Slurm job flags
Slurm job flags in Slurm batch format. Example:
#SBATCH --partition=normal
#SBATCH --time=01:00:00
Create custom IPython kernel inside custom conda environment
Create a conda environment as described here:
conda activate <your_conda_env>
conda install ipykernel
Or:
python3 -m pip install ipykernel
ipykernel install --user --name <KERNELNAME> --display-name "<DISPLAY NAME>"
Create my own Apptainer/Singularity container
Container package requirements
python >= 3.10
jupyterhub
optional, but useful:
jupyterlab
Example Apptainer/Singularity recipe
Build containers: Apptainer
Base recipe
Bootstrap: docker
From: debian
%post
apt -y update
export DEBIAN_FRONTEND=noninteractive
apt -y install zsh locales
localedef -i en_US -c -f UTF-8 -A /usr/share/locale/locale.alias en_US.UTF-8
python3 -m pip install jupyterhub
Install custom Python kernel inside the container (python 3.12)
Install Lmod with the JupyterLab-Lmod extension
Make Slurm Tools inside my container available
Container Location
All containers with type .sif will be automatically detected in $HOME/.jupyter/pc2-jupyterhub/
Your new built container can only be placed in your $HOME
directory: $HOME/.jupyter/pc2-jupyterhub/
Alternatively you can create a link from your $PC2PFS
to your $HOME
directory:
Access remote JupyterHub server with the local Visual Studio Code instance
You need following extensions for Visual Studio Code:
https://marketplace.visualstudio.com/items?itemName=ms-toolsai.jupyter
https://marketplace.visualstudio.com/items?itemName=ms-toolsai.jupyter-hub
Create an acccess token in the JupyterHub web interface:
Follow following instructions described here: https://marketplace.visualstudio.com/items?itemName=ms-toolsai.jupyter-hub
You need to start a Jupyter session using our JupyterHub web interface. After successful start, you can copy the URL starting with
https://jh.pc2.uni-paderborn.de/user/.../...
View Slurm job logs
If the path of the Slurm Job output has not been changed explicity, it can be found here by default:
Noctua 1: $HOME/.jupyter/last_jh_noctua1.log
Noctua 2: $HOME/.last_jh_noctua2.log
PC² Support
If you have any other problems that won’t be solved, please contact the pc2-support@uni-paderborn.de
Troubleshooting
JupyterLab
“A Jupyter server is running.” message
This message appears because user settings managed by JupyterLab do not match the new JupyterLab version.
Try deleting
~/.jupyter/
in your$HOME
directory as follows:rm -r ~/.jupyter/
If you want to keep your custom user settings, write an email to pc2-support@uni-paderborn.de