Getting Started
Binaries
We provide the following modules with Python binaries:
lang/Pythonlang/Anaconda3
Please contact pc2-support@uni-paderborn.de if you need other binaries or distributions.
Installing packages
Environments
We highly recommend to install python packages in local environment rather than in a global package depot. Common choices are conda environments or venv environments (see here for a non-conda overview).
Where to install packages?
The recommended place for Python packages/environments is PC2PFS, i.e. under /scratch. Note that /scratch is not shared between Noctua 1 and Noctua 2 and you will thus need to maintain separate package depots/environments for each system. This is the recommended approach because both systems have very different architectures (e.g. Intel vs AMD CPUs) such that binaries may easily be incompatible.
| Panel | ||||||
|---|---|---|---|---|---|---|
| ||||||
Note that HOME is not a good choice for storing Python packages since 1) it is not designed for parallel use, 2) will very likely lead to quota issues (HOME is limited to 20GB), and 3) causes unnecessary backups of reproducible data. |
Changing global install location
Pip
Without intervention, and if your’e not using local environments, pip install -U <package> will install packages into $HOME/.local, which is discouraged for the reasons mentioned above. To change the default install location, we recommend to put something like the following into your .bashrc:
| Code Block |
|---|
export PYTHONUSERBASE=/scratch/<group>/<username>/.local |
Conda
By default, conda will install package data and environments into $HOME/.conda. This should be avoided for the reasons mentioned above. To change the global default, execute the following commands, which will set the corresponding paths in your $HOME/.condarc :
| Code Block |
|---|
conda config --add envs_dirs /scratch/<group>/<username>/.conda/envs
conda config --add pkgs_dirs /scratch/<group>/<username>/.conda/pkgs |
Additionally, you may create a link for $HOME/.cache pointing to /scratch/<group>/<username>/.cache
| Code Block |
|---|
cd /scratch/<group>/<username>/
mkdir .cache
cd
mv .cache .cache-old #Just as a precaution. May be removed
ln -s /scratch/<group>/<username>/.cache |
However, be aware that the directory $HOME/.cache is used by many applications. Since /scratch ist cluster local, this may be sometimes confusing.