{"type":"doc","content":[{"type":"heading","attrs":{"level":1},"content":[{"text":"Recommendations","type":"text"}]},{"type":"orderedList","attrs":{"order":1},"content":[{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"Use environments (e.g. conda or Python virtual environments)","type":"text","marks":[{"type":"strong"}]}]}]},{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"Use the group directory on the project (i.e. ","type":"text","marks":[{"type":"strong"}]},{"text":"/pc2/grous///","type":"text","marks":[{"type":"code"}]},{"text":")","type":"text","marks":[{"type":"strong"}]}]}]},{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"Don't use the parallel file system (","type":"text","marks":[{"type":"strong"}]},{"text":"/scratch/","type":"text","marks":[{"type":"code"}]},{"text":"…) for conda or Python virtual environments","type":"text","marks":[{"type":"strong"}]},{"text":".","type":"text"}]}]}]},{"type":"paragraph","content":[{"text":"Note that ","type":"text"},{"text":"HOME is not a good choice","type":"text","marks":[{"type":"strong"}]},{"text":" for storing Python packages since","type":"text"}]},{"type":"orderedList","attrs":{"order":1},"content":[{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"it is not designed for parallel use,","type":"text"}]}]},{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"it will very likely lead to quota issues (HOME is limited to 20 GB).","type":"text"}]}]}]},{"type":"heading","attrs":{"level":1},"content":[{"text":"Self-Learning Mini-Tutorials","type":"text"}]},{"type":"heading","attrs":{"level":2},"content":[{"text":"Conda","type":"text"}]},{"type":"codeBlock","content":[{"text":"which python\n\n# load module\n\nml lang\n\nml Miniforge3\n\nwhich python\n\nconda create --name myenv python=3.9.5\n\nconda activate myenv # won't work if shell integration hasn't been set up already\n\n# shell integration\n\nconda init\n\n. $HOME/.bashrc # update shell\n\n# or `source $(conda info --base)/etc/profile.d/conda.sh` for immediate + non-permanent\n\nconda activate myenv # works now\n\nwhich python\n\nconda list\n\nconda install numpy\n\n# so far so good, but where is our environment (and the packages etc.) located?\n\nconda env list\n\n# it shouldn't live in $HOME but on the parallel FS\n\n# how to resolve this?\n\n# 1) provide `--prefix ` to `conda create` (and then full path to `conda activate`)\n\n# 2) conda config --add envs_dirs /scratch///.conda/envs\n\n# conda config --add pkgs_dirs /scratch///.conda/pkgs\n\n# let's recreate it properly...\n\nconda deactivate myenv\n\n# conda env remove --name myenv .... but let's just rm -rf $HOME/.conda\n\nconda config --add envs_dirs /scratch/pc2-mitarbeiter/bauerc/.conda/envs\n\nconda config --add pkgs_dirs /scratch/pc2-mitarbeiter/bauerc/.conda/pkgs\n\ncat $HOME/.condarc\n\n# (potentially) redo the above....\n\n# then `which python`","type":"text"}]},{"type":"heading","attrs":{"level":2},"content":[{"text":"Pip (global)","type":"text"}]},{"type":"codeBlock","content":[{"text":"which python\n\npip # command not found\n\n# load module\n\nml lang\n\nml Python/3.9.5\n\nwhich python\n\nwhich pip\n\n# note that the default path for global packages etc. is $HOME/.local/lib/python/site-packages\n\n# to change it we can put the following into our .bashrc\n\n# export PYTHONUSERBASE=/scratch///.local\n\nexport PYTHONUSERBASE=/scratch/pc2-mitarbeiter/bauerc/.local\n\n# you might also want to add\n\nexport PATH=/scratch/pc2-mitarbeiter/bauerc/.local/bin:$PATH\n\npip install numpy\n\nls /scratch/pc2-mitarbeiter/bauerc/lib/python3.9/site-packages","type":"text"}]},{"type":"paragraph","content":[{"text":"However, if you want/need to use it, ","type":"text"},{"text":"we generally recommend to use `pip` within a conda environment!","type":"text","marks":[{"type":"strong"}]}]},{"type":"heading","attrs":{"level":2},"content":[{"text":"Poetry","type":"text","marks":[{"type":"link","attrs":{"href":"https://python-poetry.org/"}}]}]},{"type":"codeBlock","content":[{"text":"export POETRY_CACHE_DIR=/scratch/pc2-mitarbeiter/bauerc/.pypoetry\n\n# or `poetry config cache-dir /scratch/pc2-mitarbeiter/bauerc/.pypoetry` (config file is here: `$HOME/.config/pypoetry/config.toml`)\n\npoetry new myproject\n\ncd myproject\n\npoetry show # list installed packages\n\npoetry add numpy\n\npoetry show\n\n# how to run something in the environment or use it interactively?\n\nwhich python # still points to the general one.... WHY?!\n\npython\n\n import numpy # fails.... WHY?!\n\npoetry run which python\n\npoetry shell\n\n which python\n\n python\n\n import numpy # works within the poetry shell\n\n exit # leave the shell\n\n# where is the actual virtualenv: `poetry env list --full-path`","type":"text"}]},{"type":"heading","attrs":{"level":2},"content":[{"text":"Virtualenv","type":"text","marks":[{"type":"link","attrs":{"href":"https://virtualenv.pypa.io/en/latest/index.html#"}}]},{"text":" (this is not `python -m venv`!)","type":"text"}]},{"type":"codeBlock","content":[{"text":"#virtualenv --python=\"/usr/bin/python2.6\" \"/path/to/new/virtualenv/\"\n\nvirtualenv venv\n\nsource venv/bin/activate # activate local virtual environment\n\nwhich python\n\npip list\n\npip install numpy\n\npython\n\n import numpy # works\n\n \n\ndeactivate # deactivates local virtual environment\n\npython\n\nimport numpy # fails (as expected)","type":"text"}]},{"type":"paragraph"}],"version":1}

Browser not supported