AiiDA (https://www.aiida.net/) can be used in a number of different ways with PC2 HPC systems:

1. Running AiiDA on a different system, such as a local computer, and submitting calculations to PC2 HPC systems.

2. Running calculations, as well as AiiDA itself, on PC2 HPC systems.

Because these two situations need different setup steps, we will describe them independently below. We recommend the first possibility, i.e., running AiiDA on a local system and submitting calculations to PC2 HPC system because a local setup gives you more configuration freedom and also the possibility to, at the same time, submit also to other HPC systems. However, if you donÄt have a suitable local computer we include here also a guide for the second possibility of running AiIDA completely on PC2 HPC systems. If you are unsure what the best way for you is, please contact us at pc2-support@uni-paderborn.de.

The two setups have individual advantages and disadvantages:

Running AiiDA on a Local System and Submitting Calculations to PC2 HPC Systems

This setup assumes that you are running AiiDA on a local computer or server outside of PC2. For the setup of AiIDA on the local computer follow the detailed guide https://aiida.readthedocs.io/projects/aiida-core/en/stable/installation/index.html. In case of questions or problems or questions please don't hesitate to contact us via pc2-support@uni-paderborn.de.

Create a Computer

To be able to submit calculations to a PC2 HPC system, create a computer in AiIDA by running the following on your local system where AiiDA is running:

Replace:

• [work dir] with a path to a directory on the parallel file system that AiiDA is going to use as a temporary working directory for your jobs, e.g., /scratch/hpc-prf-[project abbreviation]/{username}/AiiDA/scratch. {username} is automatically replaced by AiiDA with your user name.

Configure the Computer

For access via ssh the computer needs to be configured by running:

verdi -p [AiiDA profile name] computer configure core.ssh Noctua2

with the following settings:

Replace:

• [path to your ssh private key]: with the path to your SSH key that is enabled for login to PC2 cluster systems. Please note that the ssh-agent is required so that password-protected ssh-keys work with AiiDA. Password-protected ssh-keys are required for PC2 HPC systems. See also our guide on SSH keys.

• [user name]: your PC2 user name

Testing the Computer

Run verdi computer test Noctua2 to test the configuration. In case of questions or problems or questions please don't hesitate to contact us via pc2-support@uni-paderborn.de.

Create a Code

The next steps show with the example of Quantum Espresso pw.x how you can use the existing software on PC2 HPC systems with AiiDA. In this example we use Quantum Espresso from the module chem/QuantumESPRESSO/7.3-foss-2023a.

1. Run AiiDA Quantum Espresso plugins:

   pip install aiida-quantumespresso verdi plugin list aiida.calculations aiida-pseudo install sssp #for the example below

2. Run verdi code create core.code.installed and enter

3. Test the code:

Running a Calculation

1. Import an atomic structure

2. List structures and their IDs:

3. Run the verdi shell (verdi shell) with

The last lines specify the properties of the compute job in terms of partition, time limit, and resources. More options can be found at https://aiida.readthedocs.io/projects/aiida-core/en/stable/topics/calculations/usage.html#options. The meaning is as follows:

• builder.metadata.options.max_wallclock_second: time limit of the job in seconds

• builder.metadata.options.queue_name: name of the partition, e.g., normal, largemem,…

• builder.metadata.options.withmpi: should the job be run with MPI

• builder.metadata.options.resources

  • num_machines: sbatch option -N/--nodes=

  • num_mpiprocs_per_machine: sbatch option --ntasks-per-node=

  • num_cores_per_mpiproc: sbatch option -c/--cpus-per-task=

• builder.metadata.options.account: the name of your compute project, required if you are a member of multiple compute projects

• builder.metadata.options.qos: in case you want to use special QoS like devel or express

• builder.metadata.options.max_memory_kb: maximal memory per node in case you want to overwrite the default values of memory per node
  
  

4. Run the job with AiiDA

   • If you run AiIDA locally with RabbitMQ:

   • If you run AiiDA locally without RabbitMQ:

      

5. You can monitor your jobs with verdi process list -a

Running AiiDA on PC2 Systems

We describe here the setup of AiiDA on PC2 HPC systems with Sqlite and PostgresSQL but without RabbitMQ. RabbitMQ is problematic on HPC systems because care must be taken about access control (see https://www.rabbitmq.com/docs/access-control). Please contact pc2-support@uni-paderborn.de if you need RabbitMQ.

To run AiiDA on PC2 systems, there are three options:

1. Run AiiDA on one of the login nodes, i.e., n2login[1-6]

2. Run AiiDA on the PC2 Jupyter Hub

3. Run AiiDA in a long-running compute job (at most 21 days)

Basic Setup of AiiDA

1. Load a suitable Python module, e.g.,

2. Create a Python virtual environment and activate it

   We recommend placing Python virtual environments in the PC2 groups directories, i.e., /pc2/groups/hpc-prf-[your project abbreviation]

3. install AiiDA

4. If you want to use AiIDA in the PC2 Jupyter Hub, you need to also register the virtual environment as a Jupyter Hub kernel:

to load the AiiDA Jupyter notebook extension so that you can for example use verdi as

Database Setup

As a database backend, you can either use SQLite or PostgreSQL

Setup for SQLite

1. Create an AiiDA profile

As the directory of the backend, you can choose a directory on the parallel file system.

Setup for PostgreSQL

1. Set up a PostgreSQL database

   1. Load PostgreSQL module

   2. Copy sample configuration

   3. Change the settings in the config postgresql.conf:

      • port = : choose some random port (>1024, <65536)

      • listen_addresses = 'localhost'

      • unix_socket_directories = '/tmp'

      • unix_socket_permissions = 0700

   4. Create and initialize the data directory. The data directory should be on the parallel file system:

   5. Try to start PostgreSQL:

   6. (while PostgreSQL is running) Create database:

   7. (while PostgreSQL is running) Configure PostgreSQL for AiiDA by running psql -p [port number]

   8. (while PostgreSQL is running) Test database connection:

2. Create an AiiDA profile

   • PostgreSQL engine [postgresql_psycopg2]:

   • PostgreSQL hostname [localhost]:

   • PostgreSQL port [5432]: [port number]

   • PostgreSQL username: aiida

   • PostgreSQL password: [passwort for user aiida chosen above]

   • PostgreSQL database name: aiida

   • File repository URI: A directory on the parallel file system. it needs to start with file:// , i.e., file:///scratch/hpc-prf-[project abbreviation]/...

Note that the PostgreSQL database needs to be running for AiiDA to work:

• When running AiiDA on one of the login nodes: Run the PostgreSQL database in a tmux or screen session on the same login node

• When running AiiDA on the PC2 Jupyter Hub: Run PostgreSQL database in a tmux or screen session in the console in Jupyter Lab instance

• When running AiiDA in a long-running compute job: Start PostgreSQL database in the job before AiiDA in the background, i.e.,

Create a Computer

To be able to submit calculations to a PC2 HPC system, create a computer in AiIDA by running the following on your local system where AiiDA is running:

Replace:

• [work dir] with a path to a directory on the parallel file system that AiiDA is going to use as a temporary working directory for your jobs, e.g., /scratch/hpc-prf-[project abbreviation]/{username}/AiiDA/scratch. {username} is automatically replaced by AiiDA with your user name.

Configure the Computer

Configure the computer for your AiIDA profile by running:

verdi -p [AiiDA profile name] computer configure core.local Noctua2

• Use login shell when executing command [Y/n]: n

• Connection cooldown time (s) [0.0]:

Testing the Computer

Run verdi computer test Noctua2 to test the configuration. In case of questions or problems or questions please don't hesitate to contact us via pc2-support@uni-paderborn.de.

Then you can follow the steps to create code and run a calculation above.