...
The config should look similar to this when you are done
| Code Block |
|---|
concurrent = 1 check_interval = 0 shutdown_timeout = 0 [general] executor = "slurm" data_dir = "/scratch/PATH/TO/WORK/DIR/.../data" concurrent = 1 check_interval = 0 shutdown_timeout = 0 [session_server] session_timeout = 1800 [[runners]] name = "Jacamar Test Runner" url = "https://git.uni-paderborn.de/" token = "TOKEN" executor = "custom" limit = 0 request_concurrency = 1 environment = ["PATH=/opt/software/pc2/EB-SW/software/gitlab-runner/latest/bin:/opt/software/pc2/EB-SW/software/gitlab-runner/latest/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin"] pre_get_sources_script="module reset" [runners.cache] MaxUploadedArchiveSize = 0 [runners.custom] config_exec_timeout = 3600 config_exec = "/opt/software/pc2/EB-SW/software/jacamar/latest/bin/jacamar" config_args = ["--no-auth", "config", "--configuration", "/scratch/PATH/TO/WORK/DIR/jacamar-config.toml"] prepare_exec = "/opt/software/pc2/EB-SW/software/jacamar/latest/bin/jacamar" prepare_args = ["--no-auth", "prepare"] run_exec = "/opt/software/pc2/EB-SW/software/jacamar/latest/bin/jacamar" run_args = ["--no-auth", "run"] cleanup_exec = "/opt/software/pc2/EB-SW/software/jacamar/latest/bin/jacamar" cleanup_args = ["--no-auth", "cleanup", "--configuration", "/scratch/PATH/TO/WORK/DIR/jacamar-config.toml"] |
...
To test the GitLab runner, execute it with the jacamar configuration:
...
As long as your GitLab runner is executed on Noctua, it will process the CI jobs from your project.
Concurrent CI job execution
There are multiple layers of concurrency in the runner configuration. You can define multiple [[runners]] sections to let Jacamar create multiple runners with different configurations, but this is not required for concurrency.
The concurrent variable at the top defines the total limit for all runners combined. Further, each runner has two variables in its section: limit and request_concurrency. Both limit the number of concurrent CI jobs a runner will execute.
For example, if you have 1 runner, set concurrent to 5, set limit to 0 to deactivate the limit variable and set request_concurrency to 5 to execute at most 5 CI jobs concurrently.
You can find more information in the respective documentation of Jacamar variables and Gitlab Runner variables.
4. Add CI file to repository
...
In the .gitlab-ci.yml you will need to specify the variable SCHEDULER_PARAMETERS to make it work with our Slurm installation. In this variable, you should specify your project account and the partition where the jobs should be executed.
Also the id_tokens variable has to be specified as demonstrated in the example (context).
Example
| Code Block |
|---|
test:
stage: build
id_tokens:
CI_JOB_JWT:
aud: https://git.uni-paderborn.de
variables:
SCHEDULER_PARAMETERS: "-A PROJECT_ACCOUNT -p normal -t 0:05:00"
script:
- echo "Hello from " $(hostname) |
...
Shared runners can only be created for the whole Gitlab instance. Group runner can be created for groups where the owner status is available. In every other case register a runner and create a service file for each repository.
Troubleshooting
...