Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Tip
titleJob array

This method is preferred over multiple submissions (e.g. with a for loop) because:

  • reduces job queue load - each job will compete for resources simultaneously with everyone else in the queue, instead of one after the other
  • easier management - modification of all jobs is possible by calling the main (e.g. 14575[]) or individual (e.g. 14575[3]) job identifier

The environment variables defined by PBS during their execution are:

  • PBS_ARRAY_INDEX - number of sub-jobs in the job array (e.g. one to nine in the example above)
  • PBS_ARRAY_ID - identifier of the main job field
  • PBS_JOBID - subjob identifier in the job field

...

NameDescription
PBS_JOBIDJob identifier provided by PBS when a job is submitted. Created after executing the qsub command
PBS_JOBNAMEThe name of the job provided by the user. The default name is the name of the submitted script
PBS_NODEFILEList of work nodes, or processor cores on which the job is executed
PBS_O_WORKDIRThe working directory in which the job was submitted, i.e. in which qsub command was called
OMP_NUM_THREADSAn OpenMP variable that PBS exports to the environment, which is equal to the value of the ncpus option from the PBS script header
NCPUSNumber of cores requested. Matches the value from the ncpus option from the PBS script header
TMPDIRPath to temporary directory

...

Tip
titleSpecifying the working directory

While in PBS the path for the output and error files is specified in the directory in which they are executed, the input and output files of the program itself are loaded/saved in the $HOME directory by default. PBS does not have an option to specify the job execution in the current directory we are in, so it is necessary to change the directory manually.

If you want to switch to the directory where the script was started, after the header you have to write:

cd $PBS_O_WORKDIR

If you want to run jobs with high storage load (I/O intensive) job execution is not recommended to run from $PBS_O_WORKDIR-a but from $TMPDIR location, which will use fast storage. Read more about using fast storage and temporary results below.

...

PBS makes it possible to define the necessary resources in several ways. The main unit for resource allocation is the so-called "Chunk" or piece of node. A chunk is defined with the select option. The number of processor cores per chunk can be defined with ncpus, the number of mpi processes with mpiprocs and the amount of working memory with mem. It is also possible to define walltime (maximum job execution time) and place (chunk allocation method by nodes).

...

In this case, the user gets 4 processor cores and a total of 14GB of memory on one chunk. When jobs are described without the select option, it is not possible to "chain resources" (separate the required resources with a colon, it is necessary to put the -l option on a new line for each resource)

...

Tip

OpenMP applications require the variable OMP_NUM_THREADS to be defined. The PBS system automatically associates it with the value of the ncpus variable, defined in the header of the PBS script.

The value of the variable select  from from the header of the PBS script corresponds to the number of the MPI process.

...

Code Block
languagebash
qsub -W depend=<type>:<job_ID>[:<job_ID>] ...

Where <type> can  can be:

  • after* - starting the current one with respect to the others
    • after - execution of the current one after the start of execution of the specified ones
    • afterok - execution of the current one after the successful completion of the specified ones
    • afternotok -execution of the current after an error in the completion of the specified
    • afterany - execution of the current one after the end of the specified ones
  • before* - starting the others with respect to the current one
    • before - execution of the specified ones after the start of the current one
    • beforeok - execution of the specified ones after the successful completion of the current one
    • beforenotok - execution of the specified ones after the an error in the completion of the current one
    • beforeany - execution of the specified ones after the end of the current one
  • on:<number> - execution of a job that will depend on the subsequently specified number of before* types  types of jobs
Note
A job with a directive -W depend=... will not be submitted if the specified job IDs do not exist (or if they are not in a queue)

...