w11 - /man/cat1/njobihtm.0

NJOBIHTM(1)                  Retro Project Manual                  NJOBIHTM(1)


NAME
       njobihtm - number of jobs considering hyper-threading and memory

SYNOPSIS
       njobihtm [-h] [-m nnn[MG]] [-v]

DESCRIPTION
       The  standard  system  nproc(1)  command is often used to determine the
       number of parallel jobs, e.g. in a 'make -j'.  It  simply  returns  the
       total  number  of  available  threads.   On many processors with hyper-
       threading the usage of all hyper-threads is contra-productive  and  can
       lead  to a reduction of the throughput.  Experience showed that the op-
       timum throughput is reached when one quarter of the  hyper-threads  are
       used.  If  jobs  require  a  substantial amount of memory this can also
       limit the number of parallel jobs.

       njobihtm determines the number of CPU-intensive jobs based on the  num-
       ber of CPU and memory resources. The 'ihtm' stands for 'intelligent hy-
       per-threading and memory'. The script

         - determines the number of physical cores and the number  of  threads
           per core.
         - adds a quarter of the hyper-threads unless the -n option is given.
         - if  -m  is given, determines the memory size, assumes that at least
           one GB should be available for general usage, and limits the number
           of jobs accordingly.

       The number of jobs is written to stdout, and can be used like nproc(1).

OPTIONS
       -n       ignore hyper-threads, use only physical cores.

       -m size  gives  the  required  physical  memory  per job.  size must be
                given as an integer with either an 'M' or 'G',  indicating  MB
                or GB.

       -v       the  found  system  parameters and the reasoning is printed to
                stderr.

EXAMPLES
       make -j `njobihtm` all
           Start make(1) with a reasonable number of jobs.

       njobihtm -v -m=2G
           Determines the number of jobs with 2 GB memory per job. On a system
           with  4  cores  and  hyper-threading and 32 GB installed memory one
           gets due to the -v the output
               #cpus:        8
               #thread/cpu:  2
               #cores:       4
               #mem(MB):     31731
               #job (cpus):  5
               #job (mem):   14
               5
           Note that the -v output goes to stderr, only the answer '5' to std-
           out.

       njobihtm -v -n -m=3G
           Determines the number of jobs with 3 GB memory per job. Only physi-
           cal cores are used. On a system with 4 cores and  32  GB  installed
           memory one gets due to the -v the output
               #cpus:        8
               #thread/cpu:  2
               #cores:       4
               #mem(MB):     31731
               #job (cpus):  4
               #job (mem):   9
               4


AUTHOR
       Walter F.J. Mueller <W.F.J.Mueller@gsi.de>


Retro Project                     2022-06-03                       NJOBIHTM(1)