Ctrl+K

loadbalancer#

loadbalancer(maxcpu=0.9, maxmem=0.9, index=None, interval=None, cpu_interval=0.1, maxtime=36000, label=None, verbose=True, **kwargs)[source]#

Delay execution while CPU load is too high – a very simple load balancer.

Parameters:

  • maxcpu (float) – the maximum CPU load to allow for the task to still start

  • maxmem (float) – the maximum memory usage to allow for the task to still start

  • index (int) – the index of the task – used to start processes asynchronously (default None)

  • interval (float) – the time delay to poll to see if CPU load is OK (default 0.5 seconds)

  • cpu_interval (float) – number of seconds over which to estimate CPU load (default 0.1; too small gives inaccurate readings)

  • maxtime (float) – maximum amount of time to wait to start the task (default 36000 seconds (10 hours))

  • label (str) – the label to print out when outputting information about task delay or start (default None)

  • verbose (bool) – whether or not to print information about task delay or start (default True)

Examples:

# Simplest usage -- delay if CPU or memory load is >80%
sc.loadbalancer()

# Use a maximum CPU load of 50%, maximum memory of 90%, and stagger the start by process number
for nproc in processlist:
    sc.loadbalancer(maxload=0.5, maxmem=0.8, index=nproc)
New in version 2.0.0: maxmem argument; maxload renamed maxcpu
New in version 3.0.0: maxcpu and maxmem set to 0.9 by default
On this page