Re: Linux /proc/loadavg

[Paul Smith]

On Sat, 2022-01-15 at 10:29 +, Sven C. Dack wrote:
> I have recently been looking into this again, because of a new
> feature I am currently testing, and so far can only repeat what I
> have already said in the past. The /proc/loadavg file under Linux
> with a vanilla kernel is behaving as expected, meaning, the number of
> active processes can very well exceed the number of cores.

I spent some time yesterday building a large codebase with different
options and configurations.

I agree with your assessment: the value here is the number of runnable
jobs, not the number of running jobs.  I'm not exactly sure where the
original issue came from, so I'll fix this for the next release.

Re: Linux /proc/loadavg (graphs)

[Sven C. Dack]

Here are some graphs. These show the difference when using Linux
/proc/loadavg. Red on the left shows the default behaviour of make-4.3
when using the -l option, green on the right shows the modified
behaviour. The upper graphs show the number of processes on the system,
the lower graphs show the memory use.

https://i.imgur.com/9oshAfc.png


On 15/01/2022 10:29, Sven C. Dack wrote:

Hello,

the ChangeLog says the following:

2019-12-17  Paul Smith 

...

    * src/job.c (load_too_high): Disable Linux /proc/loadavg for now
    This new feature has a problem: if you provide a load which is
    larger than the number of cores then it will always run every job.
    Before we can enable it we need to at the least learn how to clamp
    this value to the number of cores.  To experiment with it, set
    PROC_FD_INIT to -2 in job.c to re-enable the feature.


I have recently been looking into this again, because of a new feature I
am currently testing, and so far can only repeat what I have already
said in the past. The /proc/loadavg file under Linux with a vanilla
kernel is behaving as expected, meaning, the number of active processes
can very well exceed the number of cores.

To quote from the Linux source code documentation under
Documentation/filesystems/proc.rst:

 loadavg  Load average of last 1, 5 & 15 minutes;
    number of processes currently runnable (running or on
ready queue);
    total number of processes in system;
    last pid created.
    All fields are separated by one space except "number of
    processes currently runnable" and "total number of
processes
    in system", which are separated by a slash ('/').
Example:
    0.61 0.61 0.55 3/828 22084

Here it states that it is the number of running or ready processes. Any
implementation of /proc/loadavg which counts only one running process
per core is effectively counting the number of active cores, but not the
number of running or runnable processes, and should therefore be
considered buggy or deviant.

Thus, when there is a deviant implementation of /proc/loadavg out there,
by a modified Linux kernel or perhaps another OS, then we should gate
this off either through autoconf or a simple #ifdef linux #endif.

Sven