Re: Load average calculation?
On Mon, 3 Apr 2000, Richard Wackerbarth wrote: On Mon, 03 Apr 2000, Donn Miller wrote: I think we ought to re-examine the definition of load average. By load, we mean an actual load on the cpu, and waiting processes aren't really exerting a cpu load. So, by that reasoning I say waiting processes don't count. I think you have an incorrect (incomplete) definition. Traditionally, the load was the number of processes WANTING to run. Tasks which are waiting for "user" I/O are excluded. However, tasks waiting to be swapped in should be counted. This sounds to me like the right approach from point of view of a program like sendmail that reacts to load average. If you get in to a position in which the system is spending all of its time accessing the disc (so that the number of processes actually USING the CPU is quite low 'cos they are all waiting for pages to be swapped back in) is launching off yet more work /really/ the right thing to do? If all it does is result in even more thrashing then couldn't that actually *cut* the load average as calculated using the first of those definitions and push the system even further into the quagmire? OK, so you could easily argue that load average is a far from perfect measure for something of this sort but the second definition does sound more useful IMHO...it gives a better measure of the number of processes which can't yet complete their next chunk of work because of limited system resources. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: Load average calculation?
: a more accurate measure of load. : : :Ahh, and since nearly everything is done on this system via NFS, I can :imagine that several things are waiting for NFS responses. : :It's probably more accurate, but from a PR standpoint it makes it "look" :like FreeBSD is choking under the load, when it really isn't. Or am I the :only one that even cares about this? :) : :Kevin Heh. You can always hack your kernel source locally. I think the code you want to mess around with is /usr/src/sys/vm/vm_meter.c, the 'loadav' procedure. Replacing the 'FALLTHROUGH' comment with a 'break;' statement ought to do the trick for you. I don't think we should change anything in the official tree. -Matt Matthew Dillon [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: Load average calculation?
At 11:10 PM -0500 2000/4/2, Kevin Day wrote: It's probably more accurate, but from a PR standpoint it makes it "look" like FreeBSD is choking under the load, when it really isn't. Or am I the only one that even cares about this? :) It's also extremely confusing for Linux users/admins who are used to the system rolling over and dying if the load average ever gets over 2.0 (and panic'ing if the load average goes over 4.0), and who see FreeBSD capable of surviving (if not necessarily performing very well) with load averages as high as 100 or even 200. -- These are my opinions -- not to be taken as official Skynet policy == Brad Knowles, [EMAIL PROTECTED]|| Belgacom Skynet SA/NV Systems Architect, Mail/News/FTP/Proxy Admin || Rue Colonel Bourg, 124 Phone/Fax: +32-2-706.13.11/12.49 || B-1140 Brussels http://www.skynet.be || Belgium To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: Load average calculation?
On Sun, 2 Apr 2000 23:10:59 -0500 (CDT), Kevin Day [EMAIL PROTECTED] said: It's probably more accurate, but from a PR standpoint it makes it "look" like FreeBSD is choking under the load, when it really isn't. Actually, you have it backwards -- it makes it look as if FreeBSD is *not* choking under what appears to be a very heavy load -GAWollman -- Garrett A. Wollman | O Siem / We are all family / O Siem / We're all the same [EMAIL PROTECTED] | O Siem / The fires of freedom Opinions not those of| Dance in the burning flame MIT, LCS, CRS, or NSA| - Susan Aglukark and Chad Irschick To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: Load average calculation?
On Sun, 2 Apr 2000 23:10:59 -0500 (CDT), Kevin Day [EMAIL PROTECTED] said: It's probably more accurate, but from a PR standpoint it makes it "look" like FreeBSD is choking under the load, when it really isn't. Actually, you have it backwards -- it makes it look as if FreeBSD is *not* choking under what appears to be a very heavy load -GAWollman Well, my first impression was "Well, before doing this task the load average was only 0.20, now it's 4.0, obviously it can't keep up now." Which could probably be extended to "Under Linux the load average for running my database is only 0.20, FreeBSD's is 4.0, Linux must be faster." Granted it's flawed logic, but it's all a matter of perception at times. Kevin To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: Load average calculation?
:I'm not sure if this is -current fodder or not, but since it's still :happening in -current, I'll ask. : :We recently upgraded a server from 2.2.8 to 4.0(the same behavior is shown :on 5.0-current, too). Before, with the exact same load, we'd see load :averages from between 0.20 and 0.30. Now, we're getting: : :load averages: 4.16, 4.23, 4.66 : :Top shows the same CPU percentages, just a much higher load average for the :same work being done. Did the load average calculation change, or something :with the scheduler differ? Customers are complaining that the load average :is too high, which is kinda silly, since 4.0 seems noticably faster in some :cases. : :Any ideas? : :Kevin I believe the load average was changed quite a while ago to reflect not only runnable processes but also processes stuck in disk-wait. It's a more accurate measure of load. It's always been that way in BSD. -DG David Greenman Co-founder/Principal Architect, The FreeBSD Project - http://www.freebsd.org Creator of high-performance Internet servers - http://www.terasolutions.com Pave the road of life with opportunities. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: Load average calculation?
Brad Knowles wrote: At 11:10 PM -0500 2000/4/2, Kevin Day wrote: It's probably more accurate, but from a PR standpoint it makes it "look" like FreeBSD is choking under the load, when it really isn't. Or am I the only one that even cares about this? :) It's also extremely confusing for Linux users/admins who are used to the system rolling over and dying if the load average ever gets over 2.0 (and panic'ing if the load average goes over 4.0), and who see FreeBSD capable of surviving (if not necessarily performing very well) with load averages as high as 100 or even 200. Won't this also goof up programs like Exim (an SMTP MTA), that have some settings available for how to handle messages under various loads (process now, queue for later, etc)? Barry To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: Load average calculation?
At 1:11 PM -0500 2000/4/3, Barry Pederson wrote: Won't this also goof up programs like Exim (an SMTP MTA), that have some settings available for how to handle messages under various loads (process now, queue for later, etc)? If there has been an actual change in how the load average is calculated, then any program that changes it's behaviour based on the load average may have problems. This would certainly include SMTP MTAs such as sendmail, Exim, etc However, more recent comments lead me to question whether there actually has been a change in the way load is calculated. -- These are my opinions -- not to be taken as official Skynet policy == Brad Knowles, [EMAIL PROTECTED]|| Belgacom Skynet SA/NV Systems Architect, Mail/News/FTP/Proxy Admin || Rue Colonel Bourg, 124 Phone/Fax: +32-2-706.13.11/12.49 || B-1140 Brussels http://www.skynet.be || Belgium To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: Load average calculation?
Brad Knowles wrote: If there has been an actual change in how the load average is calculated, then any program that changes it's behaviour based on the load average may have problems. This would certainly include SMTP MTAs such as sendmail, Exim, etc I agree. IMO, the load avg. formula should stick as close as possible to those in previous releases of FreeBSD. OTOH, maybe those apps that need to query the load avg. are flawed anyways, as load avg. calculation tends to be system dependent. For example, FreeBSD, Linux, Solaris, SCO, etc. may all be running the exact same processes, but will the load avg. always be consistent across those platforms? I think not. - Donn To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: Load average calculation?
At 2:56 PM -0400 2000/4/3, Donn Miller wrote: For example, FreeBSD, Linux, Solaris, SCO, etc. may all be running the exact same processes, but will the load avg. always be consistent across those platforms? I think not. That's not a problem. Other OSes really aren't our problem, and you would expect to have different configuration files on them anyway. What *may* be a potential problem is if we change the way we calculate load average on our own OS -- that we *are* responsible for. -- These are my opinions -- not to be taken as official Skynet policy == Brad Knowles, [EMAIL PROTECTED]|| Belgacom Skynet SA/NV Systems Architect, Mail/News/FTP/Proxy Admin || Rue Colonel Bourg, 124 Phone/Fax: +32-2-706.13.11/12.49 || B-1140 Brussels http://www.skynet.be || Belgium To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: Load average calculation?
On Mon, Apr 03, 2000 at 02:56:32PM -0400, Donn Miller wrote: Brad Knowles wrote: If there has been an actual change in how the load average is calculated, then any program that changes it's behaviour based on the load average may have problems. This would certainly include SMTP MTAs such as sendmail, Exim, etc I agree. IMO, the load avg. formula should stick as close as possible to those in previous releases of FreeBSD. OTOH, maybe those apps that need to query the load avg. are flawed anyways, as load avg. calculation tends to be system dependent. On all Unix-like systems I know, the load average is the average mumber of processes running during a given time interval. I can't see what use it may have to count load for _waiting_ processes. I/O load is not process load, if a process waits for I/O completion it does not use up its timeslice. For example, FreeBSD, Linux, Solaris, SCO, etc. may all be running the exact same processes, but will the load avg. always be consistent across those platforms? I think not. I tend to disagree. - Donn cheers, Patrick To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: Load average calculation?
Patrick Mau wrote: On all Unix-like systems I know, the load average is the average mumber of processes running during a given time interval. I can't see what use it may have to count load for _waiting_ processes. I/O load is not process load, if a process waits for I/O completion it does not use up its timeslice. I think we ought to re-examine the definition of load average. By load, we mean an actual load on the cpu, and waiting processes aren't really exerting a cpu load. So, by that reasoning I say waiting processes don't count. - Donn To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: Load average calculation?
Richard Wackerbarth wrote: On Mon, 03 Apr 2000, Donn Miller wrote: I think we ought to re-examine the definition of load average. By load, we mean an actual load on the cpu, and waiting processes aren't really exerting a cpu load. So, by that reasoning I say waiting processes don't count. I think you have an incorrect (incomplete) definition. Traditionally, the load was the number of processes WANTING to run. Tasks which are waiting for "user" I/O are excluded. However, tasks waiting to be swapped in should be counted. Hmmm... I wonder how Linux computes load average? Although FreeBSD has nothing to do with Linux et. al., it would be good, IMO, to maintain some consistency with how those other unix variants determine the load avg. - Donn To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: Load average calculation?
Donn Miller wrote: Patrick Mau wrote: On all Unix-like systems I know, the load average is the average mumber of processes running during a given time interval. I can't see what use it may have to count load for _waiting_ processes. I/O load is not process load, if a process waits for I/O completion it does not use up its timeslice. I think we ought to re-examine the definition of load average. By load, we mean an actual load on the cpu, and waiting processes aren't really exerting a cpu load. So, by that reasoning I say waiting processes don't count. In this case, the "waiting" refers to the processes waiting in the _run_ queue, i.e., if the system wasn't so busy, they'd be running more often instead of waiting. In a uniprocessor system, only one process can be running at a time. When that currently running process is swapped away from the cpu, another process from the run queue is selected to run (which is actually an array of queues, lower indexes meaning higher priority). Load average is the average number of processes that are runnable, i.e., sitting in the run queue, over a certain period of time, usually computed for the most recent 1 minute, 5 minute, and 15 minute intervals. I think you guys are mixing up I/O wait with the run queue. The processes are indeed waiting, but they are waiting for a chance to get the CPU, not for someone to press a key on the keyboard or some other external event, without which, the process could not correctly continue until that event occurs. There is a _huge_ difference between the two. And yes, it is a pretty good, simple indicator of system load, and that is all it is really meant to be. If you need something fancier, or more specific to a particular load type, you'd need to roll your own using the abundant statistics that the kernel keeps and makes available. -Brian -- Brian Dean [EMAIL PROTECTED] SAS Institute Inc. [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: Load average calculation?
:We recently upgraded a server from 2.2.8 to 4.0(the same behavior is shown :on 5.0-current, too). Before, with the exact same load, we'd see load :averages from between 0.20 and 0.30. Now, we're getting: : :load averages: 4.16, 4.23, 4.66 : :Top shows the same CPU percentages, just a much higher load average for the :same work being done. Did the load average calculation change, or something :with the scheduler differ? Customers are complaining that the load average :is too high, which is kinda silly, since 4.0 seems noticably faster in some :cases. : :Any ideas? : :Kevin I believe the load average was changed quite a while ago to reflect not only runnable processes but also processes stuck in disk-wait. It's a more accurate measure of load. Ahh, and since nearly everything is done on this system via NFS, I can imagine that several things are waiting for NFS responses. It's probably more accurate, but from a PR standpoint it makes it "look" like FreeBSD is choking under the load, when it really isn't. Or am I the only one that even cares about this? :) Kevin To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: Load average calculation?
On Sun, Apr 02, 2000 at 11:10:59PM -0500, Kevin Day wrote: :We recently upgraded a server from 2.2.8 to 4.0(the same behavior is shown :on 5.0-current, too). Before, with the exact same load, we'd see load :averages from between 0.20 and 0.30. Now, we're getting: : :load averages: 4.16, 4.23, 4.66 : :Top shows the same CPU percentages, just a much higher load average for the :same work being done. Did the load average calculation change, or something :with the scheduler differ? Customers are complaining that the load average :is too high, which is kinda silly, since 4.0 seems noticably faster in some :cases. : :Any ideas? : :Kevin I believe the load average was changed quite a while ago to reflect not only runnable processes but also processes stuck in disk-wait. It's a more accurate measure of load. Ahh, and since nearly everything is done on this system via NFS, I can imagine that several things are waiting for NFS responses. It's probably more accurate, but from a PR standpoint it makes it "look" like FreeBSD is choking under the load, when it really isn't. Or am I the only one that even cares about this? :) What does the man page for 'w' say about it? At least the change should be reflected there I guess. -- Wilko Bulte Arnhem, The Netherlands http://www.tcja.nl The FreeBSD Project: http://www.freebsd.org To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: Load average calculation?
I believe the load average was changed quite a while ago to reflect not only runnable processes but also processes stuck in disk-wait. It's a more accurate measure of load. Ahh, and since nearly everything is done on this system via NFS, I can imagine that several things are waiting for NFS responses. It's probably more accurate, but from a PR standpoint it makes it "look" like FreeBSD is choking under the load, when it really isn't. Or am I the only one that even cares about this? :) What does the man page for 'w' say about it? At least the change should be reflected there I guess. getloadavg(3)(which 'w' and 'uptime' use) says: The getloadavg() function returns the number of processes in the system run queue averaged over various periods of time. The 'w' and 'uptime' manpages really don't mention anything relevant. Kevin To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: Load average calculation?
:I'm not sure if this is -current fodder or not, but since it's still :happening in -current, I'll ask. : :We recently upgraded a server from 2.2.8 to 4.0(the same behavior is shown :on 5.0-current, too). Before, with the exact same load, we'd see load :averages from between 0.20 and 0.30. Now, we're getting: : :load averages: 4.16, 4.23, 4.66 : :Top shows the same CPU percentages, just a much higher load average for the :same work being done. Did the load average calculation change, or something :with the scheduler differ? Customers are complaining that the load average :is too high, which is kinda silly, since 4.0 seems noticably faster in some :cases. : :Any ideas? : :Kevin I believe the load average was changed quite a while ago to reflect not only runnable processes but also processes stuck in disk-wait. It's a more accurate measure of load. -Matt Matthew Dillon [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
