Re: FBSD 6.2 Xeon 2.4ghz CPU and high load

2008-05-12 Thread DAve

DAve wrote:

Chuck Swiger wrote:
It might be reasonable to try hyperthreading enabled, as your type of 
load might be improved by it on




Funny that, enabling hyperthreading immediately dropped my load by half, 
I see CPU0, CPU1, CPU2, CPU3 now in top. I also see my CPU load 
reporting correctly as well. I see ranges from 10% idle to 80% idle, not 
locked at 50% and above.


That seems to have cured several ills. I will know more Monday at 8:30am 
when the business email traffic kicks in.


DAve


Just a quick note, we survived the day in good form. The servers have 
dropped their load numbers by 50% under a heavy load and by 80% under a 
normal load. More importantly, Nagios shows that SMTP is always 
responding and the load balancers are now showing a max of 34 active 
connections on each server where before they were showing 350+. 
Connections are opening and closing far far quicker.


machdep.hyperthreading_allowed=1 has been added to /etc/sysctl.conf

On a related note, I met Chuck back in 1999 in Seattle at a SeaFug 
meeting. I doubt he remembers me but he and John Polstra coached me 
through changing from a Mac Admin to a BSD admin. I've read Chuck's 
posts on multiple maillists that we both have, or do, share 
subscriptions to. Chuck, you are always helpful, never mean, and you 
encourage detective work to identify a problem rather than recite the 
upgrade mantra. Your knowledge has helped countless people over the 
years, including me. I appreciate that.


If you have a wish list, I can't find it. I would sure like to buy you a 
CD or something since I can't buy you a beer.


Thank you for your time, thanks to everyone on the list for their time.

DAve

--
In 50 years, our descendants will look back on the early years
of the internet, and much like we now look back on men with
rockets on their back and feathers glued to their arms, marvel
that we had the intelligence to wipe the drool from our chins.
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to [EMAIL PROTECTED]


Re: FBSD 6.2 Xeon 2.4ghz CPU and high load

2008-05-12 Thread Chuck Swiger

On May 12, 2008, at 2:27 PM, DAve wrote:
On a related note, I met Chuck back in 1999 in Seattle at a SeaFug  
meeting. I doubt he remembers me but he and John Polstra coached me  
through changing from a Mac Admin to a BSD admin. I've read Chuck's  
posts on multiple maillists that we both have, or do, share  
subscriptions to. Chuck, you are always helpful, never mean, and you  
encourage detective work to identify a problem rather than recite  
the upgrade mantra. Your knowledge has helped countless people  
over the years, including me. I appreciate that.


If you have a wish list, I can't find it.  I would sure like to buy  
you a CD or something since I can't buy you a beer.


Well, you're most welcome.  If you ever show up for one of the Apple  
events like a MacWorld or WWDC, you might run into me again...or at a  
Tommy's Tequila run, afterwards.  :-)


--
-Chuck

___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to [EMAIL PROTECTED]


Re: FBSD 6.2 Xeon 2.4ghz CPU and high load

2008-05-10 Thread Wojciech Puchar

and what most unix users do.


It is what a lot of unix users have done historically, but now that there is


and still most do.

It's not a Unix way versus Other OS Way thing -- its a response to the 
change
in direction hardware development has taken over the past several years. 
Chip


on multichip hardware you can do many different things too - even faster 
as it's spread over cores.



and
how much cache RAM there is on each chip.  4 cores and 8MB is just the latest
step in that evolutionary arms race.


that's much better than more gigaherts way.

any unix should support it good - with any kind of load.

today i see performance improvements are mostly towards synthetic 
benchmarks like running 8 threads of mysql server.


it looks cool on paper, but we need good performance when running 
concurrently many different things.


if one plan to use single one program - why unix at all?


as i've tested 7.0 once, it was on same computer noticably slower under 
high load of different programs.


now i read 6.* is slower than 4.* (i never user 4.*)

isn't it something wrong with it?!


It depends very much on the application load you have to support and the sort
of hardware you have available.  For the sort of multicore chips that are all 
the

rage nowadays, I'd go with 7.0 every time, even running single threaded
applications.


did you actually made a comparision with 6.*? not with paper benchmarks 
but just run 100 different things and check how responsive machine is.

___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to [EMAIL PROTECTED]


Re: FBSD 6.2 Xeon 2.4ghz CPU and high load

2008-05-10 Thread Matthew Seaman

Wojciech Puchar wrote:

It depends very much on the application load you have to support and 
the sort
of hardware you have available.  For the sort of multicore chips that 
are all the

rage nowadays, I'd go with 7.0 every time, even running single threaded
applications.


did you actually made a comparision with 6.*? not with paper 
benchmarks but just run 100 different things and check how responsive 
machine is.


My experience is of dealing with servers where each machine typically has
a small number of important applications -- frequently only /one/ application
-- which it has to run as efficiently as possible, and for a large number of
end-users.  The most telling example was a MySQL server which we originally
configured with 6.3 -- but it just collapsed under the full load when we made
it the back end for a popular web forums site.  Exactly the same hardware is in
use now running 7.0 and not only is that DB server cruising along quite happily,
but we've been able to add a bunch more web servers at the front of the site.
That's the most remarkable improvement I've seen, but it is not at all
untypical.

I can't speak to the model of needing to run hundreds of different
applications on the same server -- about the closest thing I have to that
is my personal laptop (but only dozens of apps, rather than hundreds), and
other than being vaguely aware that it seems to be working adequately, I've
never even tried to compare before and after performance.

Cheers,

Matthew


--
Dr Matthew J Seaman MA, D.Phil.   7 Priory Courtyard
 Flat 3
PGP: http://www.infracaninophile.co.uk/pgpkey Ramsgate
 Kent, CT11 9PW



signature.asc
Description: OpenPGP digital signature


Re: FBSD 6.2 Xeon 2.4ghz CPU and high load

2008-05-10 Thread Wojciech Puchar

but just run 100 different things and check how responsive machine is.


My experience is of dealing with servers where each machine typically has
a small number of important applications -- frequently only /one/ application


so why you need unix at all? :)



I can't speak to the model of needing to run hundreds of different


what is what i do. put everything on one server, only dividing things on 
many when one is unable to cope (very rare case).



applications on the same server -- about the closest thing I have to that
is my personal laptop (but only dozens of apps, rather than hundreds), and
other than being vaguely aware that it seems to be working adequately, I've


try as simple and stupid thing under load

cat /dev/zero somefile (on big partition)

on 6.* and 7.* and compare both cases.

:)
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to [EMAIL PROTECTED]


Re: FBSD 6.2 Xeon 2.4ghz CPU and high load

2008-05-10 Thread Matthew Seaman

Wojciech Puchar wrote:

but just run 100 different things and check how responsive machine is.


My experience is of dealing with servers where each machine typically has
a small number of important applications -- frequently only /one/ 
application


so why you need unix at all? :)


At the risk of belabouring the obvious:

  i) I like the price.  Free.

 ii) I like the operating environment -- CLIs aren't to everyone's taste,
 but I find they give me the freedom to do what I want without having
 to jump through a whole lot of hoops.

iii) I like the efficiency of the OS -- you get that much more performance
 out of every machine it's like having additional servers for free. 


try as simple and stupid thing under load

cat /dev/zero somefile (on big partition)

on 6.* and 7.* and compare both cases.

:)


I thought you were pillorying synthetic benchmarks upthread?  Filling up
a partition with a file of zeros is pretty unlikely as a real-world task.

I wouldn't be too disappointed if that didn't run as fast as it possibly
could, although I would be distinctly peeved if doing that on a loaded
server took up more of the system resources than it had any right or
justification to do, to the detriment of anything else running on that
machine.

Cheers,

Matthew

--
Dr Matthew J Seaman MA, D.Phil.   7 Priory Courtyard
 Flat 3
PGP: http://www.infracaninophile.co.uk/pgpkey Ramsgate
 Kent, CT11 9PW



signature.asc
Description: OpenPGP digital signature


Re: FBSD 6.2 Xeon 2.4ghz CPU and high load

2008-05-10 Thread Wojciech Puchar

At the risk of belabouring the obvious:

 i) I like the price.  Free.


no system is free too ;)


iii) I like the efficiency of the OS -- you get that much more performance
out of every machine it's like having additional servers for free.


single app writen for bare hardware would be the fastest.

you use unix only because software you use require it. simple.

___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to [EMAIL PROTECTED]


Re: FBSD 6.2 Xeon 2.4ghz CPU and high load

2008-05-10 Thread Mel
On Saturday 10 May 2008 09:10:37 Wojciech Puchar wrote:
  and what most unix users do.
 
  It is what a lot of unix users have done historically, but now that there
  is

 and still most do.

  It's not a Unix way versus Other OS Way thing -- its a response to
  the change
  in direction hardware development has taken over the past several years.
  Chip

 on multichip hardware you can do many different things too - even faster
 as it's spread over cores.

Do you realize your own arguments are in favor of moving to 7.x? Since the 
concurrency on 7.x with ULE has improved so much more, running multiprogram 
pipelines or completely different programs will improve as well.

And as a bonus you get improved threading for the programs that use them.

Secondly, the unix way would be the way that scales best and in practice, 
machines dedicated to one task scale easier then machines that do it all, 
especially since you can tune the hardware and kernel.
Thirdly, unix also got big, because it was able to split one task over 
multiple machines.

-- 
Mel

Problem with today's modular software: they start with the modules
and never get to the software part.
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to [EMAIL PROTECTED]


Re: FBSD 6.2 Xeon 2.4ghz CPU and high load

2008-05-09 Thread Matthew Seaman

DAve wrote:

Good morning.

I recently upgraded our two email gateways from 4.8 to 6.2. The required 
software was upgraded as well which consists of MailScanner and 
Sendmail. Both had been keep up to date so it was not a jump in required 
resources.


The issue I am seeing is that my server load, under the same traffic 
load, has increased 4 times or more. Where previously we saw a high load 
on the servers of 5 to 8, we are now seeing 14 to 17. Since the upgrade 
Sendmail has begun to timeout connections.


FreeBSD 6.2 is I believe slower than 4.11 for single processor systems
and processes which pretty much run single threaded -- ie. exactly what
you're trying to run.  This would cause exactly the sort of symptoms you're
seeing.

Try 7.0 instead -- it has all of the speed at multi-threaded, multi-core
type stuff but has also regained the sort of performance levels you could
get from 4.x at the sort of tasks 4.x is really good at.  You should evaluate
SCHED_4BSD vs. SCHED_ULE for your workload.  SCHED_4BSD is still the default
in 7.0, but SCHED_ULE gives better numbers for many workloads, and it only
missed being the default in 7.0 because it hadn't had enough time to settle
into the tree before the release.  SCHED_ULE will be the default from 7.1
onwards.

Cheers,

Matthew

--
Dr Matthew J Seaman MA, D.Phil.   7 Priory Courtyard
 Flat 3
PGP: http://www.infracaninophile.co.uk/pgpkey Ramsgate
 Kent, CT11 9PW



signature.asc
Description: OpenPGP digital signature


Re: FBSD 6.2 Xeon 2.4ghz CPU and high load

2008-05-09 Thread Chuck Swiger

On May 9, 2008, at 8:54 AM, DAve wrote:
The issue I am seeing is that my server load, under the same traffic  
load, has increased 4 times or more. Where previously we saw a high  
load on the servers of 5 to 8, we are now seeing 14 to 17. Since the  
upgrade Sendmail has begun to timeout connections.


You should look more into the status of the various processes, and how  
long it takes your mail scanning to process a message compared to  
previously.  It might be the case that the config under 6.2 is  
allowing more instances to run at once and is just barely nudging the  
system into excessive paging.  Once that happens, performance drops  
and the system load increases significantly.


Do a couple of ps aux | head -20 every 5 minutes or so, and put that  
data somewhere on a website, the process states will help give a  
better picture of what's going on.


[ ... ]

bash-2.05b# top
last pid: 85205;  load averages: 12.89, 13.78,  
14.66   
up 47+15:51:31  15:20:01

126 processes: 12 running, 79 sleeping, 35 zombie
CPU states: 43.8% user,  0.0% nice,  6.3% system,  0.0% interrupt,  
50.0% idle
Mem: 1008M Active, 582M Inact, 211M Wired, 78M Cache, 112M Buf, 122M  
Free

Swap: 4096M Total, 304M Used, 3792M Free, 7% Inuse

I am suspicious of the kernel being the culprit because the system  
looks as if it is not working very hard, CPU load never shows above  
50% idle. I found one thread which mentions that as an issue and  
offers a patch.


http://lists.freebsd.org/pipermail/freebsd-bugs/2007-February/022526.html

Currently I am running the SMP-GENERIC kernel and sysctl shows the  
following.


hw.model: Intel(R) Xeon(TM) CPU 2.40GHz
machdep.hlt_logical_cpus: 0
machdep.hyperthreading_allowed: 0
kern.smp.cpus: 4
I see dev.cpu.0 through dev.cpu.3

Can anyone offer a solution? Is this a known issue I can easily  
correct? At this point I am left with either rolling back to 4.11 or  
trying another OS.


It might be reasonable to try hyperthreading enabled, as your type of  
load might be improved by it on


--
-Chuck

___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to [EMAIL PROTECTED]


Re: FBSD 6.2 Xeon 2.4ghz CPU and high load

2008-05-09 Thread Wojciech Puchar
software was upgraded as well which consists of MailScanner and Sendmail. 
Both had been keep up to date so it was not a jump in required resources.


The issue I am seeing is that my server load, under the same traffic load, 
has increased 4 times or more. Where previously we saw a high load on the 
servers of 5 to 8, we are now seeing 14 to 17. Since the upgrade Sendmail has 
begun to timeout connections.


do you feel that system goes slower?
i think it's just the matter of calculation method - 6.* may calculate it 
different way.


just change in your sendmail config the values in place of xx

define(`confQUEUE_LA', `xx')
define(`confREFUSE_LA', `xx')


as just accepting mail isn't a problem i set confREFUSE_LA very high
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to [EMAIL PROTECTED]


Re: FBSD 6.2 Xeon 2.4ghz CPU and high load

2008-05-09 Thread Wojciech Puchar

FreeBSD 6.2 is I believe slower than 4.11 for single processor systems
and processes which pretty much run single threaded -- ie. exactly what
you're trying to run.  This would cause exactly the sort of symptoms you're
seeing.


and what most unix users do.


Try 7.0 instead -- it has all of the speed at multi-threaded, multi-core
type stuff but has also regained the sort of performance levels you could


so 4.11 is fastest?
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to [EMAIL PROTECTED]


Re: FBSD 6.2 Xeon 2.4ghz CPU and high load

2008-05-09 Thread Chuck Swiger

On May 9, 2008, at 11:30 AM, Wojciech Puchar wrote:
Try 7.0 instead -- it has all of the speed at multi-threaded, multi- 
core
type stuff but has also regained the sort of performance levels you  
could


so 4.11 is fastest?


For single-processor systems, FreeBSD 4.11 does very well at a lot of  
tasks.  However, Dave apparently has a 4-CPU system (~8 threads if he  
enabled hyperthreading), and for real SMP hardware, more recent  
versions of FreeBSD generally perform better than 4.x would.


--
-Chuck

___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to [EMAIL PROTECTED]


Re: FBSD 6.2 Xeon 2.4ghz CPU and high load

2008-05-09 Thread DAve

Wojciech Puchar wrote:
software was upgraded as well which consists of MailScanner and 
Sendmail. Both had been keep up to date so it was not a jump in 
required resources.


The issue I am seeing is that my server load, under the same traffic 
load, has increased 4 times or more. Where previously we saw a high 
load on the servers of 5 to 8, we are now seeing 14 to 17. Since the 
upgrade Sendmail has begun to timeout connections.


do you feel that system goes slower?
i think it's just the matter of calculation method - 6.* may calculate 
it different way.


just change in your sendmail config the values in place of xx

define(`confQUEUE_LA', `xx')
define(`confREFUSE_LA', `xx')


as just accepting mail isn't a problem i set confREFUSE_LA very high


It is already set to higher than the load we see. I don't see sendmail 
refusing connections. What happens is I try to test sendmail from 
another server and the connection never completes. I'm knockin', 
sendmail ain't answering.


DAve


--
In 50 years, our descendants will look back on the early years
of the internet, and much like we now look back on men with
rockets on their back and feathers glued to their arms, marvel
that we had the intelligence to wipe the drool from our chins.
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to [EMAIL PROTECTED]


Re: FBSD 6.2 Xeon 2.4ghz CPU and high load

2008-05-09 Thread DAve

Wojciech Puchar wrote:

FreeBSD 6.2 is I believe slower than 4.11 for single processor systems
and processes which pretty much run single threaded -- ie. exactly what
you're trying to run.  This would cause exactly the sort of symptoms 
you're

seeing.


and what most unix users do.


Try 7.0 instead -- it has all of the speed at multi-threaded, multi-core
type stuff but has also regained the sort of performance levels you could


so 4.11 is fastest?


I would be inclined to try another version if I knew what the cause of 
this issue was exactly, and I saw in the release notes that the issue 
was resolved in 7.X. But I cannot just try a new version on a production 
server as an experiment. I've hosed this up enough thinking 6.2 was out 
long enough to not surprise me.


I've not compared them on any server running multiple CPUs, but on a 
single physical CPU server I've yet to see 5.X or 6.X keep up with 4.X. 
I've been poo poo'd heartily for saying so, more than once.


I would hope, and I do think, this is easily solved. I've already had 
one private email stating a binary upgrade to 6.3 solved the same 
problem for them. I wish I could find that email again 8^(


DAve


--
In 50 years, our descendants will look back on the early years
of the internet, and much like we now look back on men with
rockets on their back and feathers glued to their arms, marvel
that we had the intelligence to wipe the drool from our chins.
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to [EMAIL PROTECTED]


Re: FBSD 6.2 Xeon 2.4ghz CPU and high load

2008-05-09 Thread DAve

Chuck Swiger wrote:

On May 9, 2008, at 11:30 AM, Wojciech Puchar wrote:

Try 7.0 instead -- it has all of the speed at multi-threaded, multi-core
type stuff but has also regained the sort of performance levels you 
could


so 4.11 is fastest?


For single-processor systems, FreeBSD 4.11 does very well at a lot of 
tasks.  However, Dave apparently has a 4-CPU system (~8 threads if he 
enabled hyperthreading), and for real SMP hardware, more recent versions 
of FreeBSD generally perform better than 4.x would.


Single CPU quad core.

ps -aux output is up, look under the FBSD dir. I also put up both 
dmesg.boot files from the servers.


http://pixelhammer.com/Dan/

I do appreciate the assistance.

DAve


--
In 50 years, our descendants will look back on the early years
of the internet, and much like we now look back on men with
rockets on their back and feathers glued to their arms, marvel
that we had the intelligence to wipe the drool from our chins.
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to [EMAIL PROTECTED]


Re: FBSD 6.2 Xeon 2.4ghz CPU and high load

2008-05-09 Thread Chuck Swiger

On May 9, 2008, at 11:55 AM, DAve wrote:
For single-processor systems, FreeBSD 4.11 does very well at a lot  
of tasks.  However, Dave apparently has a 4-CPU system (~8 threads  
if he enabled hyperthreading), and for real SMP hardware, more  
recent versions of FreeBSD generally perform better than 4.x would.


Single CPU quad core.


OK.

ps -aux output is up, look under the FBSD dir. I also put up both  
dmesg.boot files from the servers.


MailScanner is what is taking up all of the load; tuning that area is  
where you need to focus.


Things which come to mind are trying to limit the max number of  
children of that being run to something smaller, perhaps 8 or so.   
Yes, they recommend running 5 * #CPUs, but they also think their  
instances are going to be around 20MB in size, but yours are running  
at 100+ MB size.


You might find that running sa-update and sa-compile nightly might  
improve your SpamAssassin performance; I've got a crontab setup which  
runs the following nightly:


% cat /usr/local/bin/update-spamassassin
#! /bin/sh

PATH=$PATH:/usr/local/bin

sa-update --allowplugins --gpgkey  
D1C035168C1EBC08464946DA258CDB3ABDE9DC10 --channel  
saupdates.openprotect.com --channel updates.spamassassin.org

sa-compile

kill -HUP `cat /var/run/vscan/spamd.pid`

(If you aren't running spamd because MailScanner uses builtin  
interface to SpamAssassin, comment out the last line.  But do check  
the sa-compile docs, you have to make a change for it to be used)


Regards,
--
-Chuck

___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to [EMAIL PROTECTED]


Re: FBSD 6.2 Xeon 2.4ghz CPU and high load

2008-05-09 Thread DAve

Chuck Swiger wrote:

On May 9, 2008, at 11:55 AM, DAve wrote:
For single-processor systems, FreeBSD 4.11 does very well at a lot of 
tasks.  However, Dave apparently has a 4-CPU system (~8 threads if he 
enabled hyperthreading), and for real SMP hardware, more recent 
versions of FreeBSD generally perform better than 4.x would.


Single CPU quad core.


OK.

ps -aux output is up, look under the FBSD dir. I also put up both 
dmesg.boot files from the servers.


MailScanner is what is taking up all of the load; tuning that area is 
where you need to focus.


Things which come to mind are trying to limit the max number of children 
of that being run to something smaller, perhaps 8 or so.  Yes, they 
recommend running 5 * #CPUs, but they also think their instances are 
going to be around 20MB in size, but yours are running at 100+ MB size.


You might find that running sa-update and sa-compile nightly might 
improve your SpamAssassin performance; I've got a crontab setup which 
runs the following nightly:


% cat /usr/local/bin/update-spamassassin
#! /bin/sh

PATH=$PATH:/usr/local/bin

sa-update --allowplugins --gpgkey 
D1C035168C1EBC08464946DA258CDB3ABDE9DC10 --channel 
saupdates.openprotect.com --channel updates.spamassassin.org

sa-compile

kill -HUP `cat /var/run/vscan/spamd.pid`

(If you aren't running spamd because MailScanner uses builtin interface 
to SpamAssassin, comment out the last line.  But do check the sa-compile 
docs, you have to make a change for it to be used)


Regards,


I appologize I should have given more info.

We do run sa-update, and sa-compile. We also run 0 scores on most DNSBL 
tests as we run those at the mta level along with milter-greylist, 
milter-ahead, pipelining rejection, and greet pause. We have been 
running a very trimmed down and fine tuned system for about two years 
now with good results. I do think the upgrade to SA 3.2.4 is very heavy, 
considerably more resource usage than 3.1.8 which we were running prior 
to the OS upgrade.


I have not changed the settings for MailScanner from our previous 
install with respect to number of children or to batch size. Previous 
testing showed that 13 MS children with a batch size of 10 messages was 
optimal. I can certainly give that a try.  I will look at enabling 
Hyperthreading as well.


I've also found this, which may be a clue to the suggestion that a 
binary upgrade to 6.3 was a solution.


DAve
http://lists.freebsd.org/pipermail/freebsd-current/2007-April/070986.html

--
In 50 years, our descendants will look back on the early years
of the internet, and much like we now look back on men with
rockets on their back and feathers glued to their arms, marvel
that we had the intelligence to wipe the drool from our chins.
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to [EMAIL PROTECTED]


Re: FBSD 6.2 Xeon 2.4ghz CPU and high load

2008-05-09 Thread DAve

Chuck Swiger wrote:

On May 9, 2008, at 8:54 AM, DAve wrote:
The issue I am seeing is that my server load, under the same traffic 
load, has increased 4 times or more. Where previously we saw a high 
load on the servers of 5 to 8, we are now seeing 14 to 17. Since the 
upgrade Sendmail has begun to timeout connections.


You should look more into the status of the various processes, and how 
long it takes your mail scanning to process a message compared to 
previously.  It might be the case that the config under 6.2 is allowing 
more instances to run at once and is just barely nudging the system into 
excessive paging.  Once that happens, performance drops and the system 
load increases significantly.


Do a couple of ps aux | head -20 every 5 minutes or so, and put that 
data somewhere on a website, the process states will help give a better 
picture of what's going on.


[ ... ]

bash-2.05b# top
last pid: 85205;  load averages: 12.89, 13.78, 
14.66  up 
47+15:51:31  15:20:01

126 processes: 12 running, 79 sleeping, 35 zombie
CPU states: 43.8% user,  0.0% nice,  6.3% system,  0.0% interrupt, 
50.0% idle

Mem: 1008M Active, 582M Inact, 211M Wired, 78M Cache, 112M Buf, 122M Free
Swap: 4096M Total, 304M Used, 3792M Free, 7% Inuse

I am suspicious of the kernel being the culprit because the system 
looks as if it is not working very hard, CPU load never shows above 
50% idle. I found one thread which mentions that as an issue and 
offers a patch.


http://lists.freebsd.org/pipermail/freebsd-bugs/2007-February/022526.html

Currently I am running the SMP-GENERIC kernel and sysctl shows the 
following.


hw.model: Intel(R) Xeon(TM) CPU 2.40GHz
machdep.hlt_logical_cpus: 0
machdep.hyperthreading_allowed: 0
kern.smp.cpus: 4
I see dev.cpu.0 through dev.cpu.3

Can anyone offer a solution? Is this a known issue I can easily 
correct? At this point I am left with either rolling back to 4.11 or 
trying another OS.


It might be reasonable to try hyperthreading enabled, as your type of 
load might be improved by it on




Funny that, enabling hyperthreading immediately dropped my load by half, 
I see CPU0, CPU1, CPU2, CPU3 now in top. I also see my CPU load 
reporting correctly as well. I see ranges from 10% idle to 80% idle, not 
locked at 50% and above.


That seems to have cured several ills. I will know more Monday at 8:30am 
when the business email traffic kicks in.


DAve


--
In 50 years, our descendants will look back on the early years
of the internet, and much like we now look back on men with
rockets on their back and feathers glued to their arms, marvel
that we had the intelligence to wipe the drool from our chins.
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to [EMAIL PROTECTED]


Re: FBSD 6.2 Xeon 2.4ghz CPU and high load

2008-05-09 Thread Matthew Seaman

Wojciech Puchar wrote:

FreeBSD 6.2 is I believe slower than 4.11 for single processor systems
and processes which pretty much run single threaded -- ie. exactly what
you're trying to run.  This would cause exactly the sort of symptoms 
you're

seeing.


Actually I was mistaken: I saw 4.11 and 2.4GHz Xeon and assumed the OP was using
2004-era hardware.  The whole Quad Core thing just didn't register.


and what most unix users do.


It is what a lot of unix users have done historically, but now that there is 
good
support coming through for highly threaded, parallelized applications, 
developers
are going to write more and users are going to run more applications that 
exploit
that.

It's not a Unix way versus Other OS Way thing -- its a response to the 
change
in direction hardware development has taken over the past several years.   Chip
manufacturers have all but given up on the race to outdo each other on the MHz
or GHz rating of their products.  Nowadays it's all about how many CPU cores and
how much cache RAM there is on each chip.  4 cores and 8MB is just the latest
step in that evolutionary arms race.  


Try 7.0 instead -- it has all of the speed at multi-threaded, multi-core
type stuff but has also regained the sort of performance levels you could


so 4.11 is fastest?


It depends very much on the application load you have to support and the sort
of hardware you have available.  For the sort of multicore chips that are all 
the
rage nowadays, I'd go with 7.0 every time, even running single threaded
applications.

Cheers,

Matthew

--
Dr Matthew J Seaman MA, D.Phil.   7 Priory Courtyard
 Flat 3
PGP: http://www.infracaninophile.co.uk/pgpkey Ramsgate
 Kent, CT11 9PW



signature.asc
Description: OpenPGP digital signature