Alexey Popov wrote:
Hi
Kris Kennaway wrote:
So I can conclude that FreeBSD has a long standing bug in VM that
could be triggered when serving large amount of static data (much
bigger than memory size) on high rates. Possibly this only
applies to large files like mp3 or video.
It is possible, we have further work to do to conclude this though.
I forgot to mention I have pmc and kgmon profiling for good and bad
times. But I have not enough knowledge to interpret it right and
not sure if it can help.
pmc would be useful.
pmc profiling attached.
OK, the pmc traces do seem to show that it's not a lock contention
issue. That being the case I don't think the fact that different
servers perform better is directly related.
But it was evidence of mbuf lock contention in mutex profiling, wasn't
it? As far as I understand, mutex problems can exist without increasing
CPU load in pmc stats, right?
No, the lock functions will show up as using a lot of CPU. I guess the
lock profiling trace showed high numbers because you ran it for a long time.
There is also no evidence of a VM problem. What your vmstat and pmc
traces show is that your system really isn't doing much work at all,
relatively speaking.
There is also still no evidence of a disk problem. In fact your disk
seems to be almost idle in both cases you provided, only doing between
1 and 10 operations per second, which is trivial.
vmstat and network output graphs shows that the problem exists. If it is
not a disk or network or VM problem, what else could be wrong?
The vmstat output you provided so far doesn't show anything specific.
In the "good" case you are getting a much higher interrupt rate but
with the data you provided I can't tell where from. You need to run
vmstat -i at regular intervals (e.g. every 10 seconds for a minute)
during the "good" and "bad" times, since it only provides counters and
an average rate over the uptime of the system.
I'll try this, but AFAIR there was no strangeness with interrupts.
I believe the reason of high interrupt rate in "good" cases is that
server sends much traffic.
What there is evidence of is an interrupt aliasing problem between em
and USB:
irq16: uhci0 1464547796 1870
irq64: em0 1463513610 1869
I tried disabling USB in kernel, this ussie was gone, but the main
problem was left. Also I have this issue with interrupt aliasing on many
servers without problems.
OK.
Kris
_______________________________________________
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"
