Re: debugging mbuf allocation/dealocation

2008-12-29 Thread Nikola Knežević
You can use the KTR(4) facility to trace memory allocations and deallocations, logging them to memory, disk, etc. Unfortunately interpreting the data can be fairly tricky, as network leaks tend to happen over a long period of time, be stored in sockets, etc, but it's definitely possible

Re: How process size is calculated? Is it always based on the current highest available address in memory space?

2008-12-29 Thread Dan Nelson
In the last episode (Dec 28), Yuri said: malloc(3) can be controlled by MALLOC_OPTIONS to use mmap-based allocation as opposed to sbrk-based allocation. But allocations/deallocations with mmaps can eventually lead to non-continuously mmapped memory (having some non-mmapped gaps). Are these

Re: How process size is calculated? Is it always based on the current highest available address in memory space?

2008-12-29 Thread Robert Watson
On Mon, 29 Dec 2008, Dan Nelson wrote: In the last episode (Dec 28), Yuri said: malloc(3) can be controlled by MALLOC_OPTIONS to use mmap-based allocation as opposed to sbrk-based allocation. But allocations/deallocations with mmaps can eventually lead to non-continuously mmapped memory

lzo2 shows insane speed gap

2008-12-29 Thread Christian Weisgerber
The archivers/lzo2 port runs a series of regression tests after the actual build. These tests show extremely divergent behavior on different machines. There are two types of machines: Type #1: Running the tests takes roughly the same time as configure and compile did, whether it's 30

Re: lzo2 shows insane speed gap

2008-12-29 Thread Dimitry Andric
On 2008-12-29 22:25, Christian Weisgerber wrote: On the slow machines, the tests also consume a lot of system time. I've seen figures from 20 to 50%. However, ktrace shows nothing out of the ordinary. What's up with the memory on these machines? Lzo tends to take insane amounts

Re: lzo2 shows insane speed gap

2008-12-29 Thread Dimitry Andric
On 2008-12-30 00:17, Dimitry Andric wrote: What's up with the memory on these machines? Lzo tends to take insane amounts Duh, nevermind... I'm confusing this with lzma. :) Sorry for the noise. ___ freebsd-hackers@freebsd.org mailing list

Re: lzo2 shows insane speed gap

2008-12-29 Thread Nate Eldredge
On Mon, 29 Dec 2008, Christian Weisgerber wrote: The archivers/lzo2 port runs a series of regression tests after the actual build. These tests show extremely divergent behavior on different machines. There are two types of machines: Type #1: Running the tests takes roughly the same time as

Re: lzo2 shows insane speed gap

2008-12-29 Thread Yuri
Christian Weisgerber wrote: skipped My best guess at this time is that lzo2 somehow manages to induce crazy cache thrashing on some CPU models. Ideas and explanations welcome. Did you ask the author? He might be the best person to ask. Yuri

Re: lzo2 shows insane speed gap

2008-12-29 Thread Mel
On Monday 29 December 2008 12:25:00 Christian Weisgerber wrote: On the slow machines, the tests also consume a lot of system time. I've seen figures from 20 to 50%.  However, ktrace shows nothing out of the ordinary. If the program itself doesn't directly cause the system time, do interrupt

Re: lzo2 shows insane speed gap

2008-12-29 Thread Yuri
Christian Weisgerber wrote: My best guess at this time is that lzo2 somehow manages to induce crazy cache thrashing on some CPU models. Ideas and explanations welcome Try running single command that is different on different machines under valgrind (callgrind) on these machines and see that

Re: lzo2 shows insane speed gap

2008-12-29 Thread Artem Belevich
I see this performance difference on my boxes. First one has Core2Duo(E5-something), 4GB and runs RELENG_7/i386. lzotest is very fast. Second box is Core2Quad (Q9450), 8GB RAM and runs -current as of about a week ago. lzo2 binary built from ports is *slow*. However, 32-bit binary from the first

Re: lzo2 shows insane speed gap

2008-12-29 Thread Christian Weisgerber
Mel fbsd.hack...@rachie.is-a-geek.net wrote: If the program itself doesn't directly cause the system time, do interrupt rates give any hint as to what does? systat -vmstat shows a conspicuously large number of traps, I think. (I'm short on comparable FreeBSD machines.) And to rule out the

Re: lzo2 shows insane speed gap

2008-12-29 Thread Christian Weisgerber
Nate Eldredge: It might be good first to rule out compiler / library differences. Sure. Let's cut this short: Slow Athlon 64 X2 5200+ 2.6 GHz, FreeBSD 8.0-CURRENT amd64 ~60 min Phenom 9350e 2.0 GHz,OpenBSD 4.4-CURRENT amd64 ~80 min UltraSPARC-IIe 500 MHz (Blade

Re: lzo2 shows insane speed gap

2008-12-29 Thread Bruce Cran
On Tue, 30 Dec 2008 01:47:47 +0100 Christian Weisgerber na...@mips.inka.de wrote: Nate Eldredge: It might be good first to rule out compiler / library differences. Sure. Let's cut this short: Slow Athlon 64 X2 5200+ 2.6 GHz, FreeBSD 8.0-CURRENT amd64 ~60 min Phenom 9350e

Re: lzo2 shows insane speed gap

2008-12-29 Thread Kevin Day
Oh, and everybody is invited to run $ cd /usr/ports/archivers/lzo2 make and check for themselves. I've used lzo2 quite a bit in the past and never saw this, so I thought I'd try this on a few boxes we have... Output is from make fetch ; time make 8-core Opteron 2350 2.0ghz, 64GB

Re: lzo2 shows insane speed gap

2008-12-29 Thread Alex Kozlov
Christian Weisgerber na...@mips.inka.de wrote: Oh, and everybody is invited to run $ cd /usr/ports/archivers/lzo2 make $cd /usr/ports/archivers/lzo2 time sudo make [...] All tests passed. Now you are ready to install LZO. real