Re: user:sys time ratio
On Sun, 30 Nov 2003, Jeff Roberson wrote: > On Sun, 30 Nov 2003, Colin Percival wrote: > > >Robert Watson suggested that I compare performance from UP and SMP kernels: > > > > # /usr/bin/time -hl sh -c 'make -s buildworld 2>&1' > /dev/null > >Real UserSys > >UP kernel 38m33.29s 27m10.09s 10m59.15s > > (retest) 38m33.18s 27m04.40s 11m05.73s > >SMP w/o HTT 41m01.54s 27m10.27s 13m29.82s > > (retest) 39m47.50s 27m08.05s 12m12.20s > >SMP w/HTT 42m17.16s 28m12.82s 14m04.93s > > (retest) 44m09.61s 28m15.31s 15m44.86s > > > >That enabling HTT degrades performance is not surprising, since I'm not > > passing the -j option to make; but a 5% performance delta between UP and > > SMP kernels is rather surprising (to me, at least), and the fact that the > > system time varies so much on the SMP kernel also seems peculiar. > > So you have enabled SMP on a system with one physical core and two > logical cores? Looks like almost a 20% slowdown in system time with the > SMP kernel. It's too bad it's enabled by default now. I suspect that > some of this is due to using the lock prefix on P4 cores. It makes the > cost of a mutex over 300 cycles vs 50. It might be interesting to do an > experiment without HTT, but with SMP enabled and the lock prefix > commented out. It would be interesting to compare the relative costs of: (a) De-inlining of mutex calls so that you can "plug" mutex implementations at link-time, with SMP and non-SMP versions of mutex operations, perhaps indicating with a loader which approach to take. (b) Using inlined SMP mutexes for all kernels. Would also be very interesting to have mutex contention information, and in particular, to know how much time was spent contending on Giant during the build... Robert N M Watson FreeBSD Core Team, TrustedBSD Projects [EMAIL PROTECTED] Senior Research Scientist, McAfee Research ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: user:sys time ratio
On Sun, 30 Nov 2003, Colin Percival wrote: >Robert Watson suggested that I compare performance from UP and SMP kernels: > > # /usr/bin/time -hl sh -c 'make -s buildworld 2>&1' > /dev/null >Real UserSys >UP kernel 38m33.29s 27m10.09s 10m59.15s > (retest) 38m33.18s 27m04.40s 11m05.73s >SMP w/o HTT 41m01.54s 27m10.27s 13m29.82s > (retest) 39m47.50s 27m08.05s 12m12.20s >SMP w/HTT 42m17.16s 28m12.82s 14m04.93s > (retest) 44m09.61s 28m15.31s 15m44.86s > >That enabling HTT degrades performance is not surprising, since I'm not > passing the -j option to make; but a 5% performance delta between UP and > SMP kernels is rather surprising (to me, at least), and the fact that the > system time varies so much on the SMP kernel also seems peculiar. So you have enabled SMP on a system with one physical core and two logical cores? Looks like almost a 20% slowdown in system time with the SMP kernel. It's too bad it's enabled by default now. I suspect that some of this is due to using the lock prefix on P4 cores. It makes the cost of a mutex over 300 cycles vs 50. It might be interesting to do an experiment without HTT, but with SMP enabled and the lock prefix commented out. I have a set of changes for ULE that should fix some of the HTT slowdown, although it is inevitable that there will always be some. If you would like to try the patch, it's available at: http://www.chesapeake.net/~jroberson/ulehtt.diff Cheers, Jeff >Is this normal? > > Colin Percival > > ___ > [EMAIL PROTECTED] mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to "[EMAIL PROTECTED]" > ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: user:sys time ratio
Robert Watson suggested that I compare performance from UP and SMP kernels: # /usr/bin/time -hl sh -c 'make -s buildworld 2>&1' > /dev/null Real UserSys UP kernel 38m33.29s 27m10.09s 10m59.15s (retest) 38m33.18s 27m04.40s 11m05.73s SMP w/o HTT 41m01.54s 27m10.27s 13m29.82s (retest) 39m47.50s 27m08.05s 12m12.20s SMP w/HTT 42m17.16s 28m12.82s 14m04.93s (retest) 44m09.61s 28m15.31s 15m44.86s That enabling HTT degrades performance is not surprising, since I'm not passing the -j option to make; but a 5% performance delta between UP and SMP kernels is rather surprising (to me, at least), and the fact that the system time varies so much on the SMP kernel also seems peculiar. Is this normal? Colin Percival ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: user:sys time ratio
On Sun, 30 Nov 2003, Colin Percival wrote: >I've got a system running 5.2-BETA from 27/11/03, with the > malloc_abort, malloc_junk, DEBUG=-g, DDB, INVARIANT*, and WITNESS* > debugging options changed (as was done in 5.1-RELEASE). >When running `make buildworld`, I see large amounts of sys time; eg, > 27 minutes user & 14 minutes sys for building 5.2, or 14 minutes user & > 10 minutes sys for building 4.9. I expected the ratio of user:sys to be > much larger than this, and mailing list traffic indicates that a 4:1 > ratio is typical. (FWIW, prior to changing the debugging options, the > user:sys time ratio was around 1:1.) >Can anyone suggest why the kernel seems to be behaving so sluggishly? > >The system hardware is P4 2.8Ghz, 865G, 2GB DDR, IDE drives; there is > very little disk activity, so I'm sure that isn't the issue; and > disabling HTT results in about a 2% improvement in both user and sys > times. It sounds like you have multiple logical cores -- have you tried building a kernel without SMP support to see what happens? Robert N M Watson FreeBSD Core Team, TrustedBSD Projects [EMAIL PROTECTED] Senior Research Scientist, McAfee Research ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: user:sys time ratio
In message <[EMAIL PROTECTED]>, Colin Percival writes: >At 15:30 30/11/2003 +0100, Poul-Henning Kamp wrote: >>In message <[EMAIL PROTECTED]>, Colin >>Percival >> writes: >> > When running `make buildworld`, I see large amounts of sys time; eg, 27 >> >minutes user & 14 minutes sys for building 5.2, or 14 minutes user & 10 >> >minutes sys for building 4.9. I expected the ratio of user:sys to be much >> >larger than this, and mailing list traffic indicates that a 4:1 ratio is >> >typical. > >I've seen UNIX systems have "typical" system/user splits from 1/9 to 9/1 >>it all depends on what you're doing. > > Sure, but buildworld is a fairly well-defined benchmark; I wouldn't >expect to see such a large difference when running exactly the same code on >different systems. The amount of system time depends on a lot of environmental factors. For instance on my amd64/SMP, the vnode pool is mis-sized, so the the namecache does not reflect what is actually already in RAM. The result is a fair bit of system time wasted instantiating vnodes from cached inode diskblocks. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 [EMAIL PROTECTED] | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: user:sys time ratio
At 15:30 30/11/2003 +0100, Poul-Henning Kamp wrote: In message <[EMAIL PROTECTED]>, Colin Percival writes: > When running `make buildworld`, I see large amounts of sys time; eg, 27 >minutes user & 14 minutes sys for building 5.2, or 14 minutes user & 10 >minutes sys for building 4.9. I expected the ratio of user:sys to be much >larger than this, and mailing list traffic indicates that a 4:1 ratio is >typical. >I've seen UNIX systems have "typical" system/user splits from 1/9 to 9/1 it all depends on what you're doing. Sure, but buildworld is a fairly well-defined benchmark; I wouldn't expect to see such a large difference when running exactly the same code on different systems. Colin Percival ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: user:sys time ratio
In message <[EMAIL PROTECTED]>, Colin Percival writes: > I've got a system running 5.2-BETA from 27/11/03, with the malloc_abort, >malloc_junk, DEBUG=-g, DDB, INVARIANT*, and WITNESS* debugging options >changed (as was done in 5.1-RELEASE). > When running `make buildworld`, I see large amounts of sys time; eg, 27 >minutes user & 14 minutes sys for building 5.2, or 14 minutes user & 10 >minutes sys for building 4.9. I expected the ratio of user:sys to be much >larger than this, and mailing list traffic indicates that a 4:1 ratio is >typical. (FWIW, prior to changing the debugging options, the user:sys time >ratio was around 1:1.) I've seen UNIX systems have "typical" system/user splits from 1/9 to 9/1 it all depends on what you're doing. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 [EMAIL PROTECTED] | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "[EMAIL PROTECTED]"
user:sys time ratio
I've got a system running 5.2-BETA from 27/11/03, with the malloc_abort, malloc_junk, DEBUG=-g, DDB, INVARIANT*, and WITNESS* debugging options changed (as was done in 5.1-RELEASE). When running `make buildworld`, I see large amounts of sys time; eg, 27 minutes user & 14 minutes sys for building 5.2, or 14 minutes user & 10 minutes sys for building 4.9. I expected the ratio of user:sys to be much larger than this, and mailing list traffic indicates that a 4:1 ratio is typical. (FWIW, prior to changing the debugging options, the user:sys time ratio was around 1:1.) Can anyone suggest why the kernel seems to be behaving so sluggishly? The system hardware is P4 2.8Ghz, 865G, 2GB DDR, IDE drives; there is very little disk activity, so I'm sure that isn't the issue; and disabling HTT results in about a 2% improvement in both user and sys times. Colin Percival ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "[EMAIL PROTECTED]"
