Re: manual page | zpool-features
On Sep 21, 2012, at 9:15 PM, Glen Barber g...@freebsd.org wrote: On Fri, Sep 21, 2012 at 11:14:36PM -0400, Darrel wrote: Welcome to the wonderful world that no one knows how UPDATING works anymore... -Garrett Thank you. Assholes [ pardon me] that they tend to be, with many exceptions- the steps would have been included in an OpenBSD update that required portions of the tree to be recompiled. You should always rebuild the tree in its entirety when upgrading. Plus, if you are running -CURRENT, you should expect some things to break on occasion. While those cases are not intentional, they do happen. Should is the operable word. If one does something that affects a non-niche group, it's a wise idea to use updating as it was supposed to be used. As it stands updating is neither used nor abused. Thanks, -Garrett___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: More kernel performance tests on FreeBSD 10.0-CURRENT
Am 09/21/12 23:39, schrieb Dimitry Andric: Hi all, As a followup to my previous post about the performance of FreeBSD 10.0 kernels compiled with different compilers (clang and gcc), I did another series of tests, now on a more modern machine (Core i5-based). I also tested the performance with different compiler optimization settings. The attached text file[1] contains more information about these tests, performance data, and my conclusions. Any errors and omissions are also my fault, so if you notice them, please let me know. The executive summary: GENERIC kernels compiled with clang 3.2 are again a little faster than those compiled with gcc 4.2.1. For gcc, compiling with -O2 also gives a slightly faster kernel than with -O1, but for clang there is no measurable difference between those flags. Again, many thanks to Gavin Atkinson for providing the required hardware. -Dimitry [1]: Also available at: http://www.andric.com/freebsd/perftest/perftest-kernel-2012-09-21a.txt At least one can say FreeBSD does not suffer from performance drain using the cutting edge clang 3.2 compared with a gcc 4.2.1 compiler, the echo from the past. Dimirty, are you planning also to benchmark clang 3.2 versus gcc 4.8.0? From the development point of view, such a benchmark would be more natural, but I do not know whether the kernel sources are gcc 4.8-friendly and would allow such a test. What is about optimization level -O3 and architectural recognition via -march=native? Neverthelesse, thanks. oh signature.asc Description: OpenPGP digital signature
Re: More kernel performance tests on FreeBSD 10.0-CURRENT
On Fri, Sep 21, 2012 at 11:39:40PM +0200, Dimitry Andric wrote: Hi all, As a followup to my previous post about the performance of FreeBSD 10.0 kernels compiled with different compilers (clang and gcc), I did another series of tests, now on a more modern machine (Core i5-based). I also tested the performance with different compiler optimization settings. The attached text file[1] contains more information about these tests, performance data, and my conclusions. Any errors and omissions are also my fault, so if you notice them, please let me know. The executive summary: GENERIC kernels compiled with clang 3.2 are again a little faster than those compiled with gcc 4.2.1. For gcc, compiling with -O2 also gives a slightly faster kernel than with -O1, but for clang there is no measurable difference between those flags. Again, many thanks to Gavin Atkinson for providing the required hardware. ... Conclusion: --- Kernels compiled with clang are a little faster in real time for building world, and in system time the difference is even larger, roughly 10%. For clang, the difference between -O1 and -O2 is not measurable, but for gcc, -O2 is slightly faster than -O1. Thank you very much for finishing the initial assessment. In my opinion, this positively closes the issue of the uncertainicity of the performance impact of the proposed clang use by default for the base system. Now, if only ports were handled. pgpQSrmz485wd.pgp Description: PGP signature
Re: More kernel performance tests on FreeBSD 10.0-CURRENT
On 2012-09-22 09:35, O. Hartmann wrote: Am 09/21/12 23:39, schrieb Dimitry Andric: ... At least one can say FreeBSD does not suffer from performance drain using the cutting edge clang 3.2 compared with a gcc 4.2.1 compiler, the echo from the past. Well, the main idea of these tests is to prove that we will have no regression, or even an improvement in performance, if we make clang 3.2 the default compiler for FreeBSD 10.0, instead of gcc 4.2.1. And that seems to be the case, at least for the kernel. That said, for one of the earlier tests, it seemed that for runtime performance, gcc 4.7.1-compiled programs (in this case clang 3.2 executables) were slightly faster than clang 3.2-compiled ones. In my opinion that result is not bad for such a relatively new compiler, against such a well-established one. :) Dimirty, are you planning also to benchmark clang 3.2 versus gcc 4.8.0? From the development point of view, such a benchmark would be more natural, but I do not know whether the kernel sources are gcc 4.8-friendly and would allow such a test. The kernel sources are currentely not very friendly to anything but our in-tree gcc and clang. We hacked our version of gcc to recognize several non-standard flags, such as -fformat-extensions, and a few others. We also implemented the -fformat-extensions flag for clang, since our custom printf format specifiers are used throughout the kernel. Ideally, we would remove all these non-standard flags and format specifiers, which would make it possible to compile the kernel with any version of gcc or clang, even external ones installed from ports, or by hand. This is not a trivial task... But maybe I'll take a shot, it would be nice to have at least one comparison against more modern gcc. I can't give any serious ETA, though. :) What is about optimization level -O3 and architectural recognition via -march=native? There are only so many things you can test, the possibilities are literally endless! I have already done a few preliminary tests for -march=native, but at least for clang, there seems to be no measureable difference in performance. The tests for gcc are still running. And indeed, -O3 is also a possibility, but again I think the difference will be very marginal, if measurable at all. -Dimitry ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: More kernel performance tests on FreeBSD 10.0-CURRENT
Hello Dimitry. Am 09/22/12 13:43, schrieb Dimitry Andric: On 2012-09-22 09:35, O. Hartmann wrote: Am 09/21/12 23:39, schrieb Dimitry Andric: ... At least one can say FreeBSD does not suffer from performance drain using the cutting edge clang 3.2 compared with a gcc 4.2.1 compiler, the echo from the past. Well, the main idea of these tests is to prove that we will have no regression, or even an improvement in performance, if we make clang 3.2 the default compiler for FreeBSD 10.0, instead of gcc 4.2.1. And that seems to be the case, at least for the kernel. That said, for one of the earlier tests, it seemed that for runtime performance, gcc 4.7.1-compiled programs (in this case clang 3.2 executables) were slightly faster than clang 3.2-compiled ones. In my opinion that result is not bad for such a relatively new compiler, against such a well-established one. :) Aggreed. You're completely right and said so, there is then, except serious bugs or miscompilations, no reason to stay with the dinosaur gcc 4.2.1 ;-) Dimirty, are you planning also to benchmark clang 3.2 versus gcc 4.8.0? From the development point of view, such a benchmark would be more natural, but I do not know whether the kernel sources are gcc 4.8-friendly and would allow such a test. The kernel sources are currentely not very friendly to anything but our in-tree gcc and clang. We hacked our version of gcc to recognize several non-standard flags, such as -fformat-extensions, and a few others. We also implemented the -fformat-extensions flag for clang, since our custom printf format specifiers are used throughout the kernel. Ideally, we would remove all these non-standard flags and format specifiers, which would make it possible to compile the kernel with any version of gcc or clang, even external ones installed from ports, or by hand. This is not a trivial task... But maybe I'll take a shot, it would be nice to have at least one comparison against more modern gcc. I can't give any serious ETA, though. :) When we used FreeBSD for scientific work, that was around 1998 - 2002, there were some attempts made to use Intel's icc compiler suite on FreeBSD in the 32Bit Linuxulator. That time I used that compiler only for compiling my modelling software, but there where reports of people made it possible to use the icc compiler also for compiling the FreeBSD system - with success as far as I know. What happened since then and more recent days that the sources got polluted by those hacks? No offense to you, but somehow this sounds that the efford has been placed in the wrong way since people revert with energy that what has been hacked with energy ;-) Neverthelesse, something is happeneing ... and this sounds good to me. What is about optimization level -O3 and architectural recognition via -march=native? There are only so many things you can test, the possibilities are literally endless! I have already done a few preliminary tests for -march=native, but at least for clang, there seems to be no measureable difference in performance. The tests for gcc are still running. I was wondering if the organisation and amount of cache present in a modern CPU is not taken into account when optimising code. Our Core2Duo CPUs still in use do have different architectural features than the more recent Core-i7 systems. Latter ones have level 3 caches. How does a compiler take advantage of those features by not given an explicit hint? And indeed, -O3 is also a possibility, but again I think the difference will be very marginal, if measurable at all. Well, speaking of marginality: some of our models rund for a week, some other long term models run for 4 weeks. Consider now a performce gain of 10% average. I'd like to have it ... it saves me time ;-) -Dimitry Oliver signature.asc Description: OpenPGP digital signature
Re: manual page | zpool-features
snip Actually, I am becoming suspicious that FreeBSD does not maintain a OpenBSD Packet Firewall that survives upgrades. Perhaps I should just take all of the Packet Firewall stuff out of my kernel and learn to use ipfw2. Darrel On the subject of OpenBSD Packet Firewall OpenBSD 4.5 version of PF firewall which is included with the base FreeBSD 8.x and 9.x releases is no longer supported by OpenBSD and very back level. The most current version of OpenBSD is 5.1. PF version 5.0 changed the syntax of the NAT statement making PF no longer backwards compatible which breaks some Freebsd standard, so updated versions of OpenBSD PF will no longer be mass ported to FreeBSD. Any bug fix code to OpenBSD PF will have to be incorporated by hand into FreeBSD's version of PF from this point on. The following will shine some more light on the subject. http://www.freebsd.org/cgi/query-pr.cgi?pr=167057 http://lists.freebsd.org/pipermail/freebsd-pf/2012-September/006740.html ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: More kernel performance tests on FreeBSD 10.0-CURRENT
On 2012-09-22 14:52, O. Hartmann wrote: ... When we used FreeBSD for scientific work, that was around 1998 - 2002, there were some attempts made to use Intel's icc compiler suite on FreeBSD in the 32Bit Linuxulator. That time I used that compiler only for compiling my modelling software, but there where reports of people made it possible to use the icc compiler also for compiling the FreeBSD system - with success as far as I know. What happened since then and more recent days that the sources got polluted by those hacks? The Intel compiler support has been largely removed, because it was not maintained. There are still remnants in cdefs.h though, and in theory it could be revived, if there was enough interest. However, Intel simply does not support anything else besides Windows and Linux for its compiler suite, and even on the Linux side you are best off if you use Red Hat or a Red Hat-based distribution such as CentOS or Scientific Linux. Some time ago I attempted to get a fairly recent Intel compiler version working on FreeBSD, but it was very tricky, and I remember I did not get everything working correctly. So unless either Intel starts supporting FreeBSD (or other BSDs), which is very unlikely, or somebody manages to get the Linux version working perfectly as a port, I don't see much sense in restoring the Intel compiler support. No offense to you, but somehow this sounds that the efford has been placed in the wrong way since people revert with energy that what has been hacked with energy ;-) I think you see this incorrectly; when I removed the Intel compiler support from the tree, it was unmaintained for several years already. Apparently there was very little interest for it. ... I have already done a few preliminary tests for -march=native, but at least for clang, there seems to be no measureable difference in performance. The tests for gcc are still running. I was wondering if the organisation and amount of cache present in a modern CPU is not taken into account when optimising code. Our Core2Duo CPUs still in use do have different architectural features than the more recent Core-i7 systems. Latter ones have level 3 caches. How does a compiler take advantage of those features by not given an explicit hint? I don't think the amount of CPU cache, or the number of levels, is taken into account, really. When you select a certain CPU type with -march, the compiler will just enable several features that are supported on that CPU, e.g. MMX, SSE, AVX and so on. It can also enable extra CPU registers, and/or switch to slightly different instruction scheduling. But since we are compiling the kernel with -mno-mmx, -mno-sse and even floating point disabled, apparently there is no real gain from specifying higher CPU types. -Dimitry ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: More kernel performance tests on FreeBSD 10.0-CURRENT
Am 09/22/12 15:52, schrieb Dimitry Andric: On 2012-09-22 14:52, O. Hartmann wrote: ... When we used FreeBSD for scientific work, that was around 1998 - 2002, there were some attempts made to use Intel's icc compiler suite on FreeBSD in the 32Bit Linuxulator. That time I used that compiler only for compiling my modelling software, but there where reports of people made it possible to use the icc compiler also for compiling the FreeBSD system - with success as far as I know. What happened since then and more recent days that the sources got polluted by those hacks? The Intel compiler support has been largely removed, because it was not maintained. There are still remnants in cdefs.h though, and in theory it could be revived, if there was enough interest. However, Intel simply does not support anything else besides Windows and Linux for its compiler suite, and even on the Linux side you are best off if you use Red Hat or a Red Hat-based distribution such as CentOS or Scientific Linux. Some time ago I attempted to get a fairly recent Intel compiler version working on FreeBSD, but it was very tricky, and I remember I did not get everything working correctly. So unless either Intel starts supporting FreeBSD (or other BSDs), which is very unlikely, or somebody manages to get the Linux version working perfectly as a port, I don't see much sense in restoring the Intel compiler support. True. It is use- and senseless, from my point of view, having ancient 32bit support only via the Linuxulator (which is 32bit only). The ICC was only useable on 32bit machines and FBSD 32bit (i386), which isn't any kind of an option nowadays. The same discussion has been triggered with CUDA and Linuxulator. No offense to you, but somehow this sounds that the efford has been placed in the wrong way since people revert with energy that what has been hacked with energy ;-) I think you see this incorrectly; when I removed the Intel compiler support from the tree, it was unmaintained for several years already. Apparently there was very little interest for it. To avoid further misunderstandings - I have no objections cleaning up the sources from unmaintained legacy. Since FreeBSD doesn't have 64Bit Linux support, the effort is wasted energy (my opinion, even if it is sometimes nice to see how it would perform ...). ... I have already done a few preliminary tests for -march=native, but at least for clang, there seems to be no measureable difference in performance. The tests for gcc are still running. I was wondering if the organisation and amount of cache present in a modern CPU is not taken into account when optimising code. Our Core2Duo CPUs still in use do have different architectural features than the more recent Core-i7 systems. Latter ones have level 3 caches. How does a compiler take advantage of those features by not given an explicit hint? I don't think the amount of CPU cache, or the number of levels, is taken into account, really. When you select a certain CPU type with -march, the compiler will just enable several features that are supported on that CPU, e.g. MMX, SSE, AVX and so on. It can also enable extra CPU registers, and/or switch to slightly different instruction scheduling. Well, I'm not that deep into compiler development. I thought that optimizations are also done on the level of caches a CPU has and the size of it. But since we are compiling the kernel with -mno-mmx, -mno-sse and even floating point disabled, apparently there is no real gain from specifying higher CPU types. I never came deeper into this logic - since I'm no operating system developer. But please correct me and, if possible, enlighten me, if there is something wrong in my understanding. Assumed, the option -march=native is switched on and the only optimisation is performed due to selection of code portions at compile time which are enclosed, say in #ifdef __AVX__ __some__nasty__vector_ops_256bitwide(); #endif which is triggered by the #define __AVX__ on Core-i7 CPUs with __AVX__ support, why is this explicitely disabled via -no-avx and friends? I would assume the developer has a reason not to use those speedy facilities, so I wouldn't expect any portion of #ifdef __AVX__ et cetera in the kernel code. The only explanation, from this naive point of view is, the compiler DOES DO some optimisations regarding the presence of such facilities and the -no-XXX options avoid those. Conclusively, I would expect a kind of performance gain when those features are made accessible. On the other hand, why are those features disabled? Intels silica is the reduced to something that gain speed from the clock cycle and the internal bandwidth due to cache sizes and clock speed and, naively spoken, all reduces to something compatible from ancients in the past. I can not fathom what the benefit of a Core i7 CPU then is compared to a Core2Duo when all the neat features are not used. A time
Re: More kernel performance tests on FreeBSD 10.0-CURRENT
On Sat, Sep 22, 2012 at 02:20:14PM +0300, Konstantin Belousov wrote: On Fri, Sep 21, 2012 at 11:39:40PM +0200, Dimitry Andric wrote: Hi all, As a followup to my previous post about the performance of FreeBSD 10.0 kernels compiled with different compilers (clang and gcc), I did another series of tests, now on a more modern machine (Core i5-based). I also tested the performance with different compiler optimization settings. The attached text file[1] contains more information about these tests, performance data, and my conclusions. Any errors and omissions are also my fault, so if you notice them, please let me know. The executive summary: GENERIC kernels compiled with clang 3.2 are again a little faster than those compiled with gcc 4.2.1. For gcc, compiling with -O2 also gives a slightly faster kernel than with -O1, but for clang there is no measurable difference between those flags. Again, many thanks to Gavin Atkinson for providing the required hardware. ... Conclusion: --- Kernels compiled with clang are a little faster in real time for building world, and in system time the difference is even larger, roughly 10%. For clang, the difference between -O1 and -O2 is not measurable, but for gcc, -O2 is slightly faster than -O1. Thank you very much for finishing the initial assessment. In my opinion, this positively closes the issue of the uncertainicity of the performance impact of the proposed clang use by default for the base system. It does not close everything. The kernel does not use floating point. I showed last week that clang may have problems with floating point. -- Steve ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
r240825 -current fault with backtrace
serial terminal not available, backtrace is a screen pic: https://picasaweb.google.com/lh/photo/b4TkAWaGPH7GkNH1VxuMJtv5Be3qwMGuK3qX4J5X-tk?feat=directlink -- thanks -kim ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: r240825 -current fault with backtrace
On Sat, Sep 22, 2012 at 2:27 PM, Gleb Smirnoff gleb...@freebsd.org wrote: On Sat, Sep 22, 2012 at 02:13:58PM -0400, Kim Culhan wrote: K serial terminal not available, backtrace is a screen pic: K K https://picasaweb.google.com/lh/photo/b4TkAWaGPH7GkNH1VxuMJtv5Be3qwMGuK3qX4J5X-tk?feat=directlink Why didn't you make dump? db call doadump Sorry, will get one. -kim ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: r240825 -current fault with backtrace
On Sat, Sep 22, 2012 at 02:13:58PM -0400, Kim Culhan wrote: K serial terminal not available, backtrace is a screen pic: K K https://picasaweb.google.com/lh/photo/b4TkAWaGPH7GkNH1VxuMJtv5Be3qwMGuK3qX4J5X-tk?feat=directlink Why didn't you make dump? db call doadump -- Totus tuus, Glebius. ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: manual page | zpool-features
snip Actually, I am becoming suspicious that FreeBSD does not maintain a OpenBSD Packet Firewall that survives upgrades. Perhaps I should just take all of the Packet Firewall stuff out of my kernel and learn to use ipfw2. Darrel On the subject of OpenBSD Packet Firewall OpenBSD 4.5 version of PF firewall which is included with the base FreeBSD 8.x and 9.x releases is no longer supported by OpenBSD and very back level. The most current version of OpenBSD is 5.1. PF version 5.0 changed the syntax of the NAT statement making PF no longer backwards compatible which breaks some Freebsd standard, so updated versions of OpenBSD PF will no longer be mass ported to FreeBSD. Any bug fix code to OpenBSD PF will have to be incorporated by hand into FreeBSD's version of PF from this point on. The following will shine some more light on the subject. http://www.freebsd.org/cgi/query-pr.cgi?pr=167057 http://lists.freebsd.org/pipermail/freebsd-pf/2012-September/006740.html Thank you. This information is good to know since I recompiled parts of Packet Firewall and then rebooted the machine with no working Packet Filter as a result. I have adjusted to the changes and am running OpenBSD 5.1 on my perimeter. Also, I am experimenting with NPF on NetBSD, which has a few bugs but generally works just fine tested with 'nmap' and the like. For FreeBSD, I will change to IPFW. It might be useful anyhow, since I have a Macintosh and will eventually probably get another. I would guess that the Macintosh firewall is still 'ipfw2', or something not too dissimilar. There is just no sense banging my head against a wall and repearting mistakes that actually do not belong to me by trying to run Packet Filter on FreeBSD. Darrel ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
[solved] manual page | zpool-features discontinue PF
snip Actually, I am becoming suspicious that FreeBSD does not maintain a OpenBSD Packet Firewall that survives upgrades. Perhaps I should just take all of the Packet Firewall stuff out of my kernel and learn to use ipfw2. Darrel On the subject of OpenBSD Packet Firewall OpenBSD 4.5 version of PF firewall which is included with the base FreeBSD 8.x and 9.x releases is no longer supported by OpenBSD and very back level. The most current version of OpenBSD is 5.1. PF version 5.0 changed the syntax of the NAT statement making PF no longer backwards compatible which breaks some Freebsd standard, so updated versions of OpenBSD PF will no longer be mass ported to FreeBSD. Any bug fix code to OpenBSD PF will have to be incorporated by hand into FreeBSD's version of PF from this point on. The following will shine some more light on the subject. http://www.freebsd.org/cgi/query-pr.cgi?pr=167057 http://lists.freebsd.org/pipermail/freebsd-pf/2012-September/006740.html Second reply. I intended to change the subject line to solved. Just for informational purposes, you might not want to do any firewall comparison on the OpenBSD misc list. A Packet Firewall developer responded to me personally, writing that the signal-to-noise ratio was too high and to refrain from posting to the list. So much for solving problems and sharing ideas. Darrel ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
