Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server
On Wed, Jan 04, 2012 at 03:52:06PM -0800, Matthew Tippett wrote: Hmm... No sure what happened there again. What I sent (pulled from my Sent folder... === Thanks for the comment Arnaud. For comparative benchmarking on Phoronix.com http://Phoronix.com, Michael invariable leaves it in the default configuration 'in the way the developers or vendor wanted it for production'. This is by rule. A quick question: why is ZFS used in the benchmark? Both operating systems were in their stock configuration aside from FreeBSD 9.0 using ZFS. UFS is the default on FreeBSD, not ZFS. FreeBSD was not left in the default configuration. -- Denny Lin ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server
On 01/04/2012 20:52, Matthew Tippett wrote: As a service to the community or vendor that publishes the tuning guide, Michael is more than willing to redo a tuned vs untuned comparison. To date, the communities have never taken us up on that offer. In part, this affects Phoronix.com http://Phoronix.com's perception in the public, but that is more of a result of a one sided discussion by a party external to a particular community (with a healthy touch of journalisticly pumped compare contrast). For the FreeBSD community, who else outside of the FreeBSD community actually runs public comparisons of FreeBSD against anything? If you just want to benchmark defaults, please, use proper defaults and don't do a *custom* ZFS setup (which was IMHO a pretty big and gross mistake). I would be interested in a benchmark that does significant tests with same hardware and REAL default setup. It would be awesome if those benchmarks were re-done that way, so we can compare an out-of-the-box experience even if that's not the last word on how a system will perform (as others said, no one uses a default setup for their servers) And, if anyone else suggests a tuned-system benchmark ... I'm ok with that too as long it's done properly and with guidance of a person of the community that can give proper advice. I'm a regular reader of Phoronix because it's basically easier than being subscribed to a couple of mailing lists, blogs and such just to get latest information on developments. And the site is fairly popular, so you hold a pretty big responsibility in there. I got really disappointed on how a bad benchmark could impact on the reputation of FreeBSD. Things that are not true that people will be repeating (not-so-long-ago I was a moderator on a Linux forum and I saw that misinformation in action, it's terrible !) Sorry if the words sound strong ... but I'm glad that these benchmarks can be re-done. Thanks for reading ! signature.asc Description: OpenPGP digital signature
Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server
Hi, On Fri, Dec 16, 2011 at 12:16 PM, matt...@phoronix.com wrote: Thanks. My request for the person documenting the tunings also runs the benchmark to ensure expected behaviour. Why should you have to tune anything ? Did you tune the Oracle Server install ? If not, you should not have to tune the FreeBSD install, that wouldn't be fair. If you tune FreeBSD, you should tune the Oracle Server install too. It is pretty easy to win at least 30% in performance for certain workload by choosing the right kernel configuration. - Arnaud The installation, execution and comparison against the benchmarks in the article is fairly simple. Note that some tuning may not be relevant or recommended (ie: some of the fs benchmarks are sensitive to barriers and other synchronous operations). I'd recommend bowing out of a benchmark with a 'we're going to be slower since the default configuration is this way for the following reason' if this is the case. Thanks 'someone'. Matthew Dec 16, 2011 8:46 AM, Adrian Chadd adr...@freebsd.org wrote: Can someone please write up a nice, concise blog post somewhere outlining all of this? Extra bonus points if it's a blog that is picked up by blogs.freebsdish.org and/or some of the other BSD sites. Guys/girls/fuzzy things - this is 2011; people look at shiny blog sites with graphs rather than mailing lists. Sorry, we lost that battle. :) Adrian ___ freebsd-performa...@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-performance To unsubscribe, send any mail to freebsd-performance-unsubscr...@freebsd.org ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server
Thanks for the comment Arnaud. For comparative benchmarking on [1]Phoronix.com, Michael inva= riable leaves it in the default configuration 'in the way the developers or= vendor wanted it for production'. This is by rule. However, i= nvariable the community or vendor for platforms that post poor scores on be= nchmark cry foul about using the default config. 'it should be tuned,= no-one deploys an untuned system' or 'the system is configured for a diffe= rent workload'. The response from us to this comes in two forms. nb= sp; 1) If it is the wrong workload for the platform, do a public pos= t explaining and analysing the results. Highlighting the rationale fo r the concious reduction in performance (ie: journaling filesystems with ba= rriers suffer in some write benchmarks for the sake of filesystem integrity= . 2) If tuning can have a material impact on the results, post a t uning guide with step by step and rationale. Ie: educate the communit= y and users. Michael and I have had many discussions with vendors an= d communities on this. In almost all cases, the vendor has either cha= nged the default configuration or accepted the results as valid. As = a service to the community or vendor that publishes the tuning guide, Micha= el is more than willing to redo a tuned vs untuned comparison. To dat= e, the communities have never taken us up on that offer. In part, thi= s affects [2]Phoronix.com's perception in the public, but that is more of a result of a one sided d= iscussion by a party external to a particular community (with a healthy tou= ch of journalisticly pumped compare contrast). For the FreeBSDcommunity, who else outside of the FreeBSD community actually runs public c= omparisons of FreeBSD against anything? Matthew -- Sent from my HP Pre3 _ On Jan 4, 2012 1:58 PM, Arnaud Lacombe lacom...@gmail.comg= t; wrote: Hi, On Fri, Dec 16, 2011 at 12:16 PM,= matt...@phoronix.com wrote: Thanks. = gt; My request for the person documenting the tunings also runs the benchma= rk to ensure expected behaviour. Why should you= have to tune anything ? Did you tune the Oracle Server install ? If = not, you should not have to tune the FreeBSD install, that wouldn't b= e fair. If you tune FreeBSD, you should tune the Oracle Server instal= l too. It is pretty easy to win at least 30% in performance for certa= in workload by choosing the right kernel configuration. = - Arnaud The installation, execution and comparison agai= nst the benchmarks in the article is fairly simple. = Note that some tuning may not be relevant or recommended (ie: s= ome of the fs benchmarks are sensitive to barriers and other syn= chronous operations). I'd recommend bowing out of a benchm= ark with a 'we're going to be slower since the default configura= tion is this way for the following reason' if this is the case.= Thanks 'someone'. Matthew Dec 16, 2011 8:46 AM, Adrian Chadd a= dr...@freebsd.org wrote: Can someone please write= up a nice, concise blog post somewhere outlining all of this?= Extra bonus points if it's a blog that is picked up = by blogs.freebsdish.org and/or some of the other BSD sites. Guys/girls/fuzzy things - this is 2011; people look at sh= iny blog sites with graphs rather than mailing lists. Sorry, we = lost that battle. :) = Adrian ___ g= t; freebsd-performa...@freebsd.org mailing list http://lists.fre= ebsd.org/mailman/listinfo/freebsd-performance To unsubscribe, se= nd any mail to freebsd-performance-unsubscr...@freebsd.org = br References 1. 3Dhttp://Phoronix.com/ 2. 3Dhttp://Phoronix.com/ ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server
On Wed, 04 Jan 2012 14:31:55 -0800 matt...@phoronix.com wrote: Thanks for the comment Arnaud. For comparative benchmarking on[1]Phoronix.com, Michael inva configuration 'in the way the developers or production'. This is by rule. However, i poor scores on be 'it should be tuned, is configured for a diffe The response from us to this comes in two forms. nb 1) If it is the wrong workload for the platform, do a public pos explaining and analysing the results. Highlighting the rationale fo r the concious reduction in performance (ie: journaling filesystems with ba filesystem integrity 2) If tuning can have a material impact on the results, post a t uning guide with step by step and rationale. Ie: educate the communit Michael and I have had many discussions with vendors an on this. In almost all cases, the vendor has either cha default configuration or accepted the results as valid. Asguide, Micha comparison. To dat offer. In part, thi public, but that is more of a result of a one sided d party external to a particular community (with a healthy tou journalisticly pumped compare contrast). For the FreeBSD community, who else outside of the FreeBSD community actually runs public c Matthew Not really related to the discussion on hand, but the above about the most unreadable email I am yet to read on the public mailing list. -- Alexander Kabaev signature.asc Description: PGP signature
Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server
Hmm... No sure what happened there again. What I sent (pulled from my Sent folder... === Thanks for the comment Arnaud. For comparative benchmarking on Phoronix.com http://Phoronix.com, Michael invariable leaves it in the default configuration 'in the way the developers or vendor wanted it for production'. This is by rule. However, invariable the community or vendor for platforms that post poor scores on benchmark cry foul about using the default config. 'it should be tuned, no-one deploys an untuned system' or 'the system is configured for a different workload'. The response from us to this comes in two forms. 1) If it is the wrong workload for the platform, do a public post explaining and analysing the results. Highlighting the rationale for the concious reduction in performance (ie: journaling filesystems with barriers suffer in some write benchmarks for the sake of filesystem integrity. 2) If tuning can have a material impact on the results, post a tuning guide with step by step and rationale. Ie: educate the community and users. Michael and I have had many discussions with vendors and communities on this. In almost all cases, the vendor has either changed the default configuration or accepted the results as valid. As a service to the community or vendor that publishes the tuning guide, Michael is more than willing to redo a tuned vs untuned comparison. To date, the communities have never taken us up on that offer. In part, this affects Phoronix.com http://Phoronix.com's perception in the public, but that is more of a result of a one sided discussion by a party external to a particular community (with a healthy touch of journalisticly pumped compare contrast). For the FreeBSD community, who else outside of the FreeBSD community actually runs public comparisons of FreeBSD against anything? Matthew === On 01/04/2012 03:49 PM, Alexander Kabaev wrote: On Wed, 04 Jan 2012 14:31:55 -0800 matt...@phoronix.com wrote: Thanks for the comment Arnaud. For comparative benchmarking on[1]Phoronix.com, Michael inva configuration 'in the way the developers or production'. This is by rule. However, i poor scores on be 'it should be tuned, is configured for a diffe The response from us to this comes in two forms.nb 1) If it is the wrong workload for the platform, do a public pos explaining and analysing the results. Highlighting the rationale fo r the concious reduction in performance (ie: journaling filesystems with ba filesystem integrity 2) If tuning can have a material impact on the results, post a t uning guide with step by step and rationale. Ie: educate the communit Michael and I have had many discussions with vendors an on this. In almost all cases, the vendor has either cha default configuration or accepted the results as valid. Asguide, Micha comparison. To dat offer. In part, thi public, but that is more of a result of a one sided d party external to a particular community (with a healthy tou journalisticly pumped compare contrast). For the FreeBSD community, who else outside of the FreeBSD community actually runs public c Matthew Not really related to the discussion on hand, but the above about the most unreadable email I am yet to read on the public mailing list. ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server
On Wed, Jan 4, 2012 at 1:58 PM, Arnaud Lacombe lacom...@gmail.com wrote: Hi, On Fri, Dec 16, 2011 at 12:16 PM, matt...@phoronix.com wrote: Thanks. My request for the person documenting the tunings also runs the benchmark to ensure expected behaviour. Why should you have to tune anything ? Did you tune the Oracle Server install ? If not, you should not have to tune the FreeBSD install, that wouldn't be fair. If you tune FreeBSD, you should tune the Oracle Server install too. It is pretty easy to win at least 30% in performance for certain workload by choosing the right kernel configuration. This assumes that Oracle doesn't do secret sauce tuning... the Vanilla CentOS/RHEL base is probably a better comparison than the Oracle custom distro. Thanks! -Garrett ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server
On Sat, Dec 24, 2011 at 04:04:31PM +0100, O. Hartmann wrote: There maybe serious reasons having the Linuxulator, i do not know. But if not, why spending rare developer resources on that? This is a classical misunderstanding of the FreeBSD development model. There is no staff standing around waiting for assignments, as with a commercial company. When committers join the project, they usually (almost always) already have a long list of things that they want to work on. And then they go work on them. Neither the core team, nor the FreeBSD Foundation, direct the project and its course of development. Some of the members of each do post emails, or stand up in front of conferences, and say you know, I think it would be really neat if someone did xyz. Sometimes this leads to results, sometimes not. As for the companies that have their own FreeBSD-derived products, often their goals are tightly focused, e.g. improve the number of packets we can pass or support our specialized hardware. Some, but not all, of the resultant work makes it back into FreeBSD. We get to say it would be really neat if ...; and, in addition, point to possible future minimization of merging and duplication of effort as a way to save costs long-term. But with these exceptions, development is primarily driven from the bottom-up (individual committers find something they are interested in working on, and then go work on it), and not the top-down as in real companies. This is the way the overwhelming majority (90+%?) of the work on FreeBSD gets done. So, there's no one assigned the tasks of closing PRs, nor working on coordinating code with the other BSDs, nor working on the Linuxolator, nor even supporting high-performance computing. It's a cooperative anarchy, not a hierarchy. mcl ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server
On 2011-Dec-24 15:49:00 +0100, O. Hartmann ohart...@mail.zedat.fu-berlin.de wrote: On 12/23/11 12:38, Daniel Kalchev wrote: Here is now it works: If you see an problem and have a solution: go fix it. Many will be grateful. If you can't fix it, but have an idea how to fix it, share it. May will be grateful. If you can't fix it and don't have any idea, just say there is a problem and stop there. There are many, many, many like you who just hold their breath. We all learn, every day. Daniel Sorry, but your crap is simply breathtaking. That was completely uncalled for. You have spent the last month or so whinging about FreeBSD but I have yet to see you provide any constructive input. Instead of whinging about ULE not doing what you want, how about you either fix it yourself or offer to fund someone to fix it for you. -- Peter Jeremy pgpljXOfVhxLu.pgp Description: PGP signature
Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server
Am 12/30/11 10:07, schrieb Mark Linimon: On Sat, Dec 24, 2011 at 04:04:31PM +0100, O. Hartmann wrote: There maybe serious reasons having the Linuxulator, i do not know. But if not, why spending rare developer resources on that? This is a classical misunderstanding of the FreeBSD development model. There is no staff standing around waiting for assignments, as with a commercial company. When committers join the project, they usually (almost always) already have a long list of things that they want to work on. And then they go work on them. Neither the core team, nor the FreeBSD Foundation, direct the project and its course of development. Some of the members of each do post emails, or stand up in front of conferences, and say you know, I think it would be really neat if someone did xyz. Sometimes this leads to results, sometimes not. As for the companies that have their own FreeBSD-derived products, often their goals are tightly focused, e.g. improve the number of packets we can pass or support our specialized hardware. Some, but not all, of the resultant work makes it back into FreeBSD. We get to say it would be really neat if ...; and, in addition, point to possible future minimization of merging and duplication of effort as a way to save costs long-term. But with these exceptions, development is primarily driven from the bottom-up (individual committers find something they are interested in working on, and then go work on it), and not the top-down as in real companies. This is the way the overwhelming majority (90+%?) of the work on FreeBSD gets done. So, there's no one assigned the tasks of closing PRs, nor working on coordinating code with the other BSDs, nor working on the Linuxolator, nor even supporting high-performance computing. It's a cooperative anarchy, not a hierarchy. mcl By no maen is this what I said or intended to say. oliver signature.asc Description: OpenPGP digital signature
Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server
On 23 Dec 2011 12:25, O. Hartmann ohart...@mail.zedat.fu-berlin.de wrote: Look at Steve Kargls problem. He investigated a SCHED_ULE problem in a way that is far beyond enough! He gave tests, insights of his setup, bad performance compared to SCHED_4BSD and what happend? We are still stuck with this problem and more and more people realise, that FreeBSD does have somewhere a problem and this seems to be a nasty problem not easy to find or investigate. But look at how Steve has been silenced in the past ... Benchmarks, especially published ones, reveal those pits and soemone could look into it. Another problem is this very elite-feeling closed club. Once you managed it getting into the club of committers or core team members, you'll probably fight for your seat ... You are aware that Steve is part of the 'elite club', right? Many of us rarely use our @FreeBSD.org addresses; you'd probably be surprised at the names in the Developers list. Just being a committer gives your opinions very little weight; everyone has to make their case in the same way. There's really, really no eliteism here! Chris ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Linuxulator (was: Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server)
Hi, you assume in your comment that development time wasted in the linuxulator is time lost for other development. This assumption could be valid for a commercially developed OS, but is wrong for FreeBSD. I tell this as a person who spend a lot of time with the linux ports, mentored a GSoC student who worked on the linuxulator and also put some time into the kernel parts. The use case for it is: run linux programs which are not available with source or where we are not able to get it compiled on FreeBSD with a reasonable effort. As a data point, we managed in the past to take the closed source linux version of the Intel C/C++ compiler and manipulate it in a way to run in the linuxulator but produce FreeBSD binaries. I got reports that it was used in some HPC scenarios. Wasn't it you who asked if there's a way to run CUDA on FreeBSD? Pessimistic but interested souls would not wait until there is maybe some result from open sourcing the nvidia compiler and instead either try to get something similar up and running, or to get a 64 bit version of the linuxulator. The later one may be more beneficial for more people, and may even more easy as the parts are open source and there's even some code somewhere in a VCS (maybe in perforce). Bye, Alexander. -- Send via an Android device, please forgive brevity and typographic and spelling errors. O. Hartmann ohart...@mail.zedat.fu-berlin.de hat geschrieben:On 12/23/11 12:44, Alexander Best wrote: [...] Many suggested that the Linux binaries be run via the FreeBSD Linux emulation. Unchanged. There is one problem here though, the emulation is still 32 bit. plus the current emulation layer is far from complete. a lot of stuff hasn't been implemented yet (meaning it's missing or implemented as dummy code). try running recent firefox linux binaries on freebsd. they will all crash almost instantly. cheers. alex [...] Sometimes I'm glad to have the Linuxulator, for instance using Mathematica or an older 32bit IDL or even MATLAB. But lately, I run into problems on more recent platforms like FreeBSD 9 and 10. There maybe serious reasons having the Linuxulator, i do not know. But if not, why spending rare developer resources on that? As far as I'm concerned, the only real reason having the Linuxulator is some stuff from Adobe for desktop systems, Flash. That's it. For the scientific stuff, I try to move my people towards OpenSource, since we do standard stuff and I expect students and scientists solving problems without fancy coloured clicky funny things. In production, this might be another point of view. SciLab from INRIA is great, MuPAD, MAXIMA also. But is there a real need running the Linux binary of Forefox on FreeBSD? Regards, Oliver ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Linuxulator (was: Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server)
Am 12/28/11 15:24, schrieb Alexander Leidinger: Hi, you assume in your comment that development time wasted in the linuxulator is time lost for other development. This assumption could be valid for a commercially developed OS, but is wrong for FreeBSD. I tell this as a person who spend a lot of time with the linux ports, mentored a GSoC student who worked on the linuxulator and also put some time into the kernel parts. That wasn't ment to be any offense! And shortly after I wrote this, i remembered myself that many stuff from JAVA is still completely not open due to Linux-shielded/protected stuff. Even commercial companies like DeLL seem to be incabale of offering JAVA apllets which are compatible to all platforms. I tried hard on geting their iDRAC6 stuff running on FreeBSD and recent firefox or even with diablo-jdk as a standalone JAVA application, but it lacks obviously some functionality only available in Linux firefox and/or JAVA. The use case for it is: run linux programs which are not available with source or where we are not able to get it compiled on FreeBSD with a reasonable effort. As a data point, we managed in the past to take the closed source linux version of the Intel C/C++ compiler and manipulate it in a way to run in the linuxulator but produce FreeBSD binaries. I got reports that it was used in some HPC scenarios. Yes, you're right. We also used the Intel C and Fortran compiler to run a lot of scientifc programs on some boxes, but noadays, most software, especially scientific one, is 64bit since we deal with huge datasets which need a lot of memory and/or are happy having no memory limitations doing n-body simulations. But this is times ago. I'm sure there are still applications running 32bit, but the benefit of having 64bit and so a native 64bit compiler is quite huge. Wasn't it you who asked if there's a way to run CUDA on FreeBSD? Pessimistic but interested souls would not wait until there is maybe some result from open sourcing the nvidia compiler and instead either try to get something similar up and running, or to get a 64 bit version of the linuxulator. The later one may be more beneficial for more people, and may even more easy as the parts are open source and there's even some code somewhere in a VCS (maybe in perforce). Yes, it was me who asked for a 32Bit CUDA solution, because I desperately needed OpenCL/CUDA. And I asked because I wouldn't like to leave my FreeBSD platform for achiving this, but I do not have any chance. I tried to get the CUDA stuff working on FreeBSD, but it would take me ages to fullfill and at the end I need a development environment. The BLOG mentioned and referred to achive this is quite old and outdated and also stated that one need either a full Linux installation (Gentoo) or an development box. Well, having a development box menas also having a full working Linux that could be 64bit and running my applications. It is now that way. We use Suse 11 and Ubuntu 10 boxes with TESLA boards from nVidia and I have to compile my stuff and run it on those boxes. I also administer one of such boxes and I must confress, that I'm not happy with that. For once, it might be a personal thing, on the other hand I feel lost on such cryptographic and shell-polluted administrative environments, which could only be administered by special scripted tools - and each Linux distribution seems to have its own, holy and mystical way to encrypt former clean administrative ways to do. Days ago nVidia and AMD claimed to have opened their OpenCL intermediate language/representation and and nVidia claims to have opensourced their compiler. But althought requested being member of the elite group of people having access to that piece of software, I did not get access to it. So, it seems not to be real opensourced. But I think this might be a topic of another thread, which would be very interesting to me to discuss, since I'm not that familiar with what is possible in FreeBSD and what not. I see that, from the theoretical perspective of how LLVM works, their could be a chance to get FreeBSD on par with Linux in GPGPU concerned applications, which becomes very, very important now. Bye, Alexander. Regards, Oliver -- Send via an Android device, please forgive brevity and typographic and spelling errors. O. Hartmann ohart...@mail.zedat.fu-berlin.de hat geschrieben: On 12/23/11 12:44, Alexander Best wrote: [...] Many suggested that the Linux binaries be run via the FreeBSD Linux emulation. Unchanged. There is one problem here though, the emulation is still 32 bit. plus the current emulation layer is far from complete. a lot of stuff hasn't been implemented yet (meaning it's missing or implemented as dummy code). try running recent firefox linux binaries on freebsd. they will all crash almost instantly. cheers. alex [...] Sometimes I'm glad to have the Linuxulator, for instance using Mathematica or an older 32bit
Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux, 6.1 Server
Well, the post is OT, but I need some vent. On 2011-12-19 18:34, dan...@digsys.bg wrote: For example, few checkboxes with common sysctl tuning would be perfect, even if they would be marked as Experimental, or not recommended. By following this, we push FreeBSD into the Linux style of doing things: someone else decides what is good for you, without having a clue of your circumstances. It's nice to see sb. with similar thoughts. I too find the freedom to administer your system the way you see fit to be very important. I was very saddened when I discovered that in some ways FreeBSD also forces specific behaviour and in some others builds barriers to prevent people from doing things the authors considered stupid. I don't view it as Linux way vs. FreeBSD way ( though it may be because I don't know either too well ). Rather, I see it as the MacOS way. Education is much better than building barriers and it's never true that a developer can predict all the uses of their code. And different uses call for different configurations, artificially limiting it is a time invested to reduce code's value. -- Twoje radio ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server
On 12/23/11 12:38, Daniel Kalchev wrote: On 23.12.11 12:48, O. Hartmann wrote: Look at Steve Kargls problem. He investigated a SCHED_ULE problem in a way that is far beyond enough! He gave tests, insights of his setup, bad performance compared to SCHED_4BSD and what happend? We are still stuck with this problem and more and more people realise, that FreeBSD does have somewhere a problem and this seems to be a nasty problem not easy to find or investigate. This has made me to realize, that I was having a problem with SCHED_ULE that I was not aware of until now. WOW! :) Every scheduler has some problem, some fail here some fail there. I am confident, that the case that Steve Kargls has reported will be resolved. Another problem is this very elite-feeling closed club. Once you managed it getting into the club of committers or core team members, you'll probably fight for your seat ... I dont propose for that socialists crap Linux people tend to be like, [..] You never heard of the People's Republic of Berkeley? :) As for commiter access, this sort of comments trigger the system administrator in me. I have seen enough people, who for the lack of other excuses always use but I don't have enough RIGHTS!. I am evil, I know But I follow the illusion that if people can see what benchmarks reveal, they start thinking and if the facts are starting to give a heavy load load on those rejecting the facts, they migght change their opinion or get hopefully replaced by more openminded people. Here is now it works: If you see an problem and have a solution: go fix it. Many will be grateful. If you can't fix it, but have an idea how to fix it, share it. May will be grateful. If you can't fix it and don't have any idea, just say there is a problem and stop there. There are many, many, many like you who just hold their breath. We all learn, every day. Daniel Sorry, but your crap is simply breathtaking. oh signature.asc Description: OpenPGP digital signature
Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server
On 12/23/11 12:44, Alexander Best wrote: [...] Many suggested that the Linux binaries be run via the FreeBSD Linux emulation. Unchanged. There is one problem here though, the emulation is still 32 bit. plus the current emulation layer is far from complete. a lot of stuff hasn't been implemented yet (meaning it's missing or implemented as dummy code). try running recent firefox linux binaries on freebsd. they will all crash almost instantly. cheers. alex [...] Sometimes I'm glad to have the Linuxulator, for instance using Mathematica or an older 32bit IDL or even MATLAB. But lately, I run into problems on more recent platforms like FreeBSD 9 and 10. There maybe serious reasons having the Linuxulator, i do not know. But if not, why spending rare developer resources on that? As far as I'm concerned, the only real reason having the Linuxulator is some stuff from Adobe for desktop systems, Flash. That's it. For the scientific stuff, I try to move my people towards OpenSource, since we do standard stuff and I expect students and scientists solving problems without fancy coloured clicky funny things. In production, this might be another point of view. SciLab from INRIA is great, MuPAD, MAXIMA also. But is there a real need running the Linux binary of Forefox on FreeBSD? Regards, Oliver signature.asc Description: OpenPGP digital signature
Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server
Hi, Well, I don't chime in, usually. However, enough is enough. There are many merits to both *BSD and Linux. I don't agree with benchmarks that slant either way, as I'm sure people in both camps will agree. Please be adult and just agree to disagree. Technology applicable to the problem at hand is the only useful thing that we should all agree on. Personally, I don't care if it's BSD, a Linux-variant or even Windoze if it solves the problem. So, if you would all look at all the time spent venting on this idiocy as time that could have been spent coding, debugging, etc, think how much time was wasted from people just reading these e-mails. (Yes, I do and this was an absolute waste of my time.) So, if the communication on the thread was nearly as much going to development and finding issues or the causes of the issues, maybe a scheduler problem would be tracked down, maybe a benchmark issue would be tracked down. Maybe people will stop using RC's versus releases, I don't know. I really don't care. Just please stop with finger pointing and being disgruntled and indignant. FOCUS!! I'd love to say something like Can't we all just get along but we are just so polar in our beliefs, I don't see it happening. Just drop it. If you have something constructive to say, spin off this thread and let the primary thread just die. (It should have a week ago.) Solve the benchmarking issue, solve the scheduler issue, just focus. Paul P. CTO/Owner Atlantis Services Civilized Computing From: O. Hartmann ohart...@mail.zedat.fu-berlin.de To: Daniel Kalchev dan...@digsys.bg Cc: Martin Sugioarto mar...@sugioarto.com; freebsd-current@FreeBSD.ORG; Igor Mozolevsky i...@hybrid-lab.co.uk; freebsd-performa...@freebsd.org; O. Hartmann ohart...@zedat.fu-berlin.de; dan...@freebsd.org Sent: Saturday, December 24, 2011 9:49 AM Subject: Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server On 12/23/11 12:38, Daniel Kalchev wrote: On 23.12.11 12:48, O. Hartmann wrote: Look at Steve Kargls problem. He investigated a SCHED_ULE problem in a way that is far beyond enough! He gave tests, insights of his setup, bad performance compared to SCHED_4BSD and what happend? We are still stuck with this problem and more and more people realise, that FreeBSD does have somewhere a problem and this seems to be a nasty problem not easy to find or investigate. This has made me to realize, that I was having a problem with SCHED_ULE that I was not aware of until now. WOW! :) Every scheduler has some problem, some fail here some fail there. I am confident, that the case that Steve Kargls has reported will be resolved. Another problem is this very elite-feeling closed club. Once you managed it getting into the club of committers or core team members, you'll probably fight for your seat ... I dont propose for that socialists crap Linux people tend to be like, [..] You never heard of the People's Republic of Berkeley? :) As for commiter access, this sort of comments trigger the system administrator in me. I have seen enough people, who for the lack of other excuses always use but I don't have enough RIGHTS!. I am evil, I know But I follow the illusion that if people can see what benchmarks reveal, they start thinking and if the facts are starting to give a heavy load load on those rejecting the facts, they migght change their opinion or get hopefully replaced by more openminded people. Here is now it works: If you see an problem and have a solution: go fix it. Many will be grateful. If you can't fix it, but have an idea how to fix it, share it. May will be grateful. If you can't fix it and don't have any idea, just say there is a problem and stop there. There are many, many, many like you who just hold their breath. We all learn, every day. Daniel Sorry, but your crap is simply breathtaking. oh ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server
On 12/24/2011 12:04, O. Hartmann wrote: There maybe serious reasons having the Linuxulator, i do not know. But if not, why spending rare developer resources on that? As far as I'm concerned, the only real reason having the Linuxulator is some stuff from Adobe for desktop systems, Flash. That's it. Well, Linuxulator allows me to use binary only applications of Linux in FreeBSD without too much problem. I think running Firefox in the Linuxulator is nonsense, because it's supposed that Linuxulator is there for applications that can not be ported to FreeBSD (for example: the code is not open, you bough a privative linux-or-windows-only binary app). Naturally the Linuxulator will always lag behind Linux, but it works for me and I bet I'm not the only one. I think it should only be removed if 1) no one wants to maintain it 2) It obstructs the development of new code. Otherwise, I see no logical reason in this. Regards, Alex. P.S → Also notice that this is an offtopic of the original discussion, sorry. ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server
On 23.12.11 03:17, O. Hartmann wrote: Or even look at the thread regarding to SCHED_ULE. Why has a user, experiencing really worse performance with SCHED_ULE, in a nearly scientific manner some engineer the fault? I'd expect the developer or care-taking engineer taking care in a more user friendly manner. You remember that those developers are not paid to do what they do? You remember that nobody has sold you this OS and promised support in whatever form? Still, this issue is discussed publicly and experiments are being made, I guess also new code is being experimented. If you are interested in the outcome, just follow the discussion. If you can help with something and you are willing, please do. There will be good solution to the SCHED_ULE shortcomings. FreeBSD is unique group of people, who all sit on their eggs, be it eggs they themselves produced, or they inherited one way or another. These people include all the developers and most of the system administrators and users of FreeBSD. There is no they and us. If your preference for the OS is different, you might feel more comfortable in choosing another OS, probably a commercial OS with support from the vendor. If a benchmark reveals some severe weak points in FreeBSD and I have to read about obscure tweaks of non documented sysctl, then this OS would be a no-go if I was a manager to make decissions. Luckily, managers do not care about knobs or how difficult it is for the system administrator to achieve specific goal. All they care is the bottom line in general and in short therm the goals they have set. No sane manager will care about benchmarks, as long as he gets what he wants. Back, to the Phoronix benchmarks. There has already been communication. Phoronix were given advices on how to better do some things on FreeBSD (which will make the quality of their benchmarks better and therefore more trusted). Phoronix has made their updated test suite available to FreeBSD users (that include developers) to try on their own hardware. By the way, it is in /usr/ports/benchmarks/phoronix-test-suite. Linux and FreeBSD are not enemies, they both solve the same problems with different means. Daniel ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server
On 23.12.11 08:47, Martin Sugioarto wrote: A further thing is that I cannot understand the people here sometimes. I would like that the -RELEASE versions of FreeBSD perform well without any further optimizations. The -RELEASE things is just a freeze (or, let's say tested freeze) of the corresponding branch at some time. It is the code available and tested at that time. Thus, it is safe to say that FreeBSD 8.0-RELEASE is much worse than FreeBSD RELENG_8 (still 8.2 at the moment), because years have passed between both code bases, lots of bugs have been discovered and fixed and new technologies have been integrated. Especially in this line, the compiler has changed from 4.2.1 to 4.2.2. When the distribution does not compile with the latest compiler it's simply a bug. FreeBSD is not a distribution. It also compiles with the latest compiler - LLVM. :) I find it amusing, that people want everything compiled with GCC 4.7, which is still very much developing, therefore highly unstable and (probably) full of bugs. Why should one try to penalize the other distribution and downgrade their binaries? Many suggested that the Linux binaries be run via the FreeBSD Linux emulation. Unchanged. There is one problem here though, the emulation is still 32 bit. When FreeBSD has a bad default setup, there must be a reason for that. Tell me this reason and show me that it's justified in form of some other benchmark. FreeBSD has safe default. It is supposed to work out of the box on whatever hardware you put it. As much as it has drives for that hardware, of course. Once you have working installation, you may tweak it all the way you wish. If your installation is pre-optimized, chances are it will crash all the time on you and there will be no easy way for you to fix, short of installing another distribution. Daniel ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server
On 23/12/2011 02:56, Garrett Cooper wrote: On Dec 22, 2011, at 3:58 PM, Jeremy Chadwick free...@jdc.parodius.com wrote: On Fri, Dec 23, 2011 at 12:44:14AM +0100, O. Hartmann wrote: On 12/21/11 19:41, Alexander Leidinger wrote: Hi, while the discussion continued here, some work started at some other place. Now... in case someone here is willing to help instead of talking, feel free to go to http://wiki.freebsd.org/BenchmarkAdvice and have a look what can be improved. The page is far from perfect and needs some additional people which are willing to improve it. This is only part of the problem. A tuning page in the wiki - which could be referenced from the benchmark page - would be great too. Any volunteers? A first step would be to take he tuning-man-page and wikify it. Other tuning sources are welcome too. Every FreeBSD dev with a wiki account can hand out write access to the wiki. The benchmark page gives contributor-access. If someone wants write access create a FirstnameLastname account and ask here for contributor-access. Don't worry if you think your english is not good enough, even some one-word notes can help (and _my_ english got already corrected by other people on the benchmark page). Bye, Alexander. Nice to see movement ;-) But there seems something unclear: man make.conf(5) says, that MALLOC_PRODUCTION is a knob set in /etc/make.conf. The WiJi says, MALLOC_PRODUCTION is to be set in /etc/src.conf. What's right and what's wrong now? I can say with certainty that this value belongs in /etc/make.conf (on RELENG_8 and earlier at least). src/share/mk/bsd.own.mk has no framework for MK_MALLOC_PRODUCTION, so, this is definitely a make.conf variable. Take the advice in tuning(7) with a grain of salt because a number of suggestions are really outdated. I know because I filed a PR last night after I saw how out of synch some of the defaults it claimed were with reality on 9.x+. And I know other suggestions in the manpage are dated as well ;/. There is a wiki page http://wiki.freebsd.org/SystemTuning which is currently more or less tuning(7) with some annotations, the idea being to sort out whats outdated/invalid with an aim of rewriting tuning(7) to be more accurate and useful. I'll grab any info in your pr thats not up there already to keep it updated if thats ok. Vince Thanks, -Garrett___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server
On 23.12.11 12:48, O. Hartmann wrote: Look at Steve Kargls problem. He investigated a SCHED_ULE problem in a way that is far beyond enough! He gave tests, insights of his setup, bad performance compared to SCHED_4BSD and what happend? We are still stuck with this problem and more and more people realise, that FreeBSD does have somewhere a problem and this seems to be a nasty problem not easy to find or investigate. This has made me to realize, that I was having a problem with SCHED_ULE that I was not aware of until now. WOW! :) Every scheduler has some problem, some fail here some fail there. I am confident, that the case that Steve Kargls has reported will be resolved. Another problem is this very elite-feeling closed club. Once you managed it getting into the club of committers or core team members, you'll probably fight for your seat ... I dont propose for that socialists crap Linux people tend to be like, [..] You never heard of the People's Republic of Berkeley? :) As for commiter access, this sort of comments trigger the system administrator in me. I have seen enough people, who for the lack of other excuses always use but I don't have enough RIGHTS!. I am evil, I know But I follow the illusion that if people can see what benchmarks reveal, they start thinking and if the facts are starting to give a heavy load load on those rejecting the facts, they migght change their opinion or get hopefully replaced by more openminded people. Here is now it works: If you see an problem and have a solution: go fix it. Many will be grateful. If you can't fix it, but have an idea how to fix it, share it. May will be grateful. If you can't fix it and don't have any idea, just say there is a problem and stop there. There are many, many, many like you who just hold their breath. We all learn, every day. Daniel ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server
On Fri Dec 23 11, Daniel Kalchev wrote: On 23.12.11 08:47, Martin Sugioarto wrote: A further thing is that I cannot understand the people here sometimes. I would like that the -RELEASE versions of FreeBSD perform well without any further optimizations. The -RELEASE things is just a freeze (or, let's say tested freeze) of the corresponding branch at some time. It is the code available and tested at that time. Thus, it is safe to say that FreeBSD 8.0-RELEASE is much worse than FreeBSD RELENG_8 (still 8.2 at the moment), because years have passed between both code bases, lots of bugs have been discovered and fixed and new technologies have been integrated. Especially in this line, the compiler has changed from 4.2.1 to 4.2.2. When the distribution does not compile with the latest compiler it's simply a bug. FreeBSD is not a distribution. It also compiles with the latest compiler - LLVM. :) I find it amusing, that people want everything compiled with GCC 4.7, which is still very much developing, therefore highly unstable and (probably) full of bugs. Why should one try to penalize the other distribution and downgrade their binaries? Many suggested that the Linux binaries be run via the FreeBSD Linux emulation. Unchanged. There is one problem here though, the emulation is still 32 bit. plus the current emulation layer is far from complete. a lot of stuff hasn't been implemented yet (meaning it's missing or implemented as dummy code). try running recent firefox linux binaries on freebsd. they will all crash almost instantly. cheers. alex When FreeBSD has a bad default setup, there must be a reason for that. Tell me this reason and show me that it's justified in form of some other benchmark. FreeBSD has safe default. It is supposed to work out of the box on whatever hardware you put it. As much as it has drives for that hardware, of course. Once you have working installation, you may tweak it all the way you wish. If your installation is pre-optimized, chances are it will crash all the time on you and there will be no easy way for you to fix, short of installing another distribution. Daniel ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server
On 12/23/11 07:47, Martin Sugioarto wrote: Am Fri, 23 Dec 2011 02:17:00 +0100 schrieb O. Hartmann ohart...@zedat.fu-berlin.de: Benchmarks also could lead developers to look into more details of the weak points of their OS, if they're open for that. Therefore, benchmarks are very useful. But not if any real fault of the OS is excused by a faulty becnhmarking. Hi, it is important for the project to be known and I think that the benchmarks made by Phoronix help FreeBSD to gain popularity, even they look bad sometimes. Furthermore, to make a benchmark is a lot of work and the results are useful, because at the end someone will look at it and will try to improve the results. Thank you for investing your time. I remember that I've made some tests with different platforms i386 vs amd64 with simple tools like openssl speed some time ago and got some bad results for amd64 that no one cared to explain. These bad results weren't reflected on Linux that I tested later for comparison. And most people have a weird attitude to think that the tester measures wrong instead of taking a look at it. They forget that as a FreeBSD user you would rather see FreeBSD win over Linux. I've seen that Phoronix made various benchmarks about FreeBSD compared to Linux and I can tell you that _subjectively_ the benchmarks reflect what I always thought about FreeBSD. I simply _know_ that FreeBSD is worse in concurrency behavior, I know that it has I/O trouble, I know that it is mostly faster emulating 3D games than Linux runs them natively. I knew this already _before_ you published the benchmark about the 3D performance. I cannot see any evil intentions in these benchmarks. All I can see is the wrong attitude _here_. If anyone thinks that Phoronix makes bad benchmarks, they should do these benchmarks by themselves and publish the results. As long as no one tries, Phoronix stays the best reference for me and for everyone else. There IS NO EVIL INTENTION, except, hypothetical, the benchmarker is of the age were he is called a beardless. But: In many articles, there is a very distinguished and underlined emphasizing of Linux that makes me feeling people have their Linux-glasses on. Linux is not UNIX! And if today someone tells me about the Linux-graphical subsystem X11, I turn green in my face ... X11 was, in former days, a development made on UNIX and is adopted by Linux. Ok, we all know that, most of all ... And the aspect of reference: I agree. They do something and this thread arose while they did. And don't forget, benchmarks can never be objective enough and someone will always be mad about the results. Especially, when you present them a versus battle. A further thing is that I cannot understand the people here sometimes. I would like that the -RELEASE versions of FreeBSD perform well without any further optimizations. When the distribution does not compile with the latest compiler it's simply a bug. Why should one try to penalize the other distribution and downgrade their binaries? When FreeBSD has a bad default setup, there must be a reason for that. Tell me this reason and show me that it's justified in form of some other benchmark. Well, look at the mailing list. FreeBSD is handled via this list since we are spread around the globe and even the developers are spread worldwide. But when it comes to detecting worse performnce and someone isn't capable of giving a detailed insight and mostly scientific way of investigating the fault he experienced, the discussion, if it starts, get drowned by allegations like bad testing, worse optimizations blabla. Look at Steve Kargls problem. He investigated a SCHED_ULE problem in a way that is far beyond enough! He gave tests, insights of his setup, bad performance compared to SCHED_4BSD and what happend? We are still stuck with this problem and more and more people realise, that FreeBSD does have somewhere a problem and this seems to be a nasty problem not easy to find or investigate. But look at how Steve has been silenced in the past ... Benchmarks, especially published ones, reveal those pits and soemone could look into it. Another problem is this very elite-feeling closed club. Once you managed it getting into the club of committers or core team members, you'll probably fight for your seat ... I dont propose for that socialists crap Linux people tend to be like, overcrowded townhalls full of important people with non-consense opinions. The other extreme end of this spectrum. I can not change this. And I do not know whether there is a real way-in-the-middle. But I follow the illusion that if people can see what benchmarks reveal, they start thinking and if the facts are starting to give a heavy load load on those rejecting the facts, they migght change their opinion or get hopefully replaced by more openminded people. A Vision. Oliver ___ freebsd-current@freebsd.org mailing list
Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server
On 12/23/11 10:07, Daniel Kalchev wrote: On 23.12.11 03:17, O. Hartmann wrote: Or even look at the thread regarding to SCHED_ULE. Why has a user, experiencing really worse performance with SCHED_ULE, in a nearly scientific manner some engineer the fault? I'd expect the developer or care-taking engineer taking care in a more user friendly manner. You remember that those developers are not paid to do what they do? You remember that nobody has sold you this OS and promised support in whatever form? Well, as far as I know, the FreeBSD project is funding people doing a certain work! So, the implied opposit, FreeBSD is developed free isn't true. Still, this issue is discussed publicly and experiments are being made, I guess also new code is being experimented. If you are interested in the outcome, just follow the discussion. If you can help with something and you are willing, please do. There will be good solution to the SCHED_ULE shortcomings. FreeBSD is unique group of people, who all sit on their eggs, be it eggs they themselves produced, or they inherited one way or another. These people include all the developers and most of the system administrators and users of FreeBSD. There is no they and us. What I am in this terminology? I can hardly write some scientific code for my science, I'm able to patch software a bit, that has been developed only for Linux these days (ISIS3 from the USGS, for instance, but I do not dare to publish the crap of port I produced since it is not professional). I'm with FreeBSD now since 1996/97. I'm still with the system, although I desperately need scientific grade compilers or GPGPU support. So, even if Linux offers me a really much more convenient way to do my work, I stayed with FreeBSD since there is no real alternative in terms of cleaness of the system. I have also to administer an Ubuntu and Suse server and I feel not amused by this script hell that covers the real system just to get kiddies or Windows-Admins into the admin-position. And, I dare to put some critics herein! Since I see that FreeBSD is free, why not trying to make it better and more towards perfect? If your preference for the OS is different, you might feel more comfortable in choosing another OS, probably a commercial OS with support from the vendor. This is nonesense, you know that, regarding to my case. If a benchmark reveals some severe weak points in FreeBSD and I have to read about obscure tweaks of non documented sysctl, then this OS would be a no-go if I was a manager to make decissions. Luckily, managers do not care about knobs or how difficult it is for the system administrator to achieve specific goal. All they care is the bottom line in general and in short therm the goals they have set. No sane manager will care about benchmarks, as long as he gets what he wants. Well, in real world and beyond this armchair polemics, managers at last do the decissions. Those people dropping math, physics, with no glue to how things work in nature get a degree in law, business and whatsowever and then decide. In my eyes, those are enemies of every development and progress, but this is polemics, too. I faced this many times and it is hard to convince those people not taking care of knobs. But as an admin myself, I need to know about knobs and if essential knobs are not documented, than there is a potential gone for the OS in question. Look at FreeBSD and the problem of how well sysctls and their working are documented. It needs to be fixed. Back, to the Phoronix benchmarks. There has already been communication. Phoronix were given advices on how to better do some things on FreeBSD (which will make the quality of their benchmarks better and therefore more trusted). Phoronix has made their updated test suite available to FreeBSD users (that include developers) to try on their own hardware. By the way, it is in /usr/ports/benchmarks/phoronix-test-suite. Yes, everyone interseted in this thread and communicating is aware of that fact. Linux and FreeBSD are not enemies, they both solve the same problems with different means. Daniel ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server
Am Fri, 23 Dec 2011 11:18:03 +0200 schrieb Daniel Kalchev dan...@digsys.bg: The -RELEASE things is just a freeze (or, let's say tested freeze) of the corresponding branch at some time. It is the code available and tested at that time. Hi Daniel, obviously performance is not a quality aspect, only stability. FreeBSD is not a distribution. It also compiles with the latest compiler - LLVM. :) I thought that the D in FreeBSD stands for distribution. Yes, it's ok that it compiles with LLVM. Does it also run faster in benchmarks? I find it amusing, that people want everything compiled with GCC 4.7, which is still very much developing, therefore highly unstable and (probably) full of bugs. When you don't use the software don't complain that it is buggy, because you won't find the bugs. You cannot always tell the others to make everything perfect. I don't want to have everything compiled on $COMPILER. I want that there is a reasonable quality. And for me quality is not only stability, but also speed. Many suggested that the Linux binaries be run via the FreeBSD Linux emulation. Unchanged. There is one problem here though, the emulation is still 32 bit. I'm not talking about emulation. I don't use FreeBSD to run emulated binaries. I (any many people) want efficient servers and eventually desktops. You should not expect people to tune the system for speed, when it's clear that default setting does not make any sense. People will use default settings, because they trust developers that they thought about balanced stability, security and performance. FreeBSD has safe default. This is what I am talking about. Don't complain that the benchmark does not show efficience. No one is interested in tuning FreeBSD just for a benchmark application. It is supposed to work out of the box on whatever hardware you put it. As much as it has drives for that hardware, of course. Once you have working installation, you may tweak it all the way you wish. But if you don't tweak, you get a fair result in a benchmark. This is what you will see as a user of the system. These are the default settings, that means developers chose them as the BEST choice for the system. If your installation is pre-optimized, chances are it will crash all the time on you and there will be no easy way for you to fix, short of installing another distribution. Sorry, no. If optimization makes bugs appear, there are bugs in the code (somewhere). And you will never find them when you hide them like this. You will also never see many advances in performance. -- Martin signature.asc Description: PGP signature
FreeBSD funding [was: Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1] Server
I have slightly reordered your email in my reply, in order to put the most important item last. On Fri, Dec 23, 2011 at 12:01:33PM +0100, O. Hartmann wrote: I'm still with the system, although I desperately need scientific grade compilers or GPGPU support. Your use-case, while valid, is clearly not the use-case that most of the committers working on FreeBSD face. But see below. And, I dare to put some critics herein! Since I see that FreeBSD is free, why not trying to make it better and more towards perfect? Everyone wants the product to improve. The question is, what is achievable with the current committers? That's where you see the pushback and frustration from the current committers. Look at FreeBSD and the problem of how well sysctls and their working are documented. It needs to be fixed. There's no argument that some of the FreeBSD documentation is stale. We do, however, have one committer (eadler@) who has been trying to move the sysctl documentation forwards. Participation from the wider community is key. Although sending PRs does not guarantee things will get fixed, it's currently the best way that we have. Well, as far as I know, the FreeBSD project is funding people doing a certain work! So, the implied opposite, FreeBSD is developed free isn't true. So here's the key point of your email IMHO, and the key misunderstanding. First, let me nit-pick the legalities. The FreeBSD Foundation (http://www.freebsdfoundation.org/) is a US non-profit that does fund some activities, and that's what I'll talk about here. The FreeBSD Project is the collective term for all of the committers and developers and is not an entity for US legal purposes. Second, the disclaimers: I am not a member of the FreeBSD Foundation Board of Directors, so I am not speaking for them. I have also directly benefited from Foundation funding (both travel, and via equipment they bought for portmgr), so am hardly unbiased. Now on to the gist of that matter. As a US non-profit, the Foundation is required to post its financial information to the public, and it does on its website: http://www.freebsdfoundation.org/documents/Budget2011.pdf You'll see here that the total budget for 2011 is $400k (USD). This, frankly, is miniscule. The largest line item for 2011 is $125k for project funding, which has gone towards 9 different projects (see http://www.freebsdfoundation.org/activities.shtml). For comparison, keep in mind that a commercial developers' salary in the US is upwards of $100k/yr. Even with this being a substantial increase from 2010's $83k, these numbers are tiny comared to real-world budgets. The projects that were sponsored were primarily networking-related, but also the GEM/KMS/DRI project, jails, the libc++ replacement, and clocks. I've listed those in the order that I think the most consumers of FreeBSD will be affected by. Note the absence of any work towards performance, schedulers, compilers, or numerical analysis. With a $125k budget, you're simply not going to see those on the list. The other notable line items are: hardware purchases (explicit disclaimer: portmgr has been one of the primary beneficiaries); conference sponsorship; conference travel; and salary for one employee to try to help coordinate all the above. Legal fees (things involving trademarking and licensing issues) takes up most of the remaining. I can't figure out the Linux Foundation's budget from their website, but I can tell immediately that their budget is a great many times more than $400k. Summary: on a fraction of the budget that Linux has available, we _nearly keep up_. I can't imagine what we could do with comparable funding. So, for everyone who thinks we are being well funded, here's your reality check. And please note that the Foundation is in its year-end fund drive, too. Thanks for listening. mcl ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server
On Thursday, December 22, 2011 6:58:46 pm Jeremy Chadwick wrote: On Fri, Dec 23, 2011 at 12:44:14AM +0100, O. Hartmann wrote: On 12/21/11 19:41, Alexander Leidinger wrote: Hi, while the discussion continued here, some work started at some other place. Now... in case someone here is willing to help instead of talking, feel free to go to http://wiki.freebsd.org/BenchmarkAdvice and have a look what can be improved. The page is far from perfect and needs some additional people which are willing to improve it. This is only part of the problem. A tuning page in the wiki - which could be referenced from the benchmark page - would be great too. Any volunteers? A first step would be to take he tuning-man-page and wikify it. Other tuning sources are welcome too. Every FreeBSD dev with a wiki account can hand out write access to the wiki. The benchmark page gives contributor-access. If someone wants write access create a FirstnameLastname account and ask here for contributor-access. Don't worry if you think your english is not good enough, even some one- word notes can help (and _my_ english got already corrected by other people on the benchmark page). Bye, Alexander. Nice to see movement ;-) But there seems something unclear: man make.conf(5) says, that MALLOC_PRODUCTION is a knob set in /etc/make.conf. The WiJi says, MALLOC_PRODUCTION is to be set in /etc/src.conf. What's right and what's wrong now? I can say with certainty that this value belongs in /etc/make.conf (on RELENG_8 and earlier at least). src/share/mk/bsd.own.mk has no framework for MK_MALLOC_PRODUCTION, so, this is definitely a make.conf variable. Eh, normal make variables can go in src.conf as well. They do not have to be listed in bsd.own.mk. World builds include /etc/src.conf whereas every make invocation includes /etc/make.conf via sys.mk. The only reason to use /etc/src.conf is to have a place to put variables only affect make buildworld / buildkernel but do not affect other make invocations. Also, MALLOC_PRODUCTION is generally enabled in a stable branch as part of making the stable branch, there should be no need to set it manually in a stable branch. -- John Baldwin ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server
On 23.12.11 16:47, Martin Sugioarto wrote: I thought that the D in FreeBSD stands for distribution. Yes, it's ok that it compiles with LLVM. Does it also run faster in benchmarks? It does. From a language perspective. It is a distribution, because at the times BSD was developed, it was not a complete operating system. It was supposed to be added to say ATT System V to make it networking capable etc. The Linux people use the word distribution in a different context. I don't want to have everything compiled on $COMPILER. I want that there is a reasonable quality. And for me quality is not only stability, but also speed. You can always have faster algorithm if it is not necessary to produce the right answer. But if you don't tweak, you get a fair result in a benchmark. This is what you will see as a user of the system. These are the default settings, that means developers chose them as the BEST choice for the system. Developers are not Gods. Developers have no clue on what system and for what purpose you will use the software. All they may do for you is to provide enough knobs for you to tune your system for your hardware/application and also make sure that the system scales, when you turn the knobs. Daniel ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server
On Fri, Dec 23, 2011 at 10:00:05AM -0500, John Baldwin wrote:
On Thursday, December 22, 2011 6:58:46 pm Jeremy Chadwick wrote:
On Fri, Dec 23, 2011 at 12:44:14AM +0100, O. Hartmann wrote:
On 12/21/11 19:41, Alexander Leidinger wrote:
Hi,
while the discussion continued here, some work started at some other
place. Now... in case someone here is willing to help instead of talking,
feel
free to go to http://wiki.freebsd.org/BenchmarkAdvice and have a look what
can
be improved. The page is far from perfect and needs some additional people
which are willing to improve it.
This is only part of the problem. A tuning page in the wiki - which
could be referenced from the benchmark page - would be great too. Any
volunteers? A first step would be to take he tuning-man-page and wikify it.
Other tuning sources are welcome too.
Every FreeBSD dev with a wiki account can hand out write access to the
wiki. The benchmark page gives contributor-access. If someone wants write
access create a FirstnameLastname account and ask here for contributor-access.
Don't worry if you think your english is not good enough, even some one-
word notes can help (and _my_ english got already corrected by other people
on
the benchmark page).
Bye,
Alexander.
Nice to see movement ;-)
But there seems something unclear:
man make.conf(5) says, that MALLOC_PRODUCTION is a knob set in
/etc/make.conf.
The WiJi says, MALLOC_PRODUCTION is to be set in /etc/src.conf.
What's right and what's wrong now?
I can say with certainty that this value belongs in /etc/make.conf
(on RELENG_8 and earlier at least).
src/share/mk/bsd.own.mk has no framework for MK_MALLOC_PRODUCTION,
so, this is definitely a make.conf variable.
Eh, normal make variables can go in src.conf as well. They do not have
to be listed in bsd.own.mk. World builds include /etc/src.conf whereas
every make invocation includes /etc/make.conf via sys.mk. The only reason
to use /etc/src.conf is to have a place to put variables only affect
make buildworld / buildkernel but do not affect other make invocations.
I was always under the impression src.conf(5) variables had to be
manually added to bsd.own.mk and similar bits (e.g.
src/tools/build/options/WITH_xxx which is what's used to create the
src.conf(5) man page), but upon your comment and manual investigation on
my part, I found you're indeed right. Taken from bsd.own.mk:
107 .if !defined(_WITHOUT_SRCCONF)
108 SRCCONF?= /etc/src.conf
109 .if exists(${SRCCONF})
110 .include ${SRCCONF}
111 .endif
112 .endif
As long as third-party software doesn't depend on MALLOC_PRODUCTION for
something (I don't know why something would, but who knows; maybe
there's a third-party malloc implementation which might?), then putting
it in src.conf would be fine (src/lib/libc/stdlib files reference it).
--
| Jeremy Chadwickjdc at parodius.com |
| Parodius Networking http://www.parodius.com/ |
| UNIX Systems Administrator Mountain View, CA, US |
| Making life hard for others since 1977. PGP 4BD6C0CB |
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server
On 12/23/11 15:47, Martin Sugioarto wrote: Am Fri, 23 Dec 2011 11:18:03 +0200 schrieb Daniel Kalchev dan...@digsys.bg: The -RELEASE things is just a freeze (or, let's say tested freeze) of the corresponding branch at some time. It is the code available and tested at that time. Hi Daniel, obviously performance is not a quality aspect, only stability. FreeBSD is not a distribution. It also compiles with the latest compiler - LLVM. :) I thought that the D in FreeBSD stands for distribution. Yes, it's ok that it compiles with LLVM. Does it also run faster in benchmarks? I find it amusing, that people want everything compiled with GCC 4.7, which is still very much developing, therefore highly unstable and (probably) full of bugs. When you don't use the software don't complain that it is buggy, because you won't find the bugs. You cannot always tell the others to make everything perfect. As with GCC4.7, CLANG/LLVM is still considered experimental and definitely has some issues with CPU architectures beyond Core2. Personally, I compile everthing now with CLANG on FreeBSD 9.0/10.0 as far as I don't realise any conerns towards correctness and stability. Well, the GCC 4.7 came somewhere up and I picked it up, sorry. It is much easier to replace gcc 4.7 in this thread by 4.6.2, which is now considered stable and in production. And as some of the writers in this thread mentioned, the performance gain could be enormous since gcc 4.6 does support either core i7 architectures and its new facilities, the optimizer is aware of the core/uncore design an, maybe, of the three-folded cache levels. Is the legacy gcc 4.2 aware of that? I guess not, since it does not support architectures beyond Core2. I tried using gcc 4.6.2 from ports to compile world, but I failed. Simply replacing/setting CC, CXX and CPP isn't obviosuly enough. I don't want to have everything compiled on $COMPILER. I want that there is a reasonable quality. And for me quality is not only stability, but also speed. Yes, agree. I think quality could inherit also a reasonable speed. Speed at all costs, even stability, is no option. Even for HPC systems, where jobs run uninetrupted for weeks or months (in our case). Many suggested that the Linux binaries be run via the FreeBSD Linux emulation. Unchanged. There is one problem here though, the emulation is still 32 bit. With the usage of even 32bit Linux binaries you introduce all the mess you want to avoid by using FreeBSD. But it is very often recommended to use the so called Linuxulator. I'm happy to have this opportunity (I can not run FreeBSD binaries on some Ubuntu or Centos distros). But in some cases people of the FreeBSD community rely to much on this 32bit-limited option. I always prefer native BLOBs over emulated BLOBs. I'm not talking about emulation. I don't use FreeBSD to run emulated binaries. I (any many people) want efficient servers and eventually desktops. You should not expect people to tune the system for speed, when it's clear that default setting does not make any sense. People will use default settings, because they trust developers that they thought about balanced stability, security and performance. FreeBSD has safe default. This is what I am talking about. Don't complain that the benchmark does not show efficience. No one is interested in tuning FreeBSD just for a benchmark application. It is supposed to work out of the box on whatever hardware you put it. As much as it has drives for that hardware, of course. Once you have working installation, you may tweak it all the way you wish. But if you don't tweak, you get a fair result in a benchmark. This is what you will see as a user of the system. These are the default settings, that means developers chose them as the BEST choice for the system. Well, it is a very nice moce to have conservative settings to make FreeBSD stable for everyone intend to use it out of the box. But what I really miss is a certain, group of people dedicated to HPC and secure, stable tweak achieving that. The operating system is a nature and live. It is a balance of a limited resource. One can try to balance out every potential workload that can occur and the result is a very good allround syste, But in the server or HPC area, it might be necessary to push some parts in favor of some others. When computing, I do not need high USB performance, except a responsive keyboard. I/O and CPU performance is the main goal, but this seems the most difficult part. A file or network server, for instance, would balance more towards network I/O or delivering small data pieces instead of large streaming blocks of memory. I'm certain that the tweaks would differ for both scenarios. At home or at the desktop, the situation is more complicated, since people tend to use a lot of multimedia stuff and jumping audio is also not a very pleasant thing as stuck video. If your installation is
Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server
On 12/23/11 16:24, Jeremy Chadwick wrote:
On Fri, Dec 23, 2011 at 10:00:05AM -0500, John Baldwin wrote:
On Thursday, December 22, 2011 6:58:46 pm Jeremy Chadwick wrote:
On Fri, Dec 23, 2011 at 12:44:14AM +0100, O. Hartmann wrote:
On 12/21/11 19:41, Alexander Leidinger wrote:
Hi,
while the discussion continued here, some work started at some other
place. Now... in case someone here is willing to help instead of talking,
feel
free to go to http://wiki.freebsd.org/BenchmarkAdvice and have a look what
can
be improved. The page is far from perfect and needs some additional people
which are willing to improve it.
This is only part of the problem. A tuning page in the wiki - which
could be referenced from the benchmark page - would be great too. Any
volunteers? A first step would be to take he tuning-man-page and wikify it.
Other tuning sources are welcome too.
Every FreeBSD dev with a wiki account can hand out write access to the
wiki. The benchmark page gives contributor-access. If someone wants write
access create a FirstnameLastname account and ask here for
contributor-access.
Don't worry if you think your english is not good enough, even some one-
word notes can help (and _my_ english got already corrected by other people
on
the benchmark page).
Bye,
Alexander.
Nice to see movement ;-)
But there seems something unclear:
man make.conf(5) says, that MALLOC_PRODUCTION is a knob set in
/etc/make.conf.
The WiJi says, MALLOC_PRODUCTION is to be set in /etc/src.conf.
What's right and what's wrong now?
I can say with certainty that this value belongs in /etc/make.conf
(on RELENG_8 and earlier at least).
src/share/mk/bsd.own.mk has no framework for MK_MALLOC_PRODUCTION,
so, this is definitely a make.conf variable.
Eh, normal make variables can go in src.conf as well. They do not have
to be listed in bsd.own.mk. World builds include /etc/src.conf whereas
every make invocation includes /etc/make.conf via sys.mk. The only reason
to use /etc/src.conf is to have a place to put variables only affect
make buildworld / buildkernel but do not affect other make invocations.
I was always under the impression src.conf(5) variables had to be
manually added to bsd.own.mk and similar bits (e.g.
src/tools/build/options/WITH_xxx which is what's used to create the
src.conf(5) man page), but upon your comment and manual investigation on
my part, I found you're indeed right. Taken from bsd.own.mk:
107 .if !defined(_WITHOUT_SRCCONF)
108 SRCCONF?= /etc/src.conf
109 .if exists(${SRCCONF})
110 .include ${SRCCONF}
111 .endif
112 .endif
As long as third-party software doesn't depend on MALLOC_PRODUCTION for
something (I don't know why something would, but who knows; maybe
there's a third-party malloc implementation which might?), then putting
it in src.conf would be fine (src/lib/libc/stdlib files reference it).
Then the manpage should reflect this. man src.conf does not show up
MALLOC_PRODUCTIOn, but man make.conf does. If the latter is right, then
it should be worth mentioned that make.conf is incorporating src.conf.
Just a suggestion.
Regards,
Oliver
signature.asc
Description: OpenPGP digital signature
Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server
Hi, I think this thread has gone far, far off the rails. If you're able to provide some solid debugging or willing to put in the effort to provide said solid debugging, then great. The easier you can make it for someone to fix for you (whether they're a FreeBSD committer or otherwise) the more likely it'll be fixed. There's no-one notionally in charge and paid to look after the scheduler. This is the unfortunate truth. No amount of saying but but people are paid to do this! will fix that particular point. The way that 99% of FreeBSD work gets done is when someone (who is a committer or otherwise) gets angry at how something doesn't quite work for them, and they decide to go and do something about it. The only point where a committer needs be involved is when someone wants to push their code into upstream (to borrow a Linux-ism) FreeBSD. If you're able to setup KTR and drive it + schedgraph (just like Steve has) and run this on a workload that is _repeatedly_ broken for you, then you're immediately going to have a better chance at getting it fixed. Bonus points if you can run the same benchmark on 4BSD and ULE, reporting KTR + schedgraph traces for both. That is going to be _by far_ the most helpful thing anyone can do in this ridiculously overly-verbose thread. Come on guys/girls/fuzzy creatures, you want to fix the problem? Bitching about it won't help. Unless you're like me and have an interest in Linguistics and end up writing a flame war to code generator. Then we'll likely be fine. :) Adrian ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server
Hi, I think this thread has gone far, far off the rails. If you're able to provide some solid debugging or willing to put in the effort to provide said solid debugging, then great. The easier you can make it for someone to fix for you (whether they're a FreeBSD committer or otherwise) the more likely it'll be fixed. There's no-one notionally in charge and paid to look after the scheduler. This is the unfortunate truth. No amount of saying but but people are paid to do this! will fix that particular point. The way that 99% of FreeBSD work gets done is when someone (who is a committer or otherwise) gets angry at how something doesn't quite work for them, and they decide to go and do something about it. The only point where a committer needs be involved is when someone wants to push their code into upstream (to borrow a Linux-ism) FreeBSD. If you're able to setup KTR and drive it + schedgraph (just like Steve has) and run this on a workload that is _repeatedly_ broken for you, then you're immediately going to have a better chance at getting it fixed. Bonus points if you can run the same benchmark on 4BSD and ULE, reporting KTR + schedgraph traces for both. That is going to be _by far_ the most helpful thing anyone can do in this ridiculously overly-verbose thread. Come on guys/girls/fuzzy creatr ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server
On Fri, Dec 23, 2011 at 2:38 AM, Vincent Hoffman vi...@unsane.co.uk wrote: On 23/12/2011 02:56, Garrett Cooper wrote: On Dec 22, 2011, at 3:58 PM, Jeremy Chadwick free...@jdc.parodius.com wrote: On Fri, Dec 23, 2011 at 12:44:14AM +0100, O. Hartmann wrote: On 12/21/11 19:41, Alexander Leidinger wrote: Hi, while the discussion continued here, some work started at some other place. Now... in case someone here is willing to help instead of talking, feel free to go to http://wiki.freebsd.org/BenchmarkAdvice and have a look what can be improved. The page is far from perfect and needs some additional people which are willing to improve it. This is only part of the problem. A tuning page in the wiki - which could be referenced from the benchmark page - would be great too. Any volunteers? A first step would be to take he tuning-man-page and wikify it. Other tuning sources are welcome too. Every FreeBSD dev with a wiki account can hand out write access to the wiki. The benchmark page gives contributor-access. If someone wants write access create a FirstnameLastname account and ask here for contributor-access. Don't worry if you think your english is not good enough, even some one-word notes can help (and _my_ english got already corrected by other people on the benchmark page). Bye, Alexander. Nice to see movement ;-) But there seems something unclear: man make.conf(5) says, that MALLOC_PRODUCTION is a knob set in /etc/make.conf. The WiJi says, MALLOC_PRODUCTION is to be set in /etc/src.conf. What's right and what's wrong now? I can say with certainty that this value belongs in /etc/make.conf (on RELENG_8 and earlier at least). src/share/mk/bsd.own.mk has no framework for MK_MALLOC_PRODUCTION, so, this is definitely a make.conf variable. Take the advice in tuning(7) with a grain of salt because a number of suggestions are really outdated. I know because I filed a PR last night after I saw how out of synch some of the defaults it claimed were with reality on 9.x+. And I know other suggestions in the manpage are dated as well ;/. There is a wiki page http://wiki.freebsd.org/SystemTuning which is currently more or less tuning(7) with some annotations, the idea being to sort out whats outdated/invalid with an aim of rewriting tuning(7) to be more accurate and useful. I'll grab any info in your pr thats not up there already to keep it updated if thats ok. Sure. Please take my suggestions (apart from the networking sysctls) with a grain of salt as I didn't look at the sourcecode for the filesystem ones (I was going off the top of my head and other emails I had seen passed around). I'll update the tuning 'wiki' with mention of the new networking defaults. If we want to make this manpage 'timeless', should we remove mention of defaults and go off basic guidelines (if you set this higher, you'll get better performance in scenario, X.Y.Z, etc)? Thanks! -Garrett ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server
On 23/12/2011 20:23, Garrett Cooper wrote: On Fri, Dec 23, 2011 at 2:38 AM, Vincent Hoffman vi...@unsane.co.uk wrote: On 23/12/2011 02:56, Garrett Cooper wrote: snip There is a wiki page http://wiki.freebsd.org/SystemTuning which is currently more or less tuning(7) with some annotations, the idea being to sort out whats outdated/invalid with an aim of rewriting tuning(7) to be more accurate and useful. I'll grab any info in your pr thats not up there already to keep it updated if thats ok. Sure. Please take my suggestions (apart from the networking sysctls) with a grain of salt as I didn't look at the sourcecode for the filesystem ones (I was going off the top of my head and other emails I had seen passed around). I'll update the tuning 'wiki' with mention of the new networking defaults. If we want to make this manpage 'timeless', should we remove mention of defaults and go off basic guidelines (if you set this higher, you'll get better performance in scenario, X.Y.Z, etc)? Thanks! -Garrett Good point, for tuning the defaults are probably not so important as they are likely to change at some point (as the current man page will attest) so maybe its less important to document them. Happy Christmas (or holiday of your choice ;) to you all and I hope everyone has a good new year. Vince ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server
Hi, I extended the gcc part a little bit to make it a little bit more clear when it matters. Bye, Alexander. -- Send via an Android device, please forgive brevity and typographic and spelling errors. Stefan Esser s...@freebsd.org hat geschrieben:Am 21.12.2011 22:49, schrieb Johan Hendriks: Nice page, but one thing i do not get is the following. [quote] If you compare FreeBSD / GCC 4.2.1 against, for example, Ubuntu / GCC 4.7 then the results are unlikely to tell you anything meaningful about FreeBSD vs Ubuntu. [/quote] That is a little strange in my opinion. It tells me that FreeBSD falls more and more behind on Linux. The reason is or could be that FreeBSD cannot or will not include GCC 4.7 and that FreeBSD will not be on par with Linux anymore. To compare it with Formula1 cars. If Mercedes decide to use the engine from 2 seasons back (the engine version 4.2.1) in there 2012 car, and Ferrari uses there new Engine (version 4.7). Can we not compare them anymore because of the decission from Mercedes to use the old engine? No we just say, if you want to win a race, get the Ferrari. It is the reallity, FreeBSD uses 4.2.1 as there compiler!!! As has been pointed out by others, FreeBSD ships with gcc-4.2.1 (with some local modifications and fixes) as the system compiler. If you tune up FreeBSD to use the GCC 4.7 compiler, or downgrade linux to 4.2.1, then that will tell me nothing about FreeBSD vs Linux. The gcc version distributed with FreeBSD was chosen for license reasons, not for technical reasons. If you are OK with installing a GPLv3 licensed compiler on your systems, then just do it and take advantage of the improved code generated by it. I my opinion, you benchmark the latest release of Linux, FreeBSD, Solaris, Windows and whatever OS you want to compare! As you probably know, Linux is just the kernel and the distributions add user space programs, including a compiler. You can easily create a FreeBSD distribution with more advanced compiler and use or even sell it. But the FreeBSD project was cautious to not heavily depend on a GPLv3 compiler (for reasons openly discussed at the time this decision was made). You want to benchmark the release and not a tuned version against a standard version. And that in general are the versions most of us users will use. If you compare operating systems from a technical point of view, then you'll be interested in relative performance of algorithms and methods chosen. This is best achieved by using the same compiler for each of the candidates. If you compare performance from a user point of view, you are correct that performance delivered out of the box (without complicated tuning) may be, what counts for most users. But those users that depend on best performance e.g. for a FreeBSD based embedded product or a data center, may tune the system, including compilation with a newer compiler than the system default. And what if in the future LLVM gets on par with Linux, is it stil fair to compare FreeBSD with Linux? You can always compare anything with whatever you like (even apples with oranges), but you need to be aware of what you compare and what your goals are, to be able to draw reasonable conclusions. If you want to test out of the box performance, then test with system compilers (or just those binaries delivered with the system). If you want to test for code efficiency or scalability, then use the same compilers for each system under test to remove differences introduced by the compilers (which are an external component not developed by the FreeBSD people). Or do we say, well we are on par, but it is not fair, yes we used the latest releases, but you can not blame Linux because they are still using GCC. Depends on what you want or need to measure ... No what we will see then are haleluja blogs that FreeBSD is on par with Linux. Such blog messages are not common in the FreeBSD community. FreeBSD used to have big technical and performance advantages when Linux was young, but even then, there was technical discussion between camps (and many concepts were implemented in Linux based on BSD examples; I have taken part in such discussions myself, some 15 to 20 years back). For me peformance is not a show stopper, and for the most of us i think it is not. FreeBSD for me is a clean system that does the job perfect and has a very helpful community. Well, this are valid aspects, too, and very hard to with benchmarks ;-) Regards, STefan ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server
Stefan Esser schreef: Am 21.12.2011 22:49, schrieb Johan Hendriks: Nice page, but one thing i do not get is the following. [quote] If you compare FreeBSD / GCC 4.2.1 against, for example, Ubuntu / GCC 4.7 then the results are unlikely to tell you anything meaningful about FreeBSD vs Ubuntu. [/quote] That is a little strange in my opinion. It tells me that FreeBSD falls more and more behind on Linux. The reason is or could be that FreeBSD cannot or will not include GCC 4.7 and that FreeBSD will not be on par with Linux anymore. To compare it with Formula1 cars. If Mercedes decide to use the engine from 2 seasons back (the engine version 4.2.1) in there 2012 car, and Ferrari uses there new Engine (version 4.7). Can we not compare them anymore because of the decission from Mercedes to use the old engine? No we just say, if you want to win a race, get the Ferrari. It is the reallity, FreeBSD uses 4.2.1 as there compiler!!! As has been pointed out by others, FreeBSD ships with gcc-4.2.1 (with some local modifications and fixes) as the system compiler. If you tune up FreeBSD to use the GCC 4.7 compiler, or downgrade linux to 4.2.1, then that will tell me nothing about FreeBSD vs Linux. The gcc version distributed with FreeBSD was chosen for license reasons, not for technical reasons. If you are OK with installing a GPLv3 licensed compiler on your systems, then just do it and take advantage of the improved code generated by it. It does not matter what the decission is to use the old compiler, it is a fact that the base comes with 4.2.x Does that mean we can not compare/benchmark against other distributions because they use GPLv3 stuff. No, i want to know where standard released FreeBSD stands against standard released Linux distributions. If you compare benchmark userland applictions, then it is fair to use the latest compiler for the userland software also on FreeBSD. But what if the ports tree defaults to LLVM, then again we want to know what FreeBSD does against a Linux distribution. Why because that is what most of us will be using...!! If we start to compile all the ports with gcc 4.7 to be on par in comparising and benchmarking, why spend all the time getting LLVM as the default compiler for ports also? Why not take that effort into making the WHOLE ports tree to compile with GCC4.7? Reason, because FreeBSD goes the LLVM route. That is a decission FreeBSD is making! And that choise will be the FreeBSD that is used in comparising and benchmarks on the net , not the utterly overcopiled and tuned FreeBSD against stock Ubuntu or whatever Linux distribution. If it is a good or bad choice! That we will see in the comparising/benchmarks we will be seeing when that time comes. Same goes for the scheduler! and all the other subsystems FreeBSD has choosen, that makes FreeBSD. I my opinion, you benchmark the latest release of Linux, FreeBSD, Solaris, Windows and whatever OS you want to compare! As you probably know, Linux is just the kernel and the distributions add user space programs, including a compiler. You can easily create a FreeBSD distribution with more advanced compiler and use or even sell it. But the FreeBSD project was cautious to not heavily depend on a GPLv3 compiler (for reasons openly discussed at the time this decision was made). I know Linux is a kernel, re read Linux as Linux Distribution! Yes you can use a more advanced compiler on FreeBSD, BUT you can do that on Linux also ,so where do you stop? Are you going to spend a month to compare a fullly tuned up FreeBSD system against a Linux distribution? No because the users will not spend months tuning and recompile there servers. They use the FreeBSD version that comes with the CD! And that we want to compare/benchmark against a Linux distribution. You want to benchmark the release and not a tuned version against a standard version. And that in general are the versions most of us users will use. If you compare operating systems from a technical point of view, then you'll be interested in relative performance of algorithms and methods chosen. This is best achieved by using the same compiler for each of the candidates. If you compare performance from a user point of view, you are correct that performance delivered out of the box (without complicated tuning) may be, what counts for most users. But those users that depend on best performance e.g. for a FreeBSD based embedded product or a data center, may tune the system, including compilation with a newer compiler than the system default. And what if in the future LLVM gets on par with Linux, is it stil fair to compare FreeBSD with Linux? You can always compare anything with whatever you like (even apples with oranges), but you need to be aware of what you compare and what your goals are, to be able to draw reasonable conclusions. If you want to test out of the box performance, then test with system compilers (or just those binaries delivered with the
Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server
My thoughts about benchmarking - don't forget, it's the way to get at least estimate on how your system will behave in given circumstances. When testers measured new videocard, they tested few factors, like FPS in modern games, pixel/texture fillrate, and whatever they do there else. That's because videocard have few simple and plain applications. And different vendor/generation cards are compared without problems. Different version of compiler, OS, etc - is very irrelevant. It's like say we can't compare these shovels, because they made from different sort of wood. OS is created to serve (produce some useful actions), and not to be measured and turned off forever:) So, in ideal world, there will be benchmarks on some real-world situations (like FPS for videocards, there got to be something also if not common, but at least wide-spread amongst users). Like - benchmarking fully tuned FreeBSD vs fully tuned Linux in high net-load (for example http + php + mysql). It's hard to disagree that this is very common spread use case for server OS. Also good tests would be productivity of FTP/File/Samba/Nfs/rsync servers. For desktop aspects, there's not much space for tuning (for linuxes), mostly linuxes tested out-of-box, or tuned via some gui-settings applet;, while FreeBSD-OOB needs some additional care (like get latest ports, install latest video drivers, xorg, etc., sysctl tuning probably). I'm glad there's PC-BSD, and PC-BSD can be used for desktop testing. And what to test in desktop? IMHO - WM responsiveness; - Program multi-tasking, and how productivity decreases when many background program working; (It's like, which user experience we'll get when our system is pretty heavy loaded) - Probably would be fair to compare same software in same circumstances (like same version of FF, Chromium, maybe something else). There's such extension for FF imacros - which can be used to simulate user actions, any actions in many tabs; - Overall usage experience, like measure time between program launching and window appearing (file managers, browsers, settings applet, calculator, etc.) - Sleep/Wake times with empty system(and with many programs launched ); If at all supported sleep/wake (as for me - my laptop can be slept, but deny to wake properly) - Time between you press KDE start menu icon and menu appeared; - Your variant?... This would be more careful benchmarking, not only number-crunching and heavy-archiving is used by all peoples. And this benchmarking can be at least be applied for users; They can imagine how it is - to have dolphin (KDE file manager) launched in 1.03 seconds, and alt-tabbing gives new window in 0.2 seconds, when video is playing. But what about time of calculating of Super-PI? Or archiving 4Gb file? It's mostly abstract measurement, and almost useless; I repeat - for average desktop users. I've at work PC-BSD installed on 24Gb SSD, with default ZFS setup slightly tuned (disabled prefetch), and I can say that system is great, and not sluggish. I sometimes happen to fill FS to 100%, then delete logs and continue working, without any signs of ZFS problems (I read somewhere that ZFS don't like to work when not much of free space available). KDE is old, but pretty fast. How can I measure this all with some few numbers :) ? My point is, that if not now, then in some near future benchmarking need to be more practical and applicable for users. Desktop measurements and server measurements. I hope Phoronix test suite will support desktop-experience benchmarking soon :) Thanks. -- Regards, Alexander Yerenkow ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server
On 22.12.11 11:02, Johan Hendriks wrote: Stefan Esser schreef: Am 21.12.2011 22:49, schrieb Johan Hendriks: If you tune up FreeBSD to use the GCC 4.7 compiler, or downgrade linux to 4.2.1, then that will tell me nothing about FreeBSD vs Linux. The gcc version distributed with FreeBSD was chosen for license reasons, not for technical reasons. If you are OK with installing a GPLv3 licensed compiler on your systems, then just do it and take advantage of the improved code generated by it. It does not matter what the decission is to use the old compiler, it is a fact that the base comes with 4.2.x Does that mean we can not compare/benchmark against other distributions because they use GPLv3 stuff. If you intend to compare operating systems 'as shipped', then forget about comparing third party programs on top of these operating systems. Compare only any software that is already available with the operating system, as shipped. There is plenty of software in the base system and this software is specifically different (say) in FreeBSD and Linux. Anyone made such comparison? No, i want to know where standard released FreeBSD stands against standard released Linux distributions. If you compare benchmark userland applictions, then it is fair to use the latest compiler for the userland software also on FreeBSD. But what if the ports tree defaults to LLVM, then again we want to know what FreeBSD does against a Linux distribution. Why because that is what most of us will be using...!! It is pretty easy to tell the ports system to use latest gcc, as described here http://www.freebsd.org/doc/en/articles/custom-gcc/article.html If we start to compile all the ports with gcc 4.7 to be on par in comparising and benchmarking, why spend all the time getting LLVM as the default compiler for ports also? Because, FreeBSD attempts to be as GPL free as possible. GPL is incompatible with many of the applications of FreeBSD. Why not take that effort into making the WHOLE ports tree to compile with GCC4.7? This has already been done. See above. Reason, because FreeBSD goes the LLVM route. That is a decission FreeBSD is making! LLVM, as well as GCC are just choices in FreeBSD. What FreeBSD is making is, to make it safe to use LLVM to compile everything on FreeBSD, including the kernel. As you may have already noticed, some ports require to be compiled with a specific version of gcc, or a very specific compiler -- there is nothing wrong with this -- this is external software after all. And that choise will be the FreeBSD that is used in comparising and benchmarks on the net , not the utterly overcopiled and tuned FreeBSD against stock Ubuntu or whatever Linux distribution. Earlier on this thread I mentioned, that FreeBSD and Linux philosophies differ. While Linux (well, some distributions, to be correct) will try to optimize certain parts of the OS/applications in order to do well in benchmarks -- FreeBSD takes a different approach. The FreeBSD (and BSD UNIX, in general) approach is do the right thing. This may produce the results slower, but the environment is more stable in general and in the long run, the results noticeable better. This argument, by the way reminds me of the ATT vs BSDI lawsuit, where at the time UCB was involved there was lengthly discussion (at court), about sloppy programming, but we had to have something for the deadline (ATT) vs well, we have designed the architecture we think is appropriate, there might be few unimplemented things, but we are working on it. Same goes for the scheduler! and all the other subsystems FreeBSD has choosen, that makes FreeBSD. What about the scheduler? Yes you can use a more advanced compiler on FreeBSD, BUT you can do that on Linux also ,so where do you stop? Can you compile the entire Linux system, kernel and userland and external packages with LLVM? Are you going to spend a month to compare a fullly tuned up FreeBSD system against a Linux distribution? I would do that, if: - I have a task for which I need tuned system (that is, hardware would be at limits); - The application is available on both; - There is evidence or suggestion that the application under Linux will perform much better; No because the users will not spend months tuning and recompile there servers. They use the FreeBSD version that comes with the CD! On servers? :-) And that we want to compare/benchmark against a Linux distribution. No, we don't. We run our servers, we don't want to compare/benchmark them with Linux for no reason. We want however to identify design or implementation weaknesses in FreeBSD and fix these. This rarely happens by comparing to Linux distributions. There are better things to be observed in say, Solaris. If we were selling FreeBSD, we would be interested in publishing benchmarks that demonstrate how superior to Linux it is. We would tune FreeBSD to beat Linux in most
Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server
On 22 December 2011 05:54, Daniel Kalchev dan...@digsys.bg wrote: On 22.12.11 00:33, Igor Mozolevsky wrote: Using the same argument one can say that Ferrari F430 vs Toyota Prius is a meaningless comparison because the under-the-hood equipment is different. Of course, it is meaningless, the Ferrari will lose big time in the fuel consumption comparison! I believe it will also lose the price comparison as well. Not to speak the availability comparison. That's an oxymoron, right? The comparison cannot be meaningless---the reality is F430 will indeed use up more fuel than Prius. If a benchmark demonstrates a true reality, how can that benchmark be possibly meaningless??? Same benchmark might be irrelevant to someone who wants to know how fast they can get from A to B, but irrelevant is not a synonym for meaningless! You say that comparison is meaningless, yet you intend to compare those two cars? I didn't say that at all, I was demonstrating fallacy of the argument that the comparisons were meaningless. Any 'benchmark' has a goal. You first define the goal and then measure how different contenders achieve it. Reaching the goal may have several measurable metrics, that you will use to later declare the winner in each. Besides, you need to define a baseline and be aware of what theoretical max/min values are possible. Treating a benchmark as a binary win/lose is rather naive, it's not a competition, and (I hope) no serious person ever does that. A proper benchmark shows true strength and weaknesses so than a well-informed intelligent decision can be taken by an individual according to that individual's needs. The caveat, of course, is making your methodology clear and methods repeatable! Cheers, -- Igor M. ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server
On 22.12.11 11:56, Igor Mozolevsky wrote: On 22 December 2011 05:54, Daniel Kalchevdan...@digsys.bg wrote: Of course, it is meaningless, the Ferrari will lose big time in the fuel consumption comparison! I believe it will also lose the price comparison as well. Not to speak the availability comparison. That's an oxymoron, right? The comparison cannot be meaningless---the reality is F430 will indeed use up more fuel than Prius. If a benchmark demonstrates a true reality, how can that benchmark be possibly meaningless??? Same benchmark might be irrelevant to someone who wants to know how fast they can get from A to B, but irrelevant is not a synonym for meaningless! That benchmark is especially meaningless and a waste of time, because by design the Prius is constructed to consume 'less' fuel at the cost of lower engine power and the Ferrari is designed to waste fuel for the sake of high engine power. Of course, you can compare them, but this is not exactly benchmark. As for how fast to get from point A to point B. If you observe speed limits, that will depend only on the pilot, no? :) Both cars are sufficiently faster than the imposed speed limits. The same can be said for the FreeBSD and the Linux platforms. Today. Years ago, Linux was much worse, but they.. hm.. learned. :) On commodity hardware, you can expect about the same results from both OS. There will be differences due to drivers, different optimizations etc. On very specific hardware, such as systems with many CPUs and lots of memory, you may see one better than the other -- this in most cases will be relevant to tuning, but also to overall system architecture. Any 'benchmark' has a goal. You first define the goal and then measure how different contenders achieve it. Reaching the goal may have several measurable metrics, that you will use to later declare the winner in each. Besides, you need to define a baseline and be aware of what theoretical max/min values are possible. Treating a benchmark as a binary win/lose is rather naive, it's not a competition, and (I hope) no serious person ever does that. A proper benchmark shows true strength and weaknesses so than a well-informed intelligent decision can be taken by an individual according to that individual's needs. The caveat, of course, is making your methodology clear and methods repeatable! Err... a benchmark produces metrics. It does not produce conclusions. Or at least, should not :) It takes context and understanding of both the subject and methodology used to draw any conclusion out of particular benchmark. No benchmark shows strengths and weaknesses -- these are subject of interpretation and any 'score' you have in a benchmark might be the result of poor benchmark design and/or implementation. You may make an very scientific, well documented and repeatable benchmark, such as this one: time dd if=/dev/zero of=/dev/null .. then optimize your particular OS to run it at the highest possible rate... and so what? Do you know what this benchmark measures? :) Daniel ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server
On 22 December 2011 10:12, Daniel Kalchev dan...@digsys.bg wrote: As for how fast to get from point A to point B. If you observe speed limits, that will depend only on the pilot, no? :) Both cars are sufficiently faster than the imposed speed limits. You are ignoring acceleration, handling, and other factors... Besides, you're missing the point: *given same conditions* a benchmark allows one to show how A performs compared to B, which is why I said it is important to keep everything else constant! At the end of the day, what users, sysadmins, c want to know is given hardware configuration H and requirement R will software X outperform software Y or Z. The components and the bells and whistles of X, Y or Z are, quite often, irrelevant (unless one has some silly idealogical reason, for example). On very specific hardware, such as systems with many CPUs and lots of memory, you may see one better than the other -- this in most cases will be relevant to tuning, but also to overall system architecture. Are you saying that careful tuning will give you _orders of magnitude_ performance increase? Got numbers to back that up? ;-) You may make an very scientific, well documented and repeatable benchmark, such as this one: time dd if=/dev/zero of=/dev/null .. then optimize your particular OS to run it at the highest possible rate... and so what? Do you know what this benchmark measures? :) Yes, do you? I hope you are not being deliberately obtuse here... Besides, I would criticise your test in this example: have you tried running that with, say, bs=1g count=1000? Is there a difference how fast FreeBSD completes that vs how fast a Linux box does the same? The point of documenting a repeatable benchmark is to enable the person interpreting the results to see what was done (and verify) to achieve the result and treat that result accordingly. Cheers, -- Igor ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server
On 22.12.11 12:50, Igor Mozolevsky wrote: On 22 December 2011 10:12, Daniel Kalchevdan...@digsys.bg wrote: As for how fast to get from point A to point B. If you observe speed limits, that will depend only on the pilot, no? :) Both cars are sufficiently faster than the imposed speed limits. You are ignoring acceleration, handling, and other factors... Besides, you're missing the point: *given same conditions* a benchmark allows one to show how A performs compared to B, which is why I said it is important to keep everything else constant! At the end of the day, what users, sysadmins,c want to know is given hardware configuration H and requirement R will software X outperform software Y or Z. The components and the bells and whistles of X, Y or Z are, quite often, irrelevant (unless one has some silly idealogical reason, for example). None of the benchmarks measure 'comfort'. None of the benchmarks measure how the system 'feels' while performing an numerical computation. The benchmarks measure how soon the computations are finished. You typically achieve that by tuning the OS to say, ignore any interactivity at the cost of focusing all resources to compute-intensive tasks. If you use the same hardware, the CPU can do only so much and if there are any differences, that will be in how the OS asks the CPU to do other things, besides the task you benchmark. You need to define your criteria. Otherwise the benchmark cannot be used to make comparisons. On very specific hardware, such as systems with many CPUs and lots of memory, you may see one better than the other -- this in most cases will be relevant to tuning, but also to overall system architecture. Are you saying that careful tuning will give you _orders of magnitude_ performance increase? Got numbers to back that up? ;-) Ah.. now we are talking :) Two things: Someone once said, that you may have an very fast computation if only you need not make sure the results are correct. So yes, you can! :) It is all too easy to make things worse, from the theoretical baseline. So often we measure not how 'good' an OS is, but how 'bad' it actually manages the hardware. Well.. there is also some hardware that has limitations and you need to define the benchmark in a specific way to not touch them. Or you may have specific OS trying to avoid touching them -- and thus providing you with 'performance'. You may make an very scientific, well documented and repeatable benchmark, such as this one: time dd if=/dev/zero of=/dev/null .. then optimize your particular OS to run it at the highest possible rate... and so what? Do you know what this benchmark measures? :) Yes, do you? I hope you are not being deliberately obtuse here... I know, that different people will see different things being measured here. Let's see if someone else will jump in. (which is the purpose of this example) Besides, I would criticise your test in this example: have you tried running that with, say, bs=1g count=1000? That would measure different things. :) Is there a difference how fast FreeBSD completes that vs how fast a Linux box does the same? Why not? I would expect there will be difference in how fast different versions of FreeBSD complete it as well. It could be also interesting to measure (although it's somewhat subjective) how interactive both systems stay during this task. Daniel ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server
On 12/21/11 19:41, Alexander Leidinger wrote: Hi, while the discussion continued here, some work started at some other place. Now... in case someone here is willing to help instead of talking, feel free to go to http://wiki.freebsd.org/BenchmarkAdvice and have a look what can be improved. The page is far from perfect and needs some additional people which are willing to improve it. This is only part of the problem. A tuning page in the wiki - which could be referenced from the benchmark page - would be great too. Any volunteers? A first step would be to take he tuning-man-page and wikify it. Other tuning sources are welcome too. Every FreeBSD dev with a wiki account can hand out write access to the wiki. The benchmark page gives contributor-access. If someone wants write access create a FirstnameLastname account and ask here for contributor-access. Don't worry if you think your english is not good enough, even some one-word notes can help (and _my_ english got already corrected by other people on the benchmark page). Bye, Alexander. Nice to see movement ;-) But there seems something unclear: man make.conf(5) says, that MALLOC_PRODUCTION is a knob set in /etc/make.conf. The WiJi says, MALLOC_PRODUCTION is to be set in /etc/src.conf. What's right and what's wrong now? Oliver signature.asc Description: OpenPGP digital signature
Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server
On Fri, Dec 23, 2011 at 12:44:14AM +0100, O. Hartmann wrote: On 12/21/11 19:41, Alexander Leidinger wrote: Hi, while the discussion continued here, some work started at some other place. Now... in case someone here is willing to help instead of talking, feel free to go to http://wiki.freebsd.org/BenchmarkAdvice and have a look what can be improved. The page is far from perfect and needs some additional people which are willing to improve it. This is only part of the problem. A tuning page in the wiki - which could be referenced from the benchmark page - would be great too. Any volunteers? A first step would be to take he tuning-man-page and wikify it. Other tuning sources are welcome too. Every FreeBSD dev with a wiki account can hand out write access to the wiki. The benchmark page gives contributor-access. If someone wants write access create a FirstnameLastname account and ask here for contributor-access. Don't worry if you think your english is not good enough, even some one-word notes can help (and _my_ english got already corrected by other people on the benchmark page). Bye, Alexander. Nice to see movement ;-) But there seems something unclear: man make.conf(5) says, that MALLOC_PRODUCTION is a knob set in /etc/make.conf. The WiJi says, MALLOC_PRODUCTION is to be set in /etc/src.conf. What's right and what's wrong now? I can say with certainty that this value belongs in /etc/make.conf (on RELENG_8 and earlier at least). src/share/mk/bsd.own.mk has no framework for MK_MALLOC_PRODUCTION, so, this is definitely a make.conf variable. -- | Jeremy Chadwickjdc at parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, US | | Making life hard for others since 1977. PGP 4BD6C0CB | ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server
On 12/22/11 10:02, Johan Hendriks wrote: Stefan Esser schreef: Am 21.12.2011 22:49, schrieb Johan Hendriks: Nice page, but one thing i do not get is the following. [quote] If you compare FreeBSD / GCC 4.2.1 against, for example, Ubuntu / GCC 4.7 then the results are unlikely to tell you anything meaningful about FreeBSD vs Ubuntu. [/quote] That is a little strange in my opinion. It tells me that FreeBSD falls more and more behind on Linux. The reason is or could be that FreeBSD cannot or will not include GCC 4.7 and that FreeBSD will not be on par with Linux anymore. To compare it with Formula1 cars. If Mercedes decide to use the engine from 2 seasons back (the engine version 4.2.1) in there 2012 car, and Ferrari uses there new Engine (version 4.7). Can we not compare them anymore because of the decission from Mercedes to use the old engine? No we just say, if you want to win a race, get the Ferrari. It is the reallity, FreeBSD uses 4.2.1 as there compiler!!! As has been pointed out by others, FreeBSD ships with gcc-4.2.1 (with some local modifications and fixes) as the system compiler. If you tune up FreeBSD to use the GCC 4.7 compiler, or downgrade linux to 4.2.1, then that will tell me nothing about FreeBSD vs Linux. The gcc version distributed with FreeBSD was chosen for license reasons, not for technical reasons. If you are OK with installing a GPLv3 licensed compiler on your systems, then just do it and take advantage of the improved code generated by it. It does not matter what the decission is to use the old compiler, it is a fact that the base comes with 4.2.x Does that mean we can not compare/benchmark against other distributions because they use GPLv3 stuff. No, i want to know where standard released FreeBSD stands against standard released Linux distributions. If you compare benchmark userland applictions, then it is fair to use the latest compiler for the userland software also on FreeBSD. But what if the ports tree defaults to LLVM, then again we want to know what FreeBSD does against a Linux distribution. Why because that is what most of us will be using...!! Who ever tried to use gcc 4.6 to compile the base system knows that it is no eays task and it isn't so easy to simply change the compiler! This is also true for a lot of ports. If it is so easy to use a more modern compiler as some of the statements made here would suggest, then I would expect a dedicated chapter in the handbook! In such a case, every systems administrator trying to make a long-term decission what operating system might be the base for the future, does not need to be an enthusiastic of either BSD or Linux to understand how to tweak - fanboys, developers or enthusiasts have already choosen, apart from any rational or reason. What matters are advantages which can be approved. Downlad the DVD, install the OS, do some adjustments regarding to some pages of the manual, choose a propper set of applicable software to benchmark, compile, benchmark the system. Phoronix did so. Well, it's hard for me to find the chapter in the handbook which describes the performance tuning of SCHED_ULE and its sysctl tweaks, someone may call me stupid and point me to the page, please ... If we start to compile all the ports with gcc 4.7 to be on par in comparising and benchmarking, why spend all the time getting LLVM as the default compiler for ports also? Why not take that effort into making the WHOLE ports tree to compile with GCC4.7? Reason, because FreeBSD goes the LLVM route. That is a decission FreeBSD is making! Yes, and it is legitime to question that and bring pro and contra for that decission. But since FreeBSD is obviously a small club of people sitting like a duck on eggs (and, by the way, not their own genuine invented eggs, more or less reingeneered eggs), those decissions get more obscure than they seem to be anyway. And that choise will be the FreeBSD that is used in comparising and benchmarks on the net , not the utterly overcopiled and tuned FreeBSD against stock Ubuntu or whatever Linux distribution. If it is a good or bad choice! That we will see in the comparising/benchmarks we will be seeing when that time comes. Same goes for the scheduler! and all the other subsystems FreeBSD has choosen, that makes FreeBSD. ... sometimes the underdog has to pick up what's left ... I my opinion, you benchmark the latest release of Linux, FreeBSD, Solaris, Windows and whatever OS you want to compare! As you probably know, Linux is just the kernel and the distributions add user space programs, including a compiler. You can easily create a FreeBSD distribution with more advanced compiler and use or even sell it. But the FreeBSD project was cautious to not heavily depend on a GPLv3 compiler (for reasons openly discussed at the time this decision was made). I know Linux is a kernel, re read Linux as Linux Distribution! Yes you can use a more advanced
Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server
On 12/22/11 10:56, Igor Mozolevsky wrote: On 22 December 2011 05:54, Daniel Kalchev dan...@digsys.bg wrote: [...] Any 'benchmark' has a goal. You first define the goal and then measure how different contenders achieve it. Reaching the goal may have several measurable metrics, that you will use to later declare the winner in each. Besides, you need to define a baseline and be aware of what theoretical max/min values are possible. Treating a benchmark as a binary win/lose is rather naive, it's not a competition, and (I hope) no serious person ever does that. A proper benchmark shows true strength and weaknesses so than a well-informed intelligent decision can be taken by an individual according to that individual's needs. The caveat, of course, is making your methodology clear and methods repeatable! Cheers, -- Benchmarks also could lead developers to look into more details of the weak points of their OS, if they're open for that. Therefore, benchmarks are very useful. But not if any real fault of the OS is excused by a faulty becnhmarking. I remember that the worse threaded I/O performance of FreeBSD has been long discussed as a bad benchmarks schematics. Or even look at the thread regarding to SCHED_ULE. Why has a user, experiencing really worse performance with SCHED_ULE, in a nearly scientific manner some engineer the fault? I'd expect the developer or care-taking engineer taking care in a more user friendly manner. If a benchmark reveals some severe weak points in FreeBSD and I have to read about obscure tweaks of non documented sysctl, then this OS would be a no-go if I was a manager to make decissions. And yes, i know, FreeBSD is an free and open project. But I also know that this free and open project does not rely only on volonteers. A volunteers do not expect funding or payment. So, even freeBSD is dependend on some finacial basis and such a basis has to be taken care of. signature.asc Description: OpenPGP digital signature
Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server
On Dec 22, 2011, at 3:58 PM, Jeremy Chadwick free...@jdc.parodius.com wrote: On Fri, Dec 23, 2011 at 12:44:14AM +0100, O. Hartmann wrote: On 12/21/11 19:41, Alexander Leidinger wrote: Hi, while the discussion continued here, some work started at some other place. Now... in case someone here is willing to help instead of talking, feel free to go to http://wiki.freebsd.org/BenchmarkAdvice and have a look what can be improved. The page is far from perfect and needs some additional people which are willing to improve it. This is only part of the problem. A tuning page in the wiki - which could be referenced from the benchmark page - would be great too. Any volunteers? A first step would be to take he tuning-man-page and wikify it. Other tuning sources are welcome too. Every FreeBSD dev with a wiki account can hand out write access to the wiki. The benchmark page gives contributor-access. If someone wants write access create a FirstnameLastname account and ask here for contributor-access. Don't worry if you think your english is not good enough, even some one-word notes can help (and _my_ english got already corrected by other people on the benchmark page). Bye, Alexander. Nice to see movement ;-) But there seems something unclear: man make.conf(5) says, that MALLOC_PRODUCTION is a knob set in /etc/make.conf. The WiJi says, MALLOC_PRODUCTION is to be set in /etc/src.conf. What's right and what's wrong now? I can say with certainty that this value belongs in /etc/make.conf (on RELENG_8 and earlier at least). src/share/mk/bsd.own.mk has no framework for MK_MALLOC_PRODUCTION, so, this is definitely a make.conf variable. Take the advice in tuning(7) with a grain of salt because a number of suggestions are really outdated. I know because I filed a PR last night after I saw how out of synch some of the defaults it claimed were with reality on 9.x+. And I know other suggestions in the manpage are dated as well ;/. Thanks, -Garrett___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server
Am Fri, 23 Dec 2011 02:17:00 +0100 schrieb O. Hartmann ohart...@zedat.fu-berlin.de: Benchmarks also could lead developers to look into more details of the weak points of their OS, if they're open for that. Therefore, benchmarks are very useful. But not if any real fault of the OS is excused by a faulty becnhmarking. Hi, it is important for the project to be known and I think that the benchmarks made by Phoronix help FreeBSD to gain popularity, even they look bad sometimes. Furthermore, to make a benchmark is a lot of work and the results are useful, because at the end someone will look at it and will try to improve the results. Thank you for investing your time. I remember that I've made some tests with different platforms i386 vs amd64 with simple tools like openssl speed some time ago and got some bad results for amd64 that no one cared to explain. These bad results weren't reflected on Linux that I tested later for comparison. And most people have a weird attitude to think that the tester measures wrong instead of taking a look at it. They forget that as a FreeBSD user you would rather see FreeBSD win over Linux. I've seen that Phoronix made various benchmarks about FreeBSD compared to Linux and I can tell you that _subjectively_ the benchmarks reflect what I always thought about FreeBSD. I simply _know_ that FreeBSD is worse in concurrency behavior, I know that it has I/O trouble, I know that it is mostly faster emulating 3D games than Linux runs them natively. I knew this already _before_ you published the benchmark about the 3D performance. I cannot see any evil intentions in these benchmarks. All I can see is the wrong attitude _here_. If anyone thinks that Phoronix makes bad benchmarks, they should do these benchmarks by themselves and publish the results. As long as no one tries, Phoronix stays the best reference for me and for everyone else. And don't forget, benchmarks can never be objective enough and someone will always be mad about the results. Especially, when you present them a versus battle. A further thing is that I cannot understand the people here sometimes. I would like that the -RELEASE versions of FreeBSD perform well without any further optimizations. When the distribution does not compile with the latest compiler it's simply a bug. Why should one try to penalize the other distribution and downgrade their binaries? When FreeBSD has a bad default setup, there must be a reason for that. Tell me this reason and show me that it's justified in form of some other benchmark. -- Martin signature.asc Description: PGP signature
Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server
On Tue, Dec 20, 2011 at 03:29:25PM -0800, Jeremy Chadwick wrote: This also interested me: * Linux system crashed http://leaf.dragonflybsd.org/mailarchive/kernel/2011-11/msg8.html * OpenIndiana system crashed same way as Linux system http://leaf.dragonflybsd.org/mailarchive/kernel/2011-11/msg00017.html I cannot help but wonder if the Linux and OpenIndiana installations were more stressful on the hardware -- getting more out of the system, maybe resulting in increased power/load, which in turn resulted in the systems locking up (shoddy PSU, unstable mainboard, MCH problems, etc.). My point is that Francois states these things in such a way to imply that DragonflyBSD was more stable, Same thing can be said for FreeBSD, only Linux and OpenIndiana crashed reliably if I remember correctly. when in fact I happen to wonder the opposite point -- that is to say, Linux and OpenIndiana were trying to use the hardware more-so than DragonflyBSD, thus tickled what may be a hardware-level problem. I actually ran the benchmarks on two different machines with the same hardware -- brand new Supermicro boxes with ECC memory and no cut corners. Since then, I've found I could stop the Linux crashes by disabling some options in the BIOS setup: - advanced ACPI settings (don't remember exactly which ones) - and a new WHEA one. WHEA means Windows Hardware Error Architecture. For all I know, it may have been the only culprit but I didn't have time to verify if the machines also ran fine with only this option disabled. -- Francois Tigeot ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server
21.12.2011, 04:28, O. Hartmann ohart...@zedat.fu-berlin.de: On 12/21/11 00:29, Jeremy Chadwick wrote: On Tue, Dec 20, 2011 at 11:54:23PM +0100, O. Hartmann wrote: On 12/20/11 22:45, Samuel J. Greear wrote: http://www.osnews.com/story/25334/DragonFly_BSD_MP_Performance_Significantly_Improved PostgreSQL tests, see the linked PDF for #'s on FreeBSD, DragonFly, Linux and Solaris. Steps to reproduce these benchmarks provided. Sam On Tue, Dec 20, 2011 at 1:20 PM, Igor Mozolevsky i...@hybrid-lab.co.ukwrote: Interestingly, while people seem to be (arguably rightly) focused on criticising Phoronix's benchmarking, nobody has offered an alternative benchmark; and while (again, arguably rightly) it is important to benchmark real world performance, equally, nobody has offered any numbers in relation to, for example, HTTP or SMTP, or any other real world-application torture tests done on the aforementioned two platforms... IMO, this just goes to show that doing is hard and criticising is much easier (yes, I am aware of the irony involved in making this statement, but someone has to!) Cheers, Igor M :-) ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org Thanks for those numbers. Impressive how Matthew Dillon's project jumps forward now. And it is still impressive to see that the picture is still in the right place when it comes to a comparison to Linux. Also, OpenIndiana shows an impressive performance. Preface to my long post below: The things being discussed here are benchmarks, as in how much work can you get out of Thing. This is VERY DIFFERENT from testing interactivity in a scheduler, which is more of a test that says when Thing X is executed while heavier-Thing Y is also being executed, how much interaction is lost in Thing X. The reason people notice this when using Xorg is because it's visual, in an environment where responsiveness is absolutely mandatory above all else. Nobody is going to put up with a system where during a buildworld they go to move a window or click a mouse button or type a key and find that the window doesn't move, the mouse click is lost, or the key typed has gone into the bit bucket -- or, that those things are SEVERELY delayed, to the point where interactivity is crap. I whitnessed sticky, jumpy and non-responsive-for seconds FreeBSD servers (serving homes, NFS/SAMBA and PostgreSQL database (small)). Those seconds where enough to cut a ssh line. Not funny. Network traffic droped significantly. X/Desktop makes the problem visible, indeed. But not seeing it does not mean it isn't there. This might be the reason why FreeBSD is so much behind when it comes to X? Well... Are you talking about FreeBSD being laggy with the X and other GUI staff? Well, am I so lucky to have great responsiveness and interactivity here in X with the FreeBSD? The interactiveness was one the reasons I've switched my desktop from Windows to *nix (specifically FreeBSD). I just want to make that clear to folks. This immense thread has been Regards, Vans. ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server
Hi, while the discussion continued here, some work started at some other place. Now... in case someone here is willing to help instead of talking, feel free to go to http://wiki.freebsd.org/BenchmarkAdvice and have a look what can be improved. The page is far from perfect and needs some additional people which are willing to improve it. This is only part of the problem. A tuning page in the wiki - which could be referenced from the benchmark page - would be great too. Any volunteers? A first step would be to take he tuning-man-page and wikify it. Other tuning sources are welcome too. Every FreeBSD dev with a wiki account can hand out write access to the wiki. The benchmark page gives contributor-access. If someone wants write access create a FirstnameLastname account and ask here for contributor-access. Don't worry if you think your english is not good enough, even some one-word notes can help (and _my_ english got already corrected by other people on the benchmark page). Bye, Alexander. -- Send via an Android device, please forgive brevity and typographic and spelling errors. O. Hartmann ohart...@zedat.fu-berlin.de hat geschrieben:On 12/20/11 21:20, Igor Mozolevsky wrote: Interestingly, while people seem to be (arguably rightly) focused on criticising Phoronix's benchmarking, nobody has offered an alternative benchmark; and while (again, arguably rightly) it is important to benchmark real world performance, equally, nobody has offered any numbers in relation to, for example, HTTP or SMTP, or any other real world-application torture tests done on the aforementioned two platforms... IMO, this just goes to show that doing is hard and criticising is much easier (yes, I am aware of the irony involved in making this statement, but someone has to!) Cheers, Igor M :-) Unfortunately, M. Larabel is the only one who's performing benchmarks on FreeBSD, comparing its performance to the Linux-opponents. Adn indeed, there is a lot of criticism, but no alternative. I said unfortunately - not offensive - since Larabel and Phoronix are sadly the only ones who do actually such bechmarking. It would be much more nicer and kind to support those people. Well, in January/February we get new hardware. One box is supposed to do number crunching via 12 cores and a TESLA GPU. My colleague is developing a high parallelized peice of software for satellite data transformation. The software package is CPU bound, partially GPU, but massively memory hungry (96 to 128 GB RAM is needed). What I can offer is, since I will also work on that machine and I've free hand to administer, in the spare time of doing my PhD, installing FreeBSD 9.0/10.0 besides SuSe Linux and looking forward having one ZFS data storage drive for homes, so both systems can perform on a most recent ZFS. I'm new to Linux, not a BSD guru, nor I'm a professional programmer/developer. My skills are sufficient for the daily scientific work. So, without pressure, I'm willing to perform some HPC benchmarks under advice if the day comes and those interested in bare numbers of FreeBSD vs. Linux performance with a real-world-scientific application. I would appreciate to see some of the developers and/or FreeBSD hackers to help Phoronix setting up a proper testenvironment instead of bashing M. Larabel and his fellows. Regards, Oliver ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server
Alexander Leidinger schreef: Hi, while the discussion continued here, some work started at some other place. Now... in case someone here is willing to help instead of talking, feel free to go to http://wiki.freebsd.org/BenchmarkAdvice and have a look what can be improved. The page is far from perfect and needs some additional people which are willing to improve it. This is only part of the problem. A tuning page in the wiki - which could be referenced from the benchmark page - would be great too. Any volunteers? A first step would be to take he tuning-man-page and wikify it. Other tuning sources are welcome too. Every FreeBSD dev with a wiki account can hand out write access to the wiki. The benchmark page gives contributor-access. If someone wants write access create a FirstnameLastname account and ask here for contributor-access. Don't worry if you think your english is not good enough, even some one-word notes can help (and _my_ english got already corrected by other people on the benchmark page). Bye, Alexander. ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org Nice page, but one thing i do not get is the following. [quote] If you compare FreeBSD / GCC 4.2.1 against, for example, Ubuntu / GCC 4.7 then the results are unlikely to tell you anything meaningful about FreeBSD vs Ubuntu. [/quote] That is a little strange in my opinion. It tells me that FreeBSD falls more and more behind on Linux. The reason is or could be that FreeBSD cannot or will not include GCC 4.7 and that FreeBSD will not be on par with Linux anymore. To compare it with Formula1 cars. If Mercedes decide to use the engine from 2 seasons back (the engine version 4.2.1) in there 2012 car, and Ferrari uses there new Engine (version 4.7). Can we not compare them anymore because of the decission from Mercedes to use the old engine? No we just say, if you want to win a race, get the Ferrari. It is the reallity, FreeBSD uses 4.2.1 as there compiler!!! If you tune up FreeBSD to use the GCC 4.7 compiler, or downgrade linux to 4.2.1, then that will tell me nothing about FreeBSD vs Linux. I my opinion, you benchmark the latest release of Linux, FreeBSD, Solaris, Windows and whatever OS you want to compare! You want to benchmark the release and not a tuned version against a standard version. And that in general are the versions most of us users will use. And what if in the future LLVM gets on par with Linux, is it stil fair to compare FreeBSD with Linux? Or do we say, well we are on par, but it is not fair, yes we used the latest releases, but you can not blame Linux because they are still using GCC. No what we will see then are haleluja blogs that FreeBSD is on par with Linux. For me peformance is not a show stopper, and for the most of us i think it is not. FreeBSD for me is a clean system that does the job perfect and has a very helpful community. regards, Johan ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server
On Wed, Dec 21, 2011 at 1:49 PM, Johan Hendriks joh.hendr...@gmail.com wrote: Nice page, but one thing i do not get is the following. [quote] If you compare FreeBSD / GCC 4.2.1 against, for example, Ubuntu / GCC 4.7 then the results are unlikely to tell you anything meaningful about FreeBSD vs Ubuntu. [/quote] That is a little strange in my opinion. It tells me that FreeBSD falls more and more behind on Linux. The reason is or could be that FreeBSD cannot or will not include GCC 4.7 and that FreeBSD will not be on par with Linux anymore. When benchmarking two systems, you need to make sure that everything possible is the same (constants) and that the only differences between the two systems are what you want to benchmark (variables). For example, if you want to compare the performance of GCC-compiled binaries, then you would use the same hardware host, the same OS install, the same source code, and only change the compiler versions used to compile the benchmark binaries. That way, the only variable is version of GCC, everything else is constant, and thus the benchmark is actually testing the performance of GCC. Likewise, if you want to benchmark the performance of two OSes, you need to eliminate as many variables as possible: - same hardware - running the same benchmark binaries - using the same versions of GCC - using the same filesystems - etc That gives you the starting point. Then, you modify one of the constants above, and re-run the benchmarks. Then you modify one more of the constants above, and re-run the benchmarks. Etc. Each time, you vary only 1 thing, so that you can measure the impact of that *ONE* thing. Comparing random binary compile with GCC X on FreeBSD Y on filesystem Z on hardware config A against random binary built with GCC Q on Linux R on fileystem S on hardware config B doesn't show anything. Was the performance difference due to hardware? Filesystem? OS? GCC version? Something else? You can't use a shotgun the thread a needle. :) And what if in the future LLVM gets on par with Linux, is it stil fair to compare FreeBSD with Linux? Then you add compiler suite to the list of variables, and you make it a constant in the first run, and then vary it one piece at a time in later runs, to isolate whether or not it affects performance. In order to do a proper comparison of any two things, you have to first make them as equal as possible, and then vary things one bit at a time until you are at the default configuration for each. Only then can you really, truly, empirically say why A is better/faster/more-uber than B. Unfortunately, doing it right requires a lot of time, effort, time, and more time. -- Freddie Cash fjwc...@gmail.com ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server
On 21 December 2011 22:03, Freddie Cash fjwc...@gmail.com wrote: On Wed, Dec 21, 2011 at 1:49 PM, Johan Hendriks joh.hendr...@gmail.com wrote: Nice page, but one thing i do not get is the following. [quote] If you compare FreeBSD / GCC 4.2.1 against, for example, Ubuntu / GCC 4.7 then the results are unlikely to tell you anything meaningful about FreeBSD vs Ubuntu. [/quote] That is a little strange in my opinion. It tells me that FreeBSD falls more and more behind on Linux. The reason is or could be that FreeBSD cannot or will not include GCC 4.7 and that FreeBSD will not be on par with Linux anymore. When benchmarking two systems, you need to make sure that everything possible is the same (constants) and that the only differences between the two systems are what you want to benchmark (variables). Yes and no, but to be perfectly frank, the statement, as it stands, is a bit of a nonsense. Let me illustrate in a different way. This is macro~ vs micro~comparison of systems and depends on what you are trying to get out of the benchmark. Using the same argument one can say that Ferrari F430 vs Toyota Prius is a meaningless comparison because the under-the-hood equipment is different. Now, it is absolutely correct to say that in A vs B comparisons, only one thing should be changed and the rest should remain constant. The important thing is, however, to determine the scope of your benchmark: you are not benchmarking a component of A vs a component of B, but you are benchmarking A as a whole system and B as a whole system. Thus, the thing that changes is the system itself. Going back to F430 vs Prius, you first decide what you want to benchmark (acceleration, top speed, fuel consumption, ride comfort, c) then you measure that aspect in each of the system---you are not looking at the wiring, engine, wheels, c individually but *at a whole system*. You use the same route, time of day, driver, drive pattern, weather conditions, c, the only thing that changes is the car! Similarly, FreeBSD vs Linux, you want to a) determine what metric you want to benchmark (NFS throughput, HTTP client handling, SMPT throughput, prime number computation) and b) *scientifically* measure the system against that metric... This would essentially amount to having identical set up and tests, and only changing the hard disks (one containing Linux and another one containing FreeBSD). I don't see why this is such a difficult concept to grasp. Cheers, -- Igor M. :-) ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server
On Wed, Dec 21, 2011 at 10:49 PM, Johan Hendriks joh.hendr...@gmail.com wrote: Alexander Leidinger schreef: Hi, while the discussion continued here, some work started at some other place. Now... in case someone here is willing to help instead of talking, feel free to go to http://wiki.freebsd.org/BenchmarkAdvice and have a look what can be improved. The page is far from perfect and needs some additional people which are willing to improve it. This is only part of the problem. A tuning page in the wiki - which could be referenced from the benchmark page - would be great too. Any volunteers? A first step would be to take he tuning-man-page and wikify it. Other tuning sources are welcome too. Every FreeBSD dev with a wiki account can hand out write access to the wiki. The benchmark page gives contributor-access. If someone wants write access create a FirstnameLastname account and ask here for contributor-access. Don't worry if you think your english is not good enough, even some one-word notes can help (and _my_ english got already corrected by other people on the benchmark page). Bye, Alexander. Nice page, but one thing i do not get is the following. [quote] If you compare FreeBSD / GCC 4.2.1 against, for example, Ubuntu / GCC 4.7 then the results are unlikely to tell you anything meaningful about FreeBSD vs Ubuntu. [/quote] That is a little strange in my opinion. It tells me that FreeBSD falls more and more behind on Linux. The reason is or could be that FreeBSD cannot or will not include GCC 4.7 and that FreeBSD will not be on par with Linux anymore. (...) It does, though? It's in ports. The system compiler is for the system, but if you're compiling ports or standalone software you certainly can - and sometimes must - use something else. The point of that section seems to be if you're compiling userland software to compare, at least compile it with the same compiler, unless that's what you want to benchmark. Sensible enough. As for what the kernel is compiled with, I doubt that will have the same kind of effect as what the user software is compiled with. The kernel is compiled with very conservative settings anyway, and I don't think it really does much of the kind of heavy computation that benefits the most from better compilers. The most interesting part is probably the effect on the userland libraries. Has anyone done any tests on how much of an effect on user software performance it has if you change the compiler for the libraries in the base system? (I would guess not massive, but this is one of those things where some numbers wouldn't hurt). Oh, and remember that clang also works as a system compiler, and we're definitely not stuck on an old version of that. It produces code with performance comparable to gcc today, and I doubt it'll fall horribly behind in the foreseeable future. -- Daniel Nebdal ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server
On 21.12.11 23:49, Johan Hendriks wrote: I my opinion, you benchmark the latest release of Linux, FreeBSD, Solaris, Windows and whatever OS you want to compare! There is no 'general benchmark' as there is not one single tasks that all computers are used for. If you want to benchmark something, then you define that something, tune all test subjects appropriately for that one thing and run the same test load. You then go on and claim 'for task X, the OS Y was best, followed by ...'. This is what people have done for PostgreSQL for example. You may try to see how, with that same settings different OS will perform with varying conditions, like what the PostgreSQL test did -- performance over the network and performance to localhost. Testing a system, tuned for a file server as X workstations will not tell you much about the abilities of the different operating systems to run a web server, or either file server or X workstation. By the way, the gcc in 8-stable is gcc version 4.2.2 20070831 prerelease [FreeBSD] Daniel ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server
On 22.12.11 00:33, Igor Mozolevsky wrote: Using the same argument one can say that Ferrari F430 vs Toyota Prius is a meaningless comparison because the under-the-hood equipment is different. Of course, it is meaningless, the Ferrari will lose big time in the fuel consumption comparison! I believe it will also lose the price comparison as well. Not to speak the availability comparison. You say that comparison is meaningless, yet you intend to compare those two cars? Any 'benchmark' has a goal. You first define the goal and then measure how different contenders achieve it. Reaching the goal may have several measurable metrics, that you will use to later declare the winner in each. Besides, you need to define a baseline and be aware of what theoretical max/min values are possible. Daniel ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server
Am 21.12.2011 22:49, schrieb Johan Hendriks: Nice page, but one thing i do not get is the following. [quote] If you compare FreeBSD / GCC 4.2.1 against, for example, Ubuntu / GCC 4.7 then the results are unlikely to tell you anything meaningful about FreeBSD vs Ubuntu. [/quote] That is a little strange in my opinion. It tells me that FreeBSD falls more and more behind on Linux. The reason is or could be that FreeBSD cannot or will not include GCC 4.7 and that FreeBSD will not be on par with Linux anymore. To compare it with Formula1 cars. If Mercedes decide to use the engine from 2 seasons back (the engine version 4.2.1) in there 2012 car, and Ferrari uses there new Engine (version 4.7). Can we not compare them anymore because of the decission from Mercedes to use the old engine? No we just say, if you want to win a race, get the Ferrari. It is the reallity, FreeBSD uses 4.2.1 as there compiler!!! As has been pointed out by others, FreeBSD ships with gcc-4.2.1 (with some local modifications and fixes) as the system compiler. If you tune up FreeBSD to use the GCC 4.7 compiler, or downgrade linux to 4.2.1, then that will tell me nothing about FreeBSD vs Linux. The gcc version distributed with FreeBSD was chosen for license reasons, not for technical reasons. If you are OK with installing a GPLv3 licensed compiler on your systems, then just do it and take advantage of the improved code generated by it. I my opinion, you benchmark the latest release of Linux, FreeBSD, Solaris, Windows and whatever OS you want to compare! As you probably know, Linux is just the kernel and the distributions add user space programs, including a compiler. You can easily create a FreeBSD distribution with more advanced compiler and use or even sell it. But the FreeBSD project was cautious to not heavily depend on a GPLv3 compiler (for reasons openly discussed at the time this decision was made). You want to benchmark the release and not a tuned version against a standard version. And that in general are the versions most of us users will use. If you compare operating systems from a technical point of view, then you'll be interested in relative performance of algorithms and methods chosen. This is best achieved by using the same compiler for each of the candidates. If you compare performance from a user point of view, you are correct that performance delivered out of the box (without complicated tuning) may be, what counts for most users. But those users that depend on best performance e.g. for a FreeBSD based embedded product or a data center, may tune the system, including compilation with a newer compiler than the system default. And what if in the future LLVM gets on par with Linux, is it stil fair to compare FreeBSD with Linux? You can always compare anything with whatever you like (even apples with oranges), but you need to be aware of what you compare and what your goals are, to be able to draw reasonable conclusions. If you want to test out of the box performance, then test with system compilers (or just those binaries delivered with the system). If you want to test for code efficiency or scalability, then use the same compilers for each system under test to remove differences introduced by the compilers (which are an external component not developed by the FreeBSD people). Or do we say, well we are on par, but it is not fair, yes we used the latest releases, but you can not blame Linux because they are still using GCC. Depends on what you want or need to measure ... No what we will see then are haleluja blogs that FreeBSD is on par with Linux. Such blog messages are not common in the FreeBSD community. FreeBSD used to have big technical and performance advantages when Linux was young, but even then, there was technical discussion between camps (and many concepts were implemented in Linux based on BSD examples; I have taken part in such discussions myself, some 15 to 20 years back). For me peformance is not a show stopper, and for the most of us i think it is not. FreeBSD for me is a clean system that does the job perfect and has a very helpful community. Well, this are valid aspects, too, and very hard to with benchmarks ;-) Regards, STefan ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server
On Mon, Dec 19, 2011 at 2:16 PM, Alexander Yerenkow yeren...@gmail.com wrote: FreeBSD currently have very obscure, closed community. To get in touch, you need to subscribe to several mail lists, constantly read them, I've just found recently (my shame of course) in mail list that there is service ( pub.allbsd.org) which constantly building current versions. This is great, but at homepage of freebsd.org there is no word about it :) That's because it's not official. Do you take the risk? Would a multi-milion-dollar company do that? For your private server, sure it's probably fine. But how do you know that those files are not contaminated? (That being said, the purpose of that service is good. And the files there a most probably 100% fine. But if it's not official... then..) -- chs, if there is only one candiate, there is one one choice! ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server
On Dec 20, 2011, at 1:01 AM, Christer Solskogen wrote: On Mon, Dec 19, 2011 at 2:16 PM, Alexander Yerenkow yeren...@gmail.com wrote: FreeBSD currently have very obscure, closed community. To get in touch, you need to subscribe to several mail lists, constantly read them, I've just found recently (my shame of course) in mail list that there is service ( pub.allbsd.org) which constantly building current versions. This is great, but at homepage of freebsd.org there is no word about it :) That's because it's not official. Do you take the risk? Would a multi-milion-dollar company do that? For your private server, sure it's probably fine. But how do you know that those files are not contaminated? (That being said, the purpose of that service is good. And the files there a most probably 100% fine. But if it's not official... then..) As long as I have reliable checksums that match the what the upstream source says is the real thing, it doesn't practically matter where I get my images from. -Garrett___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server
On Tue, Dec 20, 2011 at 10:42 AM, Garrett Cooper yaneg...@gmail.com wrote: As long as I have reliable checksums that match the what the upstream source says is the real thing, it doesn't practically matter where I get my images from. Checksums compared to what? How would you know what the correct checksums for OpenBSD-current is, if it's not built by Theo? -- chs, ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server
On Dec 20, 2011, at 1:51 AM, Christer Solskogen wrote: On Tue, Dec 20, 2011 at 10:42 AM, Garrett Cooper yaneg...@gmail.com wrote: As long as I have reliable checksums that match the what the upstream source says is the real thing, it doesn't practically matter where I get my images from. Checksums compared to what? How would you know what the correct checksums for OpenBSD-current is, if it's not built by Theo? Release engineering for FreeBSD produces SHA256 checksums for all official releases. AFAIK though they're only in the announcement emails and not stored anywhere else. I can't speak for OpenBSD's release process. Thanks, -Garrett___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server
On 20.12.11 11:42, Garrett Cooper wrote: As long as I have reliable checksums that match the what the upstream source says is the real thing, it doesn't practically matter where I get my images from. Relying on checksums that are published on the same web site where you download the files from and given that most of these sites do not even use SSL so much about 'security'. Daniel ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server
Guys, I have a question about these benchmarks. Why worry about that if the CURRENT comes with debug enabled by default? http://joaobarros.blogspot.com/2005/07/freebsd-how-to-turn-off-debug-options.html On 19/12/2011, at 22:28, Petro Rossini wrote: Hi all, just a thought here: On Tue, Dec 20, 2011 at 12:45 AM, Daniel Kalchev dan...@digsys.bg wrote: As were told, Phoronix used default setup, not tuned. Not really. They created some weird test environment, at least for FreeBSD -- who knows, possibly for Linux as well. For example, ZFS is by no means a default file system in FreeBSD. You need to go trough manual steps, to enable it, to build the pool, filesystems etc. .. Of course the benchmark setup and procedure is strange but.. it could be improved, I think. Have a good collection of tuning parameters for popular cases, advertised properly so it gets hard to miss them. I am a sysadmin and, over the years, I had to run file servers, database servers, web servers, tomcats... Well, most of the time I set it up and it just works because the system in question is not maxed out, not even close to it. But if I want to squeeze the last 20% out of it googling starts, and here and there I find hints how to tune the OS, the file system, what scheduler to use etc. It would be great to have a set of case studies at hand, e.g. under the /usr/share/examples directory, that describes tweaks to have a performing postgresql server, or mysql, or apache or a desktop or.. Things I find, for example, in the BSD Magazine. Maybe benchmarks become more meaningful then.. A general remark for people doing benchmarks for comparison: you need a well-informed system engineer for the systems you compare. So, if you compare a Linux system with FreeBSD, have two experienced admins that know their OS well. Regards Peter ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server
On 20/12/2011 10:39, Daniel Kalchev wrote: On 20.12.11 11:42, Garrett Cooper wrote: As long as I have reliable checksums that match the what the upstream source says is the real thing, it doesn't practically matter where I get my images from. Relying on checksums that are published on the same web site where you download the files from and given that most of these sites do not even use SSL so much about 'security'. This does remind me of one issue that while a little off topic for this thread If i wanted to get, for example the SHA265 checksums from a verified source, how would i verify this currently? There doesnt seem to be an SSL site for www.freebsd.org and its not too hard to redirect someone to a fake website. What would be a more reasonable list to request this on? Vince Daniel ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server
On Tue, Dec 20, 2011 at 5:46 AM, Vincent Hoffman vi...@unsane.co.uk wrote: On 20/12/2011 10:39, Daniel Kalchev wrote: On 20.12.11 11:42, Garrett Cooper wrote: As long as I have reliable checksums that match the what the upstream source says is the real thing, it doesn't practically matter where I get my images from. Relying on checksums that are published on the same web site where you download the files from and given that most of these sites do not even use SSL so much about 'security'. This does remind me of one issue that while a little off topic for this thread If i wanted to get, for example the SHA265 checksums from a verified source, how would i verify this currently? There doesnt seem to be an SSL site for www.freebsd.org and its not too hard to redirect someone to a fake website. What would be a more reasonable list to request this on? And so the masses go off on a quest to answer how to obtain releases instead of staying focused on the original problem at hand.. -Garrett ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server
On Tue, Dec 20, 2011 at 4:28 AM, Chiron IO io.chi...@gmail.com wrote: Guys, I have a question about these benchmarks. Why worry about that if the CURRENT comes with debug enabled by default? http://joaobarros.blogspot.com/2005/07/freebsd-how-to-turn-off-debug-options.html In the real world problems happen and someone has to be able to, in some fashion, identify and resolve those problems. As such, shipping FreeBSD releases with INVARIANTS disabled is a mistake and any benchmarks done without INVARIANTS enabled will fail to reflect most reasonable real world use-cases. Although these benchmarks cannot stand on their own merits for many reasons, I do not see how any benchmark is automatically invalidated by using the default development configuration. Sam ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server
http://wiki.freebsd.org/DefaultDebuggingKnobs I am not aware of any linux distribution that comes with debug enabled by default, even on RC releases. It seems that this approach (debug by default) is welcome to help solve problems that might appear, but I would be happy if these benchmarks were made without such config (debug) enabled. On 20/12/2011, at 17:09, Samuel J. Greear wrote: On Tue, Dec 20, 2011 at 4:28 AM, Chiron IO io.chi...@gmail.com wrote: Guys, I have a question about these benchmarks. Why worry about that if the CURRENT comes with debug enabled by default? http://joaobarros.blogspot.com/2005/07/freebsd-how-to-turn-off-debug-options.html In the real world problems happen and someone has to be able to, in some fashion, identify and resolve those problems. As such, shipping FreeBSD releases with INVARIANTS disabled is a mistake and any benchmarks done without INVARIANTS enabled will fail to reflect most reasonable real world use-cases. Although these benchmarks cannot stand on their own merits for many reasons, I do not see how any benchmark is automatically invalidated by using the default development configuration. Sam ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server
Interestingly, while people seem to be (arguably rightly) focused on criticising Phoronix's benchmarking, nobody has offered an alternative benchmark; and while (again, arguably rightly) it is important to benchmark real world performance, equally, nobody has offered any numbers in relation to, for example, HTTP or SMTP, or any other real world-application torture tests done on the aforementioned two platforms... IMO, this just goes to show that doing is hard and criticising is much easier (yes, I am aware of the irony involved in making this statement, but someone has to!) Cheers, Igor M :-) ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server
http://www.osnews.com/story/25334/DragonFly_BSD_MP_Performance_Significantly_Improved PostgreSQL tests, see the linked PDF for #'s on FreeBSD, DragonFly, Linux and Solaris. Steps to reproduce these benchmarks provided. Sam On Tue, Dec 20, 2011 at 1:20 PM, Igor Mozolevsky i...@hybrid-lab.co.ukwrote: Interestingly, while people seem to be (arguably rightly) focused on criticising Phoronix's benchmarking, nobody has offered an alternative benchmark; and while (again, arguably rightly) it is important to benchmark real world performance, equally, nobody has offered any numbers in relation to, for example, HTTP or SMTP, or any other real world-application torture tests done on the aforementioned two platforms... IMO, this just goes to show that doing is hard and criticising is much easier (yes, I am aware of the irony involved in making this statement, but someone has to!) Cheers, Igor M :-) ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server
On 12/20/11 22:45, Samuel J. Greear wrote: http://www.osnews.com/story/25334/DragonFly_BSD_MP_Performance_Significantly_Improved PostgreSQL tests, see the linked PDF for #'s on FreeBSD, DragonFly, Linux and Solaris. Steps to reproduce these benchmarks provided. Sam On Tue, Dec 20, 2011 at 1:20 PM, Igor Mozolevsky i...@hybrid-lab.co.ukwrote: Interestingly, while people seem to be (arguably rightly) focused on criticising Phoronix's benchmarking, nobody has offered an alternative benchmark; and while (again, arguably rightly) it is important to benchmark real world performance, equally, nobody has offered any numbers in relation to, for example, HTTP or SMTP, or any other real world-application torture tests done on the aforementioned two platforms... IMO, this just goes to show that doing is hard and criticising is much easier (yes, I am aware of the irony involved in making this statement, but someone has to!) Cheers, Igor M :-) ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org Thanks for those numbers. Impressive how Matthew Dillon's project jumps forward now. And it is still impressive to see that the picture is still in the right place when it comes to a comparison to Linux. Also, OpenIndiana shows an impressive performance. But this is only one suite of testing. Scientific Linux is supposed to give the best performance for scientifi purposes, i.e. for longhaul calculations, much numerical stuff. It outperforms in a typical server application FreeBSd, were FreeBSD shoulkd have the power to serve. Is the postgresql benchmark the only way to benchmark? Well, this inspires me to gather together all the benchmarks someone could find. There were lots of compalins about FreeBSD's poor performance with BIND - once a domain of FreeBSD. Network performance seems also to be an issue if it comes to scalability. It would be nice to see what portion of the raw CPU/GPU power the OS (FreeBSD, Linux ...) delivers to scientific applications. I only know some kind of benchmarks, BYTE UNIX benchmark, LINPACK test ... Does someone know a site to look for a couple of benchmarks to test a) memory system b) scalability (apart from pgbench) c) network performance/throughput/network scalability d) portion of CPU performance the system delivers for numerical applications to the user apart from the system's own consumption e) disk I/O performance and scalability it would also be nice to discuss some nice settings and performance tunings for FreeBSD for several scenarios. I guess, starting developing benchmarking test scenarios for several purposes would lead faster to real numbers and non polemic than weird discussions ... signature.asc Description: OpenPGP digital signature
Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server
On Tue, Dec 20, 2011 at 11:54:23PM +0100, O. Hartmann wrote: On 12/20/11 22:45, Samuel J. Greear wrote: http://www.osnews.com/story/25334/DragonFly_BSD_MP_Performance_Significantly_Improved PostgreSQL tests, see the linked PDF for #'s on FreeBSD, DragonFly, Linux and Solaris. Steps to reproduce these benchmarks provided. Sam On Tue, Dec 20, 2011 at 1:20 PM, Igor Mozolevsky i...@hybrid-lab.co.ukwrote: Interestingly, while people seem to be (arguably rightly) focused on criticising Phoronix's benchmarking, nobody has offered an alternative benchmark; and while (again, arguably rightly) it is important to benchmark real world performance, equally, nobody has offered any numbers in relation to, for example, HTTP or SMTP, or any other real world-application torture tests done on the aforementioned two platforms... IMO, this just goes to show that doing is hard and criticising is much easier (yes, I am aware of the irony involved in making this statement, but someone has to!) Cheers, Igor M :-) ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org Thanks for those numbers. Impressive how Matthew Dillon's project jumps forward now. And it is still impressive to see that the picture is still in the right place when it comes to a comparison to Linux. Also, OpenIndiana shows an impressive performance. Preface to my long post below: The things being discussed here are benchmarks, as in how much work can you get out of Thing. This is VERY DIFFERENT from testing interactivity in a scheduler, which is more of a test that says when Thing X is executed while heavier-Thing Y is also being executed, how much interaction is lost in Thing X. The reason people notice this when using Xorg is because it's visual, in an environment where responsiveness is absolutely mandatory above all else. Nobody is going to put up with a system where during a buildworld they go to move a window or click a mouse button or type a key and find that the window doesn't move, the mouse click is lost, or the key typed has gone into the bit bucket -- or, that those things are SEVERELY delayed, to the point where interactivity is crap. I just want to make that clear to folks. This immense thread has been with regards to the latter -- bad interactivity/responsiveness on a system which was undergoing load that SHOULD be distributed more evenly across the system *while* keeping interactivity/responsiveness high. Historically nice/renice has been used for this task, but that was when kernels were a little less complex and I/O subsystems were less complex. Remember: we've now got schedulers for each type of thing, and who gets what priority? You get my point I'm sure. So remember: this was to discuss that aspect, with regards to ULE vs. 4BSD schedulers. Now, back to the benchmarks: This also interested me: * Linux system crashed http://leaf.dragonflybsd.org/mailarchive/kernel/2011-11/msg8.html * OpenIndiana system crashed same way as Linux system http://leaf.dragonflybsd.org/mailarchive/kernel/2011-11/msg00017.html I cannot help but wonder if the Linux and OpenIndiana installations were more stressful on the hardware -- getting more out of the system, maybe resulting in increased power/load, which in turn resulted in the systems locking up (shoddy PSU, unstable mainboard, MCH problems, etc.). My point is that Francois states these things in such a way to imply that DragonflyBSD was more stable, when in fact I happen to wonder the opposite point -- that is to say, Linux and OpenIndiana were trying to use the hardware more-so than DragonflyBSD, thus tickled what may be a hardware-level problem. But this is only one suite of testing. Scientific Linux is supposed to give the best performance for scientifi purposes, i.e. for longhaul calculations, much numerical stuff. It outperforms in a typical server application FreeBSd, were FreeBSD shoulkd have the power to serve. Is the postgresql benchmark the only way to benchmark? I sure hope not. But you know what's equally as interesting? This: http://people.freebsd.org/~kris/scaling/ Specifically circa 2008: http://people.freebsd.org/~kris/scaling/4cpu-pgsql.png http://people.freebsd.org/~kris/scaling/pgsql-16cpu-2.png http://people.freebsd.org/~kris/scaling/pgsql-16cpu.png Now, I don't know if what was used in those (pgsql sysbench) was the same thing as pg_bench in the DragonflyBSD tests, but if so, the numbers are different to a point that is preposterous. There's also this: http://people.freebsd.org/~kris/scaling/pgsql-ncpu.png Now, compare those numbers to the TPS numbers shown here: http://dl.wolfpond.org/Pg-benchmarks.pdf So um... yeah. Now, if someone here is going to say well, what was tested by Kris was FreeBSD 7.0, while what was
Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server
Bottom post this time to follow Oliver :). On 12/20/2011 02:54 PM, O. Hartmann wrote: On 12/20/11 22:45, Samuel J. Greear wrote: http://www.osnews.com/story/25334/DragonFly_BSD_MP_Performance_Significantly_Improved PostgreSQL tests, see the linked PDF for #'s on FreeBSD, DragonFly, Linux and Solaris. Steps to reproduce these benchmarks provided. Sam There are still possible issues with those benchmarks. The Xeon has known problems scaling from 6 to 12 cores (well enabling the hyperthreading), so you may find that some platforms are penalized in performance if HT is turned on. See the scaling that Phoronix has done in http://openbenchmarking.org/result/1112166-AR-1112153AR03 Most systems are good with scaling on real cores, the hyperthreading (and for that matter the Bulldozer thread affinity) can really break performance. Different platforms have different behaviours. Benchmarking is a mucky business.. Note that the benchmarks with Phoronix test suite are repeatable, once installed, you can just run ./phoronix-test-suite benchmark 1112113-AR-ORACLELIN37 to repeat (as close as the system allows) the benchmarks that started this thread. Is the postgresql benchmark the only way to benchmark? pgbench is already included in the Phoronix Test Suite (at least 9.0.1 TPC-B benchmark. Well, this inspires me to gather together all the benchmarks someone could find. There were lots of compalins about FreeBSD's poor performance with BIND - once a domain of FreeBSD. Network performance seems also to be an issue if it comes to scalability. It would be nice to see what portion of the raw CPU/GPU power the OS (FreeBSD, Linux ...) delivers to scientific applications. I only know some kind of benchmarks, BYTE UNIX benchmark, LINPACK test ... Does someone know a site to look for a couple of benchmarks to test a) memory system b) scalability (apart from pgbench) c) network performance/throughput/network scalability d) portion of CPU performance the system delivers for numerical applications to the user apart from the system's own consumption e) disk I/O performance and scalability The majority of these benchmarks are already in Phoronix Test Suite. There is monitoring capability (temp, load, CPU states, etc). The question is the mapping from system attribute to benchmark, as well as determine what the ambigious terms mean (scaling can mean on increasing workloads, as memory is increased, as cpus are increased). it would also be nice to discuss some nice settings and performance tunings for FreeBSD for several scenarios. I guess, starting developing benchmarking test scenarios for several purposes would lead faster to real numbers and non polemic than weird discussions ... This is what Michael and I are wanting to see. Adrian Chadd has offerered to help facilitate within the FreeBSD community. As mentioned before, what I'd like to see is 1) Recommendations for more rounded benchmarks from the FreeBSD perspective 2) Tuning guide documented somewhere within the community 3) Comparative results based on the communities testing. All concrete, and all achievable. Regards, Matthew ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server
On 12/21/11 00:29, Jeremy Chadwick wrote: On Tue, Dec 20, 2011 at 11:54:23PM +0100, O. Hartmann wrote: On 12/20/11 22:45, Samuel J. Greear wrote: http://www.osnews.com/story/25334/DragonFly_BSD_MP_Performance_Significantly_Improved PostgreSQL tests, see the linked PDF for #'s on FreeBSD, DragonFly, Linux and Solaris. Steps to reproduce these benchmarks provided. Sam On Tue, Dec 20, 2011 at 1:20 PM, Igor Mozolevsky i...@hybrid-lab.co.ukwrote: Interestingly, while people seem to be (arguably rightly) focused on criticising Phoronix's benchmarking, nobody has offered an alternative benchmark; and while (again, arguably rightly) it is important to benchmark real world performance, equally, nobody has offered any numbers in relation to, for example, HTTP or SMTP, or any other real world-application torture tests done on the aforementioned two platforms... IMO, this just goes to show that doing is hard and criticising is much easier (yes, I am aware of the irony involved in making this statement, but someone has to!) Cheers, Igor M :-) ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org Thanks for those numbers. Impressive how Matthew Dillon's project jumps forward now. And it is still impressive to see that the picture is still in the right place when it comes to a comparison to Linux. Also, OpenIndiana shows an impressive performance. Preface to my long post below: The things being discussed here are benchmarks, as in how much work can you get out of Thing. This is VERY DIFFERENT from testing interactivity in a scheduler, which is more of a test that says when Thing X is executed while heavier-Thing Y is also being executed, how much interaction is lost in Thing X. The reason people notice this when using Xorg is because it's visual, in an environment where responsiveness is absolutely mandatory above all else. Nobody is going to put up with a system where during a buildworld they go to move a window or click a mouse button or type a key and find that the window doesn't move, the mouse click is lost, or the key typed has gone into the bit bucket -- or, that those things are SEVERELY delayed, to the point where interactivity is crap. I whitnessed sticky, jumpy and non-responsive-for seconds FreeBSD servers (serving homes, NFS/SAMBA and PostgreSQL database (small)). Those seconds where enough to cut a ssh line. Not funny. Network traffic droped significantly. X/Desktop makes the problem visible, indeed. But not seeing it does not mean it isn't there. This might be the reason why FreeBSD is so much behind when it comes to X? I just want to make that clear to folks. This immense thread has been with regards to the latter -- bad interactivity/responsiveness on a system which was undergoing load that SHOULD be distributed more evenly across the system *while* keeping interactivity/responsiveness high. Historically nice/renice has been used for this task, but that was when kernels were a little less complex and I/O subsystems were less complex. Remember: we've now got schedulers for each type of thing, and who gets what priority? You get my point I'm sure. So remember: this was to discuss that aspect, with regards to ULE vs. 4BSD schedulers. Now, back to the benchmarks: This also interested me: * Linux system crashed http://leaf.dragonflybsd.org/mailarchive/kernel/2011-11/msg8.html * OpenIndiana system crashed same way as Linux system http://leaf.dragonflybsd.org/mailarchive/kernel/2011-11/msg00017.html I cannot help but wonder if the Linux and OpenIndiana installations were more stressful on the hardware -- getting more out of the system, maybe resulting in increased power/load, which in turn resulted in the systems locking up (shoddy PSU, unstable mainboard, MCH problems, etc.). Is FreeBSD supposed to run on dumpyard equipment? In former times, freeBSD was used on high value hardware, not the decomissioned crap with shoddy PSUs or whatsoever. If I need a server, I care about quality hardware as I do for my lab's box and my own box at home. I expect a server garde hardware to act like that and I expect the operating system to get the maximum out of that hardware. Otherwise it is not worth one shot. My point is that Francois states these things in such a way to imply that DragonflyBSD was more stable, when in fact I happen to wonder the opposite point -- that is to say, Linux and OpenIndiana were trying to use the hardware more-so than DragonflyBSD, thus tickled what may be a hardware-level problem. But this is only one suite of testing. Scientific Linux is supposed to give the best performance for scientifi purposes, i.e. for longhaul calculations, much numerical stuff. It outperforms in a typical server application FreeBSd, were
Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server
Is there a specific version of the test suite that should be used, to compare against the published results? Adrian On 20 December 2011 17:18, Matthew Tippett matt...@phoronix.com wrote: For such a system, the greatest immediate value would be to attempt to reproduce the benchmarks in question. Install PTS from www.phoronix-test-suite.com or freshports.org. Run the benchmark against those used in the article phoronix-test-suite benchmark 1112113-AR-ORACLELIN37 You will be asked to push the comparison up to openbenchmarking at the end. Matthew On 12/20/2011 01:39 PM, O. Hartmann wrote: On 12/20/11 21:20, Igor Mozolevsky wrote: Interestingly, while people seem to be (arguably rightly) focused on criticising Phoronix's benchmarking, nobody has offered an alternative benchmark; and while (again, arguably rightly) it is important to benchmark real world performance, equally, nobody has offered any numbers in relation to, for example, HTTP or SMTP, or any other real world-application torture tests done on the aforementioned two platforms... IMO, this just goes to show that doing is hard and criticising is much easier (yes, I am aware of the irony involved in making this statement, but someone has to!) Cheers, Igor M :-) Unfortunately, M. Larabel is the only one who's performing benchmarks on FreeBSD, comparing its performance to the Linux-opponents. Adn indeed, there is a lot of criticism, but no alternative. I said unfortunately - not offensive - since Larabel and Phoronix are sadly the only ones who do actually such bechmarking. It would be much more nicer and kind to support those people. Well, in January/February we get new hardware. One box is supposed to do number crunching via 12 cores and a TESLA GPU. My colleague is developing a high parallelized peice of software for satellite data transformation. The software package is CPU bound, partially GPU, but massively memory hungry (96 to 128 GB RAM is needed). What I can offer is, since I will also work on that machine and I've free hand to administer, in the spare time of doing my PhD, installing FreeBSD 9.0/10.0 besides SuSe Linux and looking forward having one ZFS data storage drive for homes, so both systems can perform on a most recent ZFS. I'm new to Linux, not a BSD guru, nor I'm a professional programmer/developer. My skills are sufficient for the daily scientific work. So, without pressure, I'm willing to perform some HPC benchmarks under advice if the day comes and those interested in bare numbers of FreeBSD vs. Linux performance with a real-world-scientific application. I would appreciate to see some of the developers and/or FreeBSD hackers to help Phoronix setting up a proper testenvironment instead of bashing M. Larabel and his fellows. Regards, Oliver ___ freebsd-sta...@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server
For such a system, the greatest immediate value would be to attempt to reproduce the benchmarks in question. Install PTS from www.phoronix-test-suite.com or freshports.org. Run the benchmark against those used in the article phoronix-test-suite benchmark 1112113-AR-ORACLELIN37 You will be asked to push the comparison up to openbenchmarking at the end. Matthew On 12/20/2011 01:39 PM, O. Hartmann wrote: On 12/20/11 21:20, Igor Mozolevsky wrote: Interestingly, while people seem to be (arguably rightly) focused on criticising Phoronix's benchmarking, nobody has offered an alternative benchmark; and while (again, arguably rightly) it is important to benchmark real world performance, equally, nobody has offered any numbers in relation to, for example, HTTP or SMTP, or any other real world-application torture tests done on the aforementioned two platforms... IMO, this just goes to show that doing is hard and criticising is much easier (yes, I am aware of the irony involved in making this statement, but someone has to!) Cheers, Igor M :-) Unfortunately, M. Larabel is the only one who's performing benchmarks on FreeBSD, comparing its performance to the Linux-opponents. Adn indeed, there is a lot of criticism, but no alternative. I said unfortunately - not offensive - since Larabel and Phoronix are sadly the only ones who do actually such bechmarking. It would be much more nicer and kind to support those people. Well, in January/February we get new hardware. One box is supposed to do number crunching via 12 cores and a TESLA GPU. My colleague is developing a high parallelized peice of software for satellite data transformation. The software package is CPU bound, partially GPU, but massively memory hungry (96 to 128 GB RAM is needed). What I can offer is, since I will also work on that machine and I've free hand to administer, in the spare time of doing my PhD, installing FreeBSD 9.0/10.0 besides SuSe Linux and looking forward having one ZFS data storage drive for homes, so both systems can perform on a most recent ZFS. I'm new to Linux, not a BSD guru, nor I'm a professional programmer/developer. My skills are sufficient for the daily scientific work. So, without pressure, I'm willing to perform some HPC benchmarks under advice if the day comes and those interested in bare numbers of FreeBSD vs. Linux performance with a real-world-scientific application. I would appreciate to see some of the developers and/or FreeBSD hackers to help Phoronix setting up a proper testenvironment instead of bashing M. Larabel and his fellows. Regards, Oliver ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server
The benchmarks themselves are versioned. So in general most of the av= ailable versions of PTS itself should be fine. PTS can be considered = an execution shell that doesn't affect the benchmark itself. Note th= at you'll download a pile of the benchmarks, build and install them. = Then you run about 49 individual steps. Matthew -- Sent from my HP Pre3 _ On Dec 20, 2011 5:30 PM, Adrian Chadd adr...@freebsd.org= ; wrote: Is there a specific version of the test suite that = should be used, to compare against the published results? Adrian On 20 December 2011 17:18, Matthew Tippett = ;matt...@phoronix.com wrote: For such a system, the greatest= immediate value would be to attempt to reproduce the benchmarks= in question. Install PTS from www.phoronix-test-suit= e.com or freshports.org. Run the benchmark against th= ose used in the article phoronix-test-su= ite benchmark 1112113-AR-ORACLELIN37 You will be aske= d to push the comparison up to openbenchmarking at the end. Matthew On 12/20/2011 01:39 PM, O. = Hartmann wrote: On 12/20/11 21:20, Igor Mozol= evsky wrote: Interestingly, while peo= ple seem to be (arguably rightly) focused on criticising= Phoronix's benchmarking, nobody has offered an alternative = gt; benchmark; and while (again, arguably rightly) it is important to benchmark real world performance, equally, nobody has offered any numbers in relation to, for example, HTTP or SMTP, = or any other real world-application torture tests done= on the aforementioned two platforms... IMO, this just g= oes to show that doing is hard and criticising is muc= h easier (yes, I am aware of the irony involved in maki= ng this statement, but someone has to!) g= t; Cheers, Igor M :-) = Unfortunately, M. Larabel is the only one who's performingbenchmarks on FreeBSD, comparing its performance to the Linu= x-opponents. Adn indeed, there is a lot of criticism, but no= alternative. I said unfortunately - not offensive - since L= arabel and Phoronix are sadly the only ones who do actually = such bechmarking. It would be much more nicer= and kind to support those people. Well, in J= anuary/February we get new hardware. One box is supposed to do g= t; number crunching via 12 cores and a TESLA GPU. My colleague is = ; developing a high parallelized peice of software for satellite data= transformation. The software package is CPU bound, partiall= y GPU, but massively memory hungry (96 to 128 GB RAM is need= ed). What I can offer is, since I will also work on that mac= hine and I've free hand to administer, in the spare time of = doing my PhD, installing FreeBSD 9.0/10.0 besides SuSe Linux= and looking forward having one ZFS data storage drive for h= omes, so both systems can perform on a most recent ZFS. I'm = new to Linux, not a BSD guru, nor I'm a professional program= mer/developer. My skills are sufficient for the daily scientific = work. So, without pressure, I'm willing to perform some HPC benchmarks= under advice if the day comes and those interested in barenumbers of FreeBSD vs. Linux performance with a real-world-s= cientific application. I would appreciate to = see some of the developers and/or FreeBSD hackers to help Ph= oronix setting up a proper testenvironment instead of bashing = ; M. Larabel and his fellows. Regards, = Oliver __= _ freebsd-sta...@freebsd.org mailing lis= t http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscribe@freebsd.= org ___ freebsd-pe= rforma...@freebsd.org mailing list http://lists.freebsd.org/mailman/l= istinfo/freebsd-performance To unsubscribe, send any mail to freebsd -performance-unsubscr...@freebsd.org ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server
Any version is fine that's PTS 3.0 or newer in terms of being compatible, since the test profiles are versioned separately and automatically fetched to match the result file. However, I'd recommended the newest (PTS 3.6) as it contains the best FreeBSD support at present in terms of hardware/software information parsing (for the automated table), etc. Michael On 12/20/2011 07:29 PM, Adrian Chadd wrote: Is there a specific version of the test suite that should be used, to compare against the published results? Adrian On 20 December 2011 17:18, Matthew Tippettmatt...@phoronix.com wrote: For such a system, the greatest immediate value would be to attempt to reproduce the benchmarks in question. Install PTS from www.phoronix-test-suite.com or freshports.org. Run the benchmark against those used in the article phoronix-test-suite benchmark 1112113-AR-ORACLELIN37 You will be asked to push the comparison up to openbenchmarking at the end. Matthew On 12/20/2011 01:39 PM, O. Hartmann wrote: On 12/20/11 21:20, Igor Mozolevsky wrote: Interestingly, while people seem to be (arguably rightly) focused on criticising Phoronix's benchmarking, nobody has offered an alternative benchmark; and while (again, arguably rightly) it is important to benchmark real world performance, equally, nobody has offered any numbers in relation to, for example, HTTP or SMTP, or any other real world-application torture tests done on the aforementioned two platforms... IMO, this just goes to show that doing is hard and criticising is much easier (yes, I am aware of the irony involved in making this statement, but someone has to!) Cheers, Igor M :-) Unfortunately, M. Larabel is the only one who's performing benchmarks on FreeBSD, comparing its performance to the Linux-opponents. Adn indeed, there is a lot of criticism, but no alternative. I said unfortunately - not offensive - since Larabel and Phoronix are sadly the only ones who do actually such bechmarking. It would be much more nicer and kind to support those people. Well, in January/February we get new hardware. One box is supposed to do number crunching via 12 cores and a TESLA GPU. My colleague is developing a high parallelized peice of software for satellite data transformation. The software package is CPU bound, partially GPU, but massively memory hungry (96 to 128 GB RAM is needed). What I can offer is, since I will also work on that machine and I've free hand to administer, in the spare time of doing my PhD, installing FreeBSD 9.0/10.0 besides SuSe Linux and looking forward having one ZFS data storage drive for homes, so both systems can perform on a most recent ZFS. I'm new to Linux, not a BSD guru, nor I'm a professional programmer/developer. My skills are sufficient for the daily scientific work. So, without pressure, I'm willing to perform some HPC benchmarks under advice if the day comes and those interested in bare numbers of FreeBSD vs. Linux performance with a real-world-scientific application. I would appreciate to see some of the developers and/or FreeBSD hackers to help Phoronix setting up a proper testenvironment instead of bashing M. Larabel and his fellows. Regards, Oliver ___ freebsd-sta...@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org ___ freebsd-sta...@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server
Hello, Samuel. You wrote 15 декабря 2011 г., 16:32:47: Other benchmarks in the Phoronix suite and their representations are similarly flawed, _ALL_ of these results should be ignored and no time should be wasted by any FreeBSD committer further evaluating this garbage. (Yes, I have been down this rabbit hole). Here is one problem: we have choice from three items: (1) Make FreeBSD looks good on benchmarks by fixing FreeBSD (2) Make FreeBSD looks good on benchmarks by fixing Phoronix (communication with them, convincing, that they benchamrks are unfare / meaningless, ets) (3) Lose [potential] userbase. You know, that these benchmarks are bad. I know. But potential (and even some current!) user doesn't. And it seems, that these benchmarks become popular over Internet. -- // Black Lion AKA Lev Serebryakov l...@freebsd.org ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server
On 19/12/2011 08:27, Lev Serebryakov wrote: Here is one problem: we have choice from three items: (1) Make FreeBSD looks good on benchmarks by fixing FreeBSD (2) Make FreeBSD looks good on benchmarks by fixing Phoronix (communication with them, convincing, that they benchamrks are unfare / meaningless, ets) (2a) Ignore Phoronix, other than explaining concisely why their numbers are complete balderdash. Publish our own benchmarks, done with care and rigour and using well defined, repeatable, peer reviewed methodology that anyone can repeat. Aggressively publicise these results. (3) Lose [potential] userbase. Indeed. Unfortunately performance is /the/ deciding factor in many OS choices, never mind that it is an impossibly complex subject to generalise to a few management-friendly numbers in a one-size-fits-all abstract way. Having only one source of published numbers suggesting that OS Foo is better *even if those numbers are completely bogus* will have a disproportionate effect. Cheers, Matthew -- Dr Matthew J Seaman MA, D.Phil. 7 Priory Courtyard Flat 3 PGP: http://www.infracaninophile.co.uk/pgpkey Ramsgate JID: matt...@infracaninophile.co.uk Kent, CT11 9PW signature.asc Description: OpenPGP digital signature
Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server
Hello, Adrian. You wrote 16 декабря 2011 г., 20:43:27: Guys/girls/fuzzy things - this is 2011; people look at shiny blog sites with graphs rather than mailing lists. Sorry, we lost that battle. :) My thoughts exactly. -- // Black Lion AKA Lev Serebryakov l...@freebsd.org ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server
Hello, Matthew. You wrote 19 декабря 2011 г., 13:13:09: (1) Make FreeBSD looks good on benchmarks by fixing FreeBSD (2) Make FreeBSD looks good on benchmarks by fixing Phoronix (communication with them, convincing, that they benchamrks are unfare / meaningless, ets) (2a) Ignore Phoronix, other than explaining concisely why their numbers are complete balderdash. Publish our own benchmarks, done with care and rigour and using well defined, repeatable, peer reviewed methodology that anyone can repeat. Aggressively publicise these results. Ok, it is The Way too, I agree. But in modern world, unfortunately (for me, and I'm sure, for many FreeBSD hackers), keywords are Aggressively publicise but not done with care and rigour and using well defined, repeatable, peer reviewed methodology that anyone can repeat (3) Lose [potential] userbase. Indeed. Unfortunately performance is /the/ deciding factor in many OS choices, never mind that it is an impossibly complex subject to generalise to a few management-friendly numbers in a one-size-fits-all abstract way. Having only one source of published numbers suggesting that OS Foo is better *even if those numbers are completely bogus* will have a disproportionate effect. Yep. -- // Black Lion AKA Lev Serebryakov l...@freebsd.org ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server
On Mon, Dec 19, 2011 at 09:13:09AM +, Matthew Seaman wrote: On 19/12/2011 08:27, Lev Serebryakov wrote: Here is one problem: we have choice from three items: (1) Make FreeBSD looks good on benchmarks by fixing FreeBSD (2) Make FreeBSD looks good on benchmarks by fixing Phoronix (communication with them, convincing, that they benchamrks are unfare / meaningless, ets) (2a) Ignore Phoronix, other than explaining concisely why their numbers are complete balderdash. Publish our own benchmarks, done with care and rigour and using well defined, repeatable, peer reviewed methodology that anyone can repeat. Aggressively publicise these results. Slashdot and others don't ignore Phoronix, so (2a) is only and option if you accept (3). My personal opinion: Phoronix may compare apples to oranges from time to time and it might be possible to catch up with Linux' results by tweaking some system parameters, but Joe Average expects a fast and working OS out-of-the-box and after reading a Phoronix benchmark, he will probably prefer Linux over FreeBSD. /me thinks that our userbase is not big enough to put off potential new or existing users, so we should question our default config values or clearly and publicly explain why the results for FreeBSD are slower because of data integrity / security / $other_reasons. (3) Lose [potential] userbase. Indeed. Unfortunately performance is /the/ deciding factor in many OS choices, never mind that it is an impossibly complex subject to generalise to a few management-friendly numbers in a one-size-fits-all abstract way. Having only one source of published numbers suggesting that OS Foo is better *even if those numbers are completely bogus* will have a disproportionate effect. pgpUttizlWefQ.pgp Description: PGP signature
Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server
On 12/19/11 09:27, Lev Serebryakov wrote: Hello, Samuel. You wrote 15 декабря 2011 г., 16:32:47: Other benchmarks in the Phoronix suite and their representations are similarly flawed, _ALL_ of these results should be ignored and no time should be wasted by any FreeBSD committer further evaluating this garbage. (Yes, I have been down this rabbit hole). Here is one problem: we have choice from three items: (1) Make FreeBSD looks good on benchmarks by fixing FreeBSD (2) Make FreeBSD looks good on benchmarks by fixing Phoronix (communication with them, convincing, that they benchamrks are unfare / meaningless, ets) (3) Lose [potential] userbase. You know, that these benchmarks are bad. I know. But potential (and even some current!) user doesn't. And it seems, that these benchmarks become popular over Internet. +1 It is not about a faky way to let a specific OS look good by any means. I'M afraid of (3), which also implies pushing more towards beeing meaningless and not anymore a alternative with a unique, remarkable criteria to be choosen as __the__ operating system of the first choice for several purposes. By the way, how such a development could look alaike is very clear when it comes to GPGPU/HPC, highly related to the availability of proper graphics card drivers, X11 development and the necessary libraries, APIs and even compilers. None of those professionals out here, none of those pushing the eyewhitness of bad performance into very deep-insight-talks about what could cause the problem has obviously ever negotiated with people of the upper floor when it comes to the choice of the OS. Within my department, the *BSD aren't even considered an option, even if they would perform best for the specified purpose (which, I regeret, is a shrinking basis now since also Linux will have ZFS). Sometimes I feel like Don Quixote, fighting against windmills. Sorry having brought up this thread and I beg for pardon for putting another scrtach into the autoerotic world of the core. signature.asc Description: OpenPGP digital signature
Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server
2011/12/19 Lev Serebryakov l...@freebsd.org: Hello, Samuel. You wrote 15 декабря 2011 г., 16:32:47: Other benchmarks in the Phoronix suite and their representations are similarly flawed, _ALL_ of these results should be ignored and no time should be wasted by any FreeBSD committer further evaluating this garbage. (Yes, I have been down this rabbit hole). Here is one problem: we have choice from three items: (1) Make FreeBSD looks good on benchmarks by fixing FreeBSD (2) Make FreeBSD looks good on benchmarks by fixing Phoronix (communication with them, convincing, that they benchamrks are unfare / meaningless, ets) (3) Lose [potential] userbase. You know, that these benchmarks are bad. I know. But potential (and even some current!) user doesn't. And it seems, that these benchmarks become popular over Internet. -- // Black Lion AKA Lev Serebryakov l...@freebsd.org Here is where you completely derail the train, let me paste again what I said before. ... Take the first test as an example, Blogbench read. This doesn't raise any red flags, right? At least not until you realize that Blogbench isn't a read test, it's a read/write test. So what they have done here is run a read/write test and then thrown away the write results for both platforms and reported only the read results. If you dig down into the actual results, http://openbenchmarking.org/result/1112113-AR-ORACLELIN37 -- you will see two Blogbench numbers, one for read and another for write. These were both taken from the same Blogbench run, so FreeBSD optimizes writes over reads, that's probably a good thing for your data but a bad thing when someone totally misrepresents benchmark results. ... FreeBSD actually does _BETTER_ (subjectively) in this test than the Linux system when you look at what is really going on. FreeBSD is favoring writes, which is _GOOD_. FreeBSD does not need to be fixed, the benchmarks need to be fixed to represent reality rather than throwing half of the results in the trash. To be quite frank, fixing FreeBSD to look good on this benchmark will make it a worse real-world OS. But you guys go ahead and foot-shoot over these ridiculous benchmarks all you want. Sam ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server
On Mon, Dec 19, 2011 at 6:49 PM, Samuel J. Greear s...@evilcode.net wrote: FreeBSD actually does _BETTER_ (subjectively) in this test than the Linux system when you look at what is really going on. FreeBSD is favoring writes, which is _GOOD_. FreeBSD does not need to be fixed, the benchmarks need to be fixed to represent reality rather than throwing half of the results in the trash. To be quite frank, fixing FreeBSD to look good on this benchmark will make it a worse real-world OS. But you guys go ahead and foot-shoot over these ridiculous benchmarks all you want. Would you prefer a blog which allows you to: A: - create/write 100 posts/s - serve/read 1000 posts/s or B: - create/write 80 posts/s - serve/read 3000 posts/s ? I would personally choose B. -- O ascii ribbon campaign - stop html mail - www.asciiribbon.org ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server
IMHO, no offence, as always. As were told, Phoronix used default setup, not tuned. So? Is average user will tune it after setup? No, he'll get same defaults, and would expect same performance as in tests, and he probably get it. The problem of FreeBSD is not it's default settings, some kind of very-safe defaults really should be there. But problem really is lacking of choosing them (defaults) during install, for average users. For example, few checkboxes with common sysctl tuning would be perfect, even if they would be marked as Experimental, or not recommended. I'm thinking it's better way to make something in one place (like in installer) rather than require make almost same actions in many (hundreds of thousands?... more?...) places (end-users forced to read mail-lists/handbooks/forums over and over for same solutions). Simple example - many connections for PostgreSQL is not available on FreeBSD out-of-box. Just google postgresql freebsd max connection and you'll see how many there bikesheds requested and same solutions posted again and again :) FreeBSD currently have very obscure, closed community. To get in touch, you need to subscribe to several mail lists, constantly read them, I've just found recently (my shame of course) in mail list that there is service ( pub.allbsd.org) which constantly building current versions. This is great, but at homepage of freebsd.org there is no word about it :) I hope we all do something good about this, and things will going to change. -- Regards, Alexander Yerenkow ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server
On 19 dec 2011, at 12:50, Samuel J. Greear s...@evilcode.net wrote: 2011/12/19 Lev Serebryakov l...@freebsd.org: Hello, Samuel. You wrote 15 декабря 2011 г., 16:32:47: Other benchmarks in the Phoronix suite and their representations are similarly flawed, _ALL_ of these results should be ignored and no time should be wasted by any FreeBSD committer further evaluating this garbage. (Yes, I have been down this rabbit hole). Here is one problem: we have choice from three items: (1) Make FreeBSD looks good on benchmarks by fixing FreeBSD (2) Make FreeBSD looks good on benchmarks by fixing Phoronix (communication with them, convincing, that they benchamrks are unfare / meaningless, ets) (3) Lose [potential] userbase. You know, that these benchmarks are bad. I know. But potential (and even some current!) user doesn't. And it seems, that these benchmarks become popular over Internet. -- // Black Lion AKA Lev Serebryakov l...@freebsd.org Here is where you completely derail the train, let me paste again what I said before. ... Take the first test as an example, Blogbench read. This doesn't raise any red flags, right? At least not until you realize that Blogbench isn't a read test, it's a read/write test. So what they have done here is run a read/write test and then thrown away the write results for both platforms and reported only the read results. If you dig down into the actual results, http://openbenchmarking.org/result/1112113-AR-ORACLELIN37 -- you will see two Blogbench numbers, one for read and another for write. These were both taken from the same Blogbench run, so FreeBSD optimizes writes over reads, that's probably a good thing for your data but a bad thing when someone totally misrepresents benchmark results. ... FreeBSD actually does _BETTER_ (subjectively) in this test than the Linux system when you look at what is really going on. FreeBSD is favoring writes, which is _GOOD_. FreeBSD does not need to be fixed, the benchmarks need to be fixed to represent reality rather than throwing half of the results in the trash. To be quite frank, fixing FreeBSD to look good on this benchmark will make it a worse real-world OS. But you guys go ahead and foot-shoot over these ridiculous benchmarks all you want. Sam I seem to remember that before ULE people were fleeing to Linux as the os to run apache on since 4BSD didn't scale all too well. That may have changed over time though. However ULE could perhaps be made aware technologies like turbo-boost, ie with few threads higher performance might be gained by utilizing all virtual cores on a physical core before spreading tasks to too different cores. Just my speculations though :) Regards Andreas Nilsson ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server
On 12/19/11 13:21, Andreas Nilsson wrote: On 19 dec 2011, at 12:50, Samuel J. Greear s...@evilcode.net wrote: 2011/12/19 Lev Serebryakov l...@freebsd.org: Hello, Samuel. You wrote 15 декабря 2011 г., 16:32:47: Other benchmarks in the Phoronix suite and their representations are similarly flawed, _ALL_ of these results should be ignored and no time should be wasted by any FreeBSD committer further evaluating this garbage. (Yes, I have been down this rabbit hole). Here is one problem: we have choice from three items: (1) Make FreeBSD looks good on benchmarks by fixing FreeBSD (2) Make FreeBSD looks good on benchmarks by fixing Phoronix (communication with them, convincing, that they benchamrks are unfare / meaningless, ets) (3) Lose [potential] userbase. You know, that these benchmarks are bad. I know. But potential (and even some current!) user doesn't. And it seems, that these benchmarks become popular over Internet. -- // Black Lion AKA Lev Serebryakov l...@freebsd.org Here is where you completely derail the train, let me paste again what I said before. ... Take the first test as an example, Blogbench read. This doesn't raise any red flags, right? At least not until you realize that Blogbench isn't a read test, it's a read/write test. So what they have done here is run a read/write test and then thrown away the write results for both platforms and reported only the read results. If you dig down into the actual results, http://openbenchmarking.org/result/1112113-AR-ORACLELIN37 -- you will see two Blogbench numbers, one for read and another for write. These were both taken from the same Blogbench run, so FreeBSD optimizes writes over reads, that's probably a good thing for your data but a bad thing when someone totally misrepresents benchmark results. ... FreeBSD actually does _BETTER_ (subjectively) in this test than the Linux system when you look at what is really going on. FreeBSD is favoring writes, which is _GOOD_. FreeBSD does not need to be fixed, the benchmarks need to be fixed to represent reality rather than throwing half of the results in the trash. To be quite frank, fixing FreeBSD to look good on this benchmark will make it a worse real-world OS. But you guys go ahead and foot-shoot over these ridiculous benchmarks all you want. Sam I seem to remember that before ULE people were fleeing to Linux as the os to run apache on since 4BSD didn't scale all too well. That may have changed over time though. However ULE could perhaps be made aware technologies like turbo-boost, ie with few threads higher performance might be gained by utilizing all virtual cores on a physical core before spreading tasks to too different cores. Just my speculations though :) Regards Andreas Nilsson Such a scheduling stratey is definitely necessary on AMDs new Bulldozer architecture, which seems to be very pitty about threads locked on the same module. Microsoft just offered a patch for Windows 7 to implant such a Bulldozer awarenes but they withdraw the patch as invalid two days after the release. The seults seem to favour FPU performance over integer performance. As Samuel Greear wrote, FreeBSD looks not that bad in some of the benchmarks but there are obviosly issues, at least the fact that Phoronix/openbenchmark.org are the only sites offering benchmarks at all. People outside the FreeBSD realm looking for opportunities, what do you think they will look first after? Phoronix/Openbenchmark.org made the first step and they seem to make FreeBSD look bad (in my opinion), whether righteous or not. Compared to several subjective impressions I have in our heterogeneous environment at the lab, Linux on the same hardware looks in several aspects much better. Oliver signature.asc Description: OpenPGP digital signature
Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server
I have already canceled few replies to this thread, but... On 19.12.11 15:16, Alexander Yerenkow wrote: IMHO, no offence, as always. I feel obliged to include the same disclaimer :-) As were told, Phoronix used default setup, not tuned. Not really. They created some weird test environment, at least for FreeBSD -- who knows, possibly for Linux as well. For example, ZFS is by no means a default file system in FreeBSD. You need to go trough manual steps, to enable it, to build the pool, filesystems etc. This is because ZFS is very powerful file system and storage manager that needs some thinking before you implement it -- then it may reward you with features not found anywhere else. Funny, ZFS is available in Linux too, and at least the file system tests might benefit from using one and the same file system. One would expect that ZFS was used for both, in a multiple-disk (way over 4 disks) setup, as one would expect to be the case for a 'server'. So? Is average user will tune it after setup? No, he'll get same defaults, and would expect same performance as in tests, and he probably get it. You forget, that the FreeBSD type and the Linux type are quite different. This is why both worlds exist. The FreeBSD way is to understand what you do and configure your environment accordingly. FreeBSD gives you flexibility to do as you please and in most of the possible configurations it will work. Maybe not optimally, but will not break on you. With FreeBSD there is never one true way to do things. The Linux way on the other hand is to follow a HowTo instruction. The Linux OS is typically optimized for these setups and as long as you follow the HOWTO you are safe and well performance-wise. If you go way out of the prescriptions in the HOWTO, you may end up with losing data, crashing system or extremely poor performance. I know, things are not that black and white, but this is the general difference. But problem really is lacking of choosing them (defaults) during install, for average users. Who are the average users? It has been repeatedly said, that the PC user is always better to start with PC-BSD, because it is FreeBSD with safe defaults suitable for a desktop. For example, few checkboxes with common sysctl tuning would be perfect, even if they would be marked as Experimental, or not recommended. By following this, we push FreeBSD into the Linux style of doing things: someone else decides what is good for you, without having a clue of your circumstances. Simple example - many connections for PostgreSQL is not available on FreeBSD out-of-box. Just google postgresql freebsd max connection and you'll see how many there bikesheds requested and same solutions posted again and again :) Still, PostgreSQL is not part of FreeBSD. The PostgreSQL port clearly says what you need to adjust in your setup in order to use it. As do most other ports. Computers do what people ask them to do -- we are far from the AI times, when the computers will assembe, configure and run themselves the way we think they should. FreeBSD currently have very obscure, closed community. Some say this is a feature ;-) To get in touch, you need to subscribe to several mail lists, constantly read them, I've just found recently (my shame of course) in mail list that there is service (pub.allbsd.org) which constantly building current versions. This is great, but at homepage of freebsd.org there is no word about it :) There is a menu Community on www.freebsd.org and an Forums entry there. You don't have to use mailing lists, of you prefer forums. I hope we all do something good about this, and things will going to change. Many bright people do a lot of things about all of these issues. If there is a problem, one needs to understand the problem, what causes the problem and what are the implications. Merely reacting on the symptoms never helps in the long run, as the core problem is not resolved. So far in this thread there is no evidence of where the problem is. There is no evidence even if there is a real problem -- except that many people get overly excited by benchmarks. To the last point I could add that, with experience, one learns that: the benchmarks done in your environment, with your settings, with your OS version, on your hardware and with your set of applications does not help me much on my hardware/software/configuration -- except if these happen to be very similar. /usr/ports/benchmarks is your friend. Daniel ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server
Hi all, just a thought here: On Tue, Dec 20, 2011 at 12:45 AM, Daniel Kalchev dan...@digsys.bg wrote: As were told, Phoronix used default setup, not tuned. Not really. They created some weird test environment, at least for FreeBSD -- who knows, possibly for Linux as well. For example, ZFS is by no means a default file system in FreeBSD. You need to go trough manual steps, to enable it, to build the pool, filesystems etc. .. Of course the benchmark setup and procedure is strange but.. it could be improved, I think. Have a good collection of tuning parameters for popular cases, advertised properly so it gets hard to miss them. I am a sysadmin and, over the years, I had to run file servers, database servers, web servers, tomcats... Well, most of the time I set it up and it just works because the system in question is not maxed out, not even close to it. But if I want to squeeze the last 20% out of it googling starts, and here and there I find hints how to tune the OS, the file system, what scheduler to use etc. It would be great to have a set of case studies at hand, e.g. under the /usr/share/examples directory, that describes tweaks to have a performing postgresql server, or mysql, or apache or a desktop or.. Things I find, for example, in the BSD Magazine. Maybe benchmarks become more meaningful then.. A general remark for people doing benchmarks for comparison: you need a well-informed system engineer for the systems you compare. So, if you compare a Linux system with FreeBSD, have two experienced admins that know their OS well. Regards Peter ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server
Am 15.12.2011, 11:10 Uhr, schrieb Michael Larabel michael.lara...@phoronix.com: On 12/15/2011 02:48 AM, Michael Ross wrote: Anyway these tests were performed on different hardware, FWIW. And with different filesystems, different compilers, different GUIs... No, the same hardware was used for each OS. The picture under the heading System Hardware / Software does not reflect that. Motherboard description differs, Chipset description for FreeBSD is empty. Regards, Michael In terms of the software, the stock software stack for each OS was used. -- Michael ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server
On 12/15/2011 04:41 AM, Michael Ross wrote: Am 15.12.2011, 11:10 Uhr, schrieb Michael Larabel michael.lara...@phoronix.com: On 12/15/2011 02:48 AM, Michael Ross wrote: Anyway these tests were performed on different hardware, FWIW. And with different filesystems, different compilers, different GUIs... No, the same hardware was used for each OS. The picture under the heading System Hardware / Software does not reflect that. Motherboard description differs, Chipset description for FreeBSD is empty. I was the on that carried out the testing and know that it was on the same system. All of the testing, including the system tables, is fully automated. Under FreeBSD sometimes the parsing of some component strings isn't as nice as Linux and other supported operating systems by the Phoronix Test Suite. For the BSD motherboard string parsing it's grabbing hw.vendor/hw.product from sysctl. Is there a better place to read the motherboard DMI information from? -- Michael Regards, Michael In terms of the software, the stock software stack for each OS was used. -- Michael ___ freebsd-sta...@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server
Am 15.12.2011, 11:55 Uhr, schrieb Michael Larabel michael.lara...@phoronix.com: On 12/15/2011 04:41 AM, Michael Ross wrote: Am 15.12.2011, 11:10 Uhr, schrieb Michael Larabel michael.lara...@phoronix.com: On 12/15/2011 02:48 AM, Michael Ross wrote: Anyway these tests were performed on different hardware, FWIW. And with different filesystems, different compilers, different GUIs... No, the same hardware was used for each OS. The picture under the heading System Hardware / Software does not reflect that. Motherboard description differs, Chipset description for FreeBSD is empty. I was the on that carried out the testing and know that it was on the same system. No offense. I'm not doubting you. But I didn't know this: All of the testing, including the system tables, is fully automated. Under FreeBSD sometimes the parsing of some component strings isn't as nice as Linux and other supported operating systems by the Phoronix Test Suite. For the BSD motherboard string parsing it's grabbing hw.vendor/hw.product from sysctl. so maybe you can understand how I got my impression. NVidia Audio and Realtek Audio. Looks different to me :-) Is there a better place to read the motherboard DMI information from? Following Steven Hartlands' suggestion, from one of my machines: /usr/ports/sysutils/dmidecode/#sysctl -a | egrep hw.vendor|hw.product /usr/ports/sysutils/dmidecode/#dmidecode -t 2 # dmidecode 2.11 SMBIOS 2.6 present. Handle 0x0002, DMI type 2, 15 bytes Base Board Information Manufacturer: FUJITSU Product Name: D2759 Version: S26361-D2759-A13 WGS04 GS02 Serial Number: 35838599 Asset Tag: - Features: Board is a hosting board Board is removable Location In Chassis: - Chassis Handle: 0x0003 Type: Motherboard Contained Object Handles: 0 Nice. Didn't know about that. Regards, Michael ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server
Hi, all, Am 15.12.2011 um 12:18 schrieb Michael Ross: Following Steven Hartlands' suggestion, from one of my machines: /usr/ports/sysutils/dmidecode/#sysctl -a | egrep hw.vendor|hw.product /usr/ports/sysutils/dmidecode/#dmidecode -t 2 # dmidecode 2.11 SMBIOS 2.6 present. Handle 0x0002, DMI type 2, 15 bytes Base Board Information Manufacturer: FUJITSU Product Name: D2759 Version: S26361-D2759-A13 WGS04 GS02 Serial Number: 35838599 Asset Tag: - Features: Board is a hosting board Board is removable Location In Chassis: - Chassis Handle: 0x0003 Type: Motherboard Contained Object Handles: 0 Without the need to install an additional port: datatomb2# kenv … smbios.bios.reldate=11/03/2011 smbios.bios.vendor=FUJITSU // American Megatrends Inc. smbios.bios.version=V4.6.4.1 R1.18.0 for D3034-A1x smbios.chassis.maker=FUJITSU smbios.chassis.serial=YLAP004857 smbios.chassis.tag=System Asset Tag smbios.chassis.version=RX100S7R2 smbios.memory.enabled=8388608 smbios.planar.maker=FUJITSU smbios.planar.product=D3034-A1 smbios.planar.serial=LJ1B-P00996 smbios.planar.version=S26361-D3034-A100 WGS01 GS02 smbios.socket.enabled=1 smbios.socket.populated=1 smbios.system.maker=FUJITSU smbios.system.product=PRIMERGY RX100 S7 smbios.system.serial=YLAP004857 smbios.system.uuid=f0493081-f5ca-e011-b8a5-a1c4d143da5f smbios.system.version=GS02 smbios.version=2.7 … Kind regards, Patrick -- punkt.de GmbH * Kaiserallee 13a * 76133 Karlsruhe Tel. 0721 9109 0 * Fax 0721 9109 100 i...@punkt.de http://www.punkt.de Gf: Jürgen Egeling AG Mannheim 108285 ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server
On Thu, Dec 15, 2011 at 04:55:16AM -0600, Michael Larabel wrote:
On 12/15/2011 04:41 AM, Michael Ross wrote:
Am 15.12.2011, 11:10 Uhr, schrieb Michael Larabel
michael.lara...@phoronix.com:
On 12/15/2011 02:48 AM, Michael Ross wrote:
Anyway these tests were performed on different hardware, FWIW.
And with different filesystems, different compilers, different GUIs...
No, the same hardware was used for each OS.
The picture under the heading System Hardware / Software does
not reflect that.
Motherboard description differs, Chipset description for FreeBSD
is empty.
I was the on that carried out the testing and know that it was on
the same system.
All of the testing, including the system tables, is fully automated.
Under FreeBSD sometimes the parsing of some component strings isn't
as nice as Linux and other supported operating systems by the
Phoronix Test Suite. For the BSD motherboard string parsing it's
grabbing hw.vendor/hw.product from sysctl.
Is there a better place to read the motherboard DMI information from?
I *think* what you're referring to is SMBIOS strings -- and these are
available from kenv(1) / kenv(2), not sysctl. But keep reading for why
SMBIOS data is not 100% reliable (greatly depends on the hardware). For
actual device strings/etc. for all devices on busses (PCI, AGP, etc.)
you can use pciconf -lvcb.
That's about as good as it's going to get via software. SMBIOS data
(e.g. smbios.{bios,chassis,planar,system}) is never going to give you
fully-identifiable data; I can point you to tons of systems where the
data inserted there is nonsense, sometimes even just ASCII spaces (and
that is the fault of the system vendor/BIOS manufacturer, not FreeBSD).
Sometimes identical strings are used across completely different
systems/boards (sometimes even server-class boards like ones from
Supermicro). And PCI vendor strings don't give you things like speeds,
frequency/voltages, etc.. Sometimes this matters. For example (just
making something up): the video benchmark was horrible on FreeBSD,
when in fact it turned out that a run of pciconf -lvcb showed your
PCIe card was running at x4 link speed instead of x16.
The best place to get your specifications from are:
* The box
* The physical hardware (by physically inspecting it)
* The user manual / product documentation/
* Purchase orders from whoever bought the hardware
* And, of course, operational speed (if possible) from the OS/userland
utilities
When I read a benchmark/review, I have to assume the person is doing
them on a system they have 100% control over, all the way down to the
hardware. Thus, they should know what exact hardware they have.
Also, when publishing results online, you should take the time to
proofread everything (with a 2nd set of eyes if possible) and be patient
and thorough. People like accuracy, especially when there's hard
data/evidence to back it up that can be made available for download.
Try to understand: so many review-esque sites consist of individuals who
do not understand even remotely what they're doing.
I'm going to give you two examples -- one personal, one word-of-mouth
but from someone I trust dearly.
I have a reverse analysis of Anantech's Intel 510 SSD review that has been
sitting in my draft folder on my blog for a month now because I'm
downright afraid to publish how their data seems completely and totally
wrong (with evidence to prove it). I'm afraid/stalling because I want
to make absolutely damn sure I'm not missing some key piece of evidence
that explains it, and I've had multiple people read it and go ...wow, I
didn't notice that, that benchmark data makes no sense, but I'm STILL
reluctant. The last thing I want to do is publish something that
sparks a controversy where it turns out I'm wrong (and I AM wrong, quite
often!).
As for the other:
http://www.overclockers.com/bulldozer-architecture-explained/
The author of this review talks about CPU arch and is praised for
writing a wonderful article that speaks the truth. But sadly that
doesn't appear to be the case. A colleague of mine is long-time friends
with another individual who is getting his Ph.D in computer architecture
and recently submit a paper to a journal (and was published/accepted)
which has published papers on things like RAID (when it was first
introduced as a concept/method), and hardware watchpoints. Said
individual read the above review and described it as, quote, the
worst article on computer architecture on the entire Internet. One of
the amusing quotes (that got me laughing since I did understand it; my
understanding of CPUs on a silicon level is limited, I'm just an old
65xxx assembly programmer...) was how the article states this is the
first time AMD has implemented branch prediction. Sigh.
Here's the kicker: said individual immediately recognised that the
article was a near dry cut-and-paste from one of two commonly-used
computer architecture books in
Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server
On Thu, Dec 15, 2011 at 05:32:47AM -0700, Samuel J. Greear wrote: Well, the only way it's going to get fixed is if someone sits down, replicates it, and starts to document exactly what it is that these benchmarks are/aren't doing. I think you will find that investigation is largely a waste of time, because not only are some of these benchmarks just downright silly, there are huge differences in the environments (compiler versions), etc., etc. leading to a largely apples/oranges comparison. But also the the analysis and reporting of the results by Phoronix is simply moronic to the point of being worse than useful, they are spreading misinformation. Take the first test as an example, Blogbench read. This doesn't raise any red flags, right? At least not until you realize that Blogbench isn't a read test, it's a read/write test. So what they have done here is run a read/write test and then thrown away the write results for both platforms and reported only the read results. If you dig down into the actual results, http://openbenchmarking.org/result/1112113-AR-ORACLELIN37 -- you will see two Blogbench numbers, one for read and another for write. These were both taken from the same Blogbench run, so FreeBSD optimizes writes over reads, that's probably a good thing for your data but a bad thing when someone totally misrepresents benchmark results. Other benchmarks in the Phoronix suite and their representations are similarly flawed, _ALL_ of these results should be ignored and no time should be wasted by any FreeBSD committer further evaluating this garbage. (Yes, I have been down this rabbit hole). For sake of argument, let's say we throw out the Phoronix benchmarks as a data source (I don't think the benchmark specifically implied or stated this is all because of SCHED_ULE though; remember, that's what we're supposed to be focusing on. There may not be a direct correlation between the Phoronix benchmarks and the ULE issue reported here...). That said: thrown out, data ignored, done. Now what? Where are we? We're right back where we were a day or two ago; meaning no closer to solving the dilemma reported by users and SCHED_ULE. Heck, we're not even sure if there is an issue, other than some folks confirming that SCHED_4BSD performs better for them (that's what started this whole thread), and there are at least a couple which have stated this. So given the above semi-devil's-advocate response -- Sam, do you have something positive or progressive to offer so we can move forward on the ULE vs. 4BSD debacle? :-) The smiley is meant to be sincere, not sarcastic. I'm getting to the point where I'm considering formulating a private mail to Jeff Roberson, requesting that he be aware of the discussion that's happening (not that he necessarily follow or read it), and that based on what I can tell we're at a roadblock -- nobody so far is absolutely certain how to benchmark and compare ULE vs. 4BSD in multiple ways, so that those of us involved here can run such utilities and provide the data somewhere central for devs to review. I only mention this because so far I haven't seen anyone really say okay, this is what we should be using for these kinds of tests. Yay nature of the beast. -- | Jeremy Chadwickjdc at parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, US | | Making life hard for others since 1977. PGP 4BD6C0CB | ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server
On 12/15/2011 08:26 AM, Sergey Matveychuk wrote: 15.12.2011 17:36, Michael Larabel пишет: On 12/15/2011 07:25 AM, Stefan Esser wrote: Am 15.12.2011 11:10, schrieb Michael Larabel: No, the same hardware was used for each OS. In terms of the software, the stock software stack for each OS was used. Just curious: Why did you choose ZFS on FreeBSD, while UFS2 (with journaling enabled) should be an obvious choice since it is more similar in concept to ext4 and since that is what most FreeBSD users will use with FreeBSD? I was running some ZFS vs. UFS tests as well and this happened to have ZFS on when I was running some other tests. Can we look at the tests? My opinion is ZFS without tuning is much slower than UFS2. http://www.phoronix.com/scan.php?page=news_itempx=MTAyNjg ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
