Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server
Any version is fine that's PTS 3.0 or newer in terms of being compatible, since the test profiles are versioned separately and automatically fetched to match the result file. However, I'd recommended the newest (PTS 3.6) as it contains the best FreeBSD support at present in terms of hardware/software information parsing (for the automated table), etc. Michael On 12/20/2011 07:29 PM, Adrian Chadd wrote: Is there a specific version of the test suite that should be used, to compare against the published results? Adrian On 20 December 2011 17:18, Matthew Tippettmatt...@phoronix.com wrote: For such a system, the greatest immediate value would be to attempt to reproduce the benchmarks in question. Install PTS from www.phoronix-test-suite.com or freshports.org. Run the benchmark against those used in the article phoronix-test-suite benchmark 1112113-AR-ORACLELIN37 You will be asked to push the comparison up to openbenchmarking at the end. Matthew On 12/20/2011 01:39 PM, O. Hartmann wrote: On 12/20/11 21:20, Igor Mozolevsky wrote: Interestingly, while people seem to be (arguably rightly) focused on criticising Phoronix's benchmarking, nobody has offered an alternative benchmark; and while (again, arguably rightly) it is important to benchmark real world performance, equally, nobody has offered any numbers in relation to, for example, HTTP or SMTP, or any other real world-application torture tests done on the aforementioned two platforms... IMO, this just goes to show that doing is hard and criticising is much easier (yes, I am aware of the irony involved in making this statement, but someone has to!) Cheers, Igor M :-) Unfortunately, M. Larabel is the only one who's performing benchmarks on FreeBSD, comparing its performance to the Linux-opponents. Adn indeed, there is a lot of criticism, but no alternative. I said unfortunately - not offensive - since Larabel and Phoronix are sadly the only ones who do actually such bechmarking. It would be much more nicer and kind to support those people. Well, in January/February we get new hardware. One box is supposed to do number crunching via 12 cores and a TESLA GPU. My colleague is developing a high parallelized peice of software for satellite data transformation. The software package is CPU bound, partially GPU, but massively memory hungry (96 to 128 GB RAM is needed). What I can offer is, since I will also work on that machine and I've free hand to administer, in the spare time of doing my PhD, installing FreeBSD 9.0/10.0 besides SuSe Linux and looking forward having one ZFS data storage drive for homes, so both systems can perform on a most recent ZFS. I'm new to Linux, not a BSD guru, nor I'm a professional programmer/developer. My skills are sufficient for the daily scientific work. So, without pressure, I'm willing to perform some HPC benchmarks under advice if the day comes and those interested in bare numbers of FreeBSD vs. Linux performance with a real-world-scientific application. I would appreciate to see some of the developers and/or FreeBSD hackers to help Phoronix setting up a proper testenvironment instead of bashing M. Larabel and his fellows. Regards, Oliver ___ freebsd-sta...@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org ___ freebsd-sta...@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server
On 12/15/2011 04:41 AM, Michael Ross wrote: Am 15.12.2011, 11:10 Uhr, schrieb Michael Larabel michael.lara...@phoronix.com: On 12/15/2011 02:48 AM, Michael Ross wrote: Anyway these tests were performed on different hardware, FWIW. And with different filesystems, different compilers, different GUIs... No, the same hardware was used for each OS. The picture under the heading System Hardware / Software does not reflect that. Motherboard description differs, Chipset description for FreeBSD is empty. I was the on that carried out the testing and know that it was on the same system. All of the testing, including the system tables, is fully automated. Under FreeBSD sometimes the parsing of some component strings isn't as nice as Linux and other supported operating systems by the Phoronix Test Suite. For the BSD motherboard string parsing it's grabbing hw.vendor/hw.product from sysctl. Is there a better place to read the motherboard DMI information from? -- Michael Regards, Michael In terms of the software, the stock software stack for each OS was used. -- Michael ___ freebsd-sta...@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server
On 12/15/2011 08:26 AM, Sergey Matveychuk wrote: 15.12.2011 17:36, Michael Larabel пишет: On 12/15/2011 07:25 AM, Stefan Esser wrote: Am 15.12.2011 11:10, schrieb Michael Larabel: No, the same hardware was used for each OS. In terms of the software, the stock software stack for each OS was used. Just curious: Why did you choose ZFS on FreeBSD, while UFS2 (with journaling enabled) should be an obvious choice since it is more similar in concept to ext4 and since that is what most FreeBSD users will use with FreeBSD? I was running some ZFS vs. UFS tests as well and this happened to have ZFS on when I was running some other tests. Can we look at the tests? My opinion is ZFS without tuning is much slower than UFS2. http://www.phoronix.com/scan.php?page=news_itempx=MTAyNjg ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server
On 12/15/2011 07:25 AM, Stefan Esser wrote: Am 15.12.2011 11:10, schrieb Michael Larabel: No, the same hardware was used for each OS. In terms of the software, the stock software stack for each OS was used. Just curious: Why did you choose ZFS on FreeBSD, while UFS2 (with journaling enabled) should be an obvious choice since it is more similar in concept to ext4 and since that is what most FreeBSD users will use with FreeBSD? I was running some ZFS vs. UFS tests as well and this happened to have ZFS on when I was running some other tests. Did you tune the ZFS ARC (e.g. vfs.zfs.arc_max=6G) for the tests? The OS was left in its stock configuration. And BTW: Did your measured run times account for the effect, that Linux keeps much more dirty data in the buffer cache (FreeBSD has a low limit on dirty buffers since under realistic load the already cached data is much more likely to be reused and thus more valuable than freshly written data; aggressively caching dirty data would significantly reduce throughput and responsiveness under high load). Given the hardware specs of the test system, I guess that Linux accepts at least 100 times the dirty data in the buffer cache, compared to FreeBSD (where this number is at most in the tens of megabyte range). If you did not, then your results do not represent a server load (which I'd expect relevant, if you are testing against Oracle Linux 6.1 server), where continuous performance is required. Tests that run on an idle system starting in a clean state and ignoring background flushing of the buffer cache after the timed program has stopped are perhaps useful for a very lowly loaded PC, but not for a system with high load average as the default. I bet that if you compared the systems under higher load (which admittedly makes it much harder to get sensible numbers for the program under test) or with reduced buffer cache size (or raise the dirty buffer limit in FreeBSD accordingly, which ought to be possible with sysctl and/or boot time tuneables, e.g. vfs.hidirtybuffers). And a last remark: Single benchmark runs do not provide reliable data. FreeBSD comes with ministat to check the significance of benchmark results. Each test should be repeated at least 5 times for meaningful averages with acceptable confidence level. The Phoronix Test Suite runs most tests a minimum of three times and if the standard deviation exceeds 3.5% the run count is dynamically increased, among other safeguards. -- Michael Regards, STefan ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
