Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server

[Denny Lin]

On Wed, Jan 04, 2012 at 03:52:06PM -0800, Matthew Tippett wrote:
> Hmm... No sure what happened there again.  What I sent (pulled from my 
> "Sent" folder...
> ===
> 
> Thanks for the comment Arnaud.   For comparative benchmarking on 
> Phoronix.com , Michael invariable leaves it in the 
> default configuration 'in the way the developers or vendor wanted it for 
> production'.  This is by rule.

A quick question: why is ZFS used in the benchmark?

"Both operating systems were in their stock configuration aside from
FreeBSD 9.0 using ZFS."

UFS is the default on FreeBSD, not ZFS. FreeBSD was not left in the
default configuration.

-- 
Denny Lin
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server

[Garrett Cooper]

On Wed, Jan 4, 2012 at 1:58 PM, Arnaud Lacombe  wrote:
> Hi,
>
> On Fri, Dec 16, 2011 at 12:16 PM,   wrote:
>> Thanks.
>>
>> My request for the person documenting the tunings also runs the benchmark to
>> ensure expected behaviour.
>>
> Why should you have to tune anything ? Did you tune the Oracle Server
> install ? If not, you should not have to tune the FreeBSD install,
> that wouldn't be fair. If you tune FreeBSD, you should tune the Oracle
> Server install too. It is pretty easy to win at least 30% in
> performance for certain workload by choosing the right kernel
> configuration.

This assumes that Oracle doesn't do secret sauce tuning... the Vanilla
CentOS/RHEL base is probably a better comparison than the Oracle
custom distro.
Thanks!
-Garrett
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server

[Alexander Kabaev]

On Wed, 04 Jan 2012 14:31:55 -0800
 wrote:

>Thanks for the comment Arnaud.   For comparative benchmarking
> on[1]Phoronix.com, Michael inva   configuration 'in the way the
> developers or   production'.  This is by rule. However, i   poor
> scores on be   'it should be tuned,   is configured for a diffe   The
> response from us to this comes in two forms. &nb   1) If it is the
> wrong workload for the platform, do a public pos   explaining and
> analysing the results.  Highlighting the rationale fo   r the
> concious reduction in performance (ie: journaling filesystems with
> ba   filesystem integrity   2) If tuning can have a material impact
> on the results, post a t   uning guide with step by step and
> rationale.  Ie: educate the communit   Michael and I have had many
> discussions with vendors an   on this.  In almost all cases, the
> vendor has either cha   default configuration or accepted the results
> as valid. Asguide, Micha   comparison.  To dat   offer.  In part,
> thi   public, but that is more of a result of a one sided d   party
> external to a particular community (with a healthy tou
> journalisticly pumped compare & contrast).  For the FreeBSD
> community, who else outside of the FreeBSD community actually runs
> public c   Matthew

Not really related to the discussion on hand, but the above about the
most unreadable email I am yet to read on the public mailing list.

-- 
Alexander Kabaev


signature.asc
Description: PGP signature

Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server

[Matthew Tippett]

Hmm... No sure what happened there again.  What I sent (pulled from my 
"Sent" folder...

===

Thanks for the comment Arnaud.   For comparative benchmarking on 
Phoronix.com , Michael invariable leaves it in the 
default configuration 'in the way the developers or vendor wanted it for 
production'.  This is by rule.


However, invariable the community or vendor for platforms that post poor 
scores on benchmark cry foul about using the default config.  'it should 
be tuned, no-one deploys an untuned system' or 'the system is configured 
for a different workload'.


The response from us to this comes in two forms.

1) If it is the wrong workload for the platform, do a public post 
explaining and analysing the results.  Highlighting the rationale for 
the concious reduction in performance (ie: journaling filesystems with 
barriers suffer in some write benchmarks for the sake of filesystem 
integrity.


2) If tuning can have a material impact on the results, post a tuning 
guide with step by step and rationale.  Ie: educate the community and users.


Michael and I have had many discussions with vendors and communities on 
this.  In almost all cases, the vendor has either changed the default 
configuration or accepted the results as valid.


As a service to the community or vendor that publishes the tuning guide, 
Michael is more than willing to redo a tuned vs untuned comparison.  To 
date, the communities have never taken us up on that offer.  In part, 
this affects Phoronix.com 's perception in the 
public, but that is more of a result of a one sided discussion by a 
party external to a particular community (with a healthy touch of 
journalisticly pumped compare & contrast).  For the FreeBSD community, 
who else outside of the FreeBSD community actually runs public 
comparisons of FreeBSD against anything?


Matthew
===

On 01/04/2012 03:49 PM, Alexander Kabaev wrote:

On Wed, 04 Jan 2012 14:31:55 -0800
  wrote:


Thanks for the comment Arnaud.   For comparative benchmarking
on[1]Phoronix.com, Michael inva   configuration 'in the way the
developers or   production'.  This is by rule. However, i   poor
scores on be   'it should be tuned,   is configured for a diffe   The
response from us to this comes in two forms.&nb   1) If it is the
wrong workload for the platform, do a public pos   explaining and
analysing the results.  Highlighting the rationale fo   r the
concious reduction in performance (ie: journaling filesystems with
ba   filesystem integrity   2) If tuning can have a material impact
on the results, post a t   uning guide with step by step and
rationale.  Ie: educate the communit   Michael and I have had many
discussions with vendors an   on this.  In almost all cases, the
vendor has either cha   default configuration or accepted the results
as valid. Asguide, Micha   comparison.  To dat   offer.  In part,
thi   public, but that is more of a result of a one sided d   party
external to a particular community (with a healthy tou
journalisticly pumped compare&  contrast).  For the FreeBSD
community, who else outside of the FreeBSD community actually runs
public c   Matthew

Not really related to the discussion on hand, but the above about the
most unreadable email I am yet to read on the public mailing list.



___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server

[matthew]

   Thanks for the comment Arnaud.   For comparative benchmarking on
[1]Phoronix.com, Michael inva= riable leaves it in the default
   configuration 'in the way the developers or= vendor wanted it for
   production'.  This is by rule.
   However, i= nvariable the community or vendor for platforms that post
   poor scores on be= nchmark cry foul about using the default config.
   'it should be tuned,= no-one deploys an untuned system' or 'the system
   is configured for a diffe= rent workload'.
   The response from us to this comes in two forms. &nb= sp;
   1) If it is the wrong workload for the platform, do a public pos= t
   explaining and analysing the results.  Highlighting the rationale fo   r the 
concious reduction in performance (ie: journaling filesystems
   with ba= rriers suffer in some write benchmarks for the sake of
   filesystem integrity= .
   2) If tuning can have a material impact on the results, post a t   uning 
guide with step by step and rationale.  Ie: educate the
   communit= y and users.
   Michael and I have had many discussions with vendors an= d communities
   on this.  In almost all cases, the vendor has either cha= nged the
   default configuration or accepted the results as valid.
   As = a service to the community or vendor that publishes the tuning
   guide, Micha= el is more than willing to redo a tuned vs untuned
   comparison.  To dat= e, the communities have never taken us up on that
   offer.  In part, thi= s affects [2]Phoronix.com's perception in the
   public, but that is more of a result of a one sided d= iscussion by a
   party external to a particular community (with a healthy tou= ch of
   journalisticly pumped compare & contrast).  For the FreeBSDcommunity, 
who else outside of the FreeBSD community actually runs
   public c= omparisons of FreeBSD against anything?
   Matthew

   -- Sent from my HP Pre3
 _

   On Jan 4, 2012 1:58 PM, Arnaud Lacombe  wrote:
   > Thanks.
   >
   &= gt; My request for the person documenting the tunings also runs the
   benchma= rk to
   > ensure expected behaviour.
   >
   Why should you= have to tune anything ? Did you tune the Oracle
   Server
   install ? If = not, you should not have to tune the FreeBSD
   install,
   that wouldn't b= e fair. If you tune FreeBSD, you should tune the
   Oracle
   Server instal= l too. It is pretty easy to win at least 30% in
   performance for certa= in workload by choosing the right kernel
   configuration.
   
   = - Arnaud
   
   > The installation, execution and comparison agai= nst the benchmarks
   in the
   > article is fairly simple.
   >= 
   > Note that some tuning may not be relevant or recommended (ie: s= ome
   of the fs
   > benchmarks are sensitive to barriers and other syn= chronous
   operations).  I'd
   > recommend bowing out of a benchm= ark with a 'we're going to be
   slower since
   > the default configura= tion is this way for the following reason' if
   this is
   > the case.= 
   >
   > Thanks 'someone'.
   >
   > Matthew
>
   >
   >  Dec 16, 2011 8:46 AM, Adrian Chadd 
   wrote:
   >
   > Can someone please write= up a nice, concise blog post somewhere
   > outlining all of this?= 
   >
   > Extra bonus points if it's a blog that is picked up = by
   > blogs.freebsdish.org and/or some of the other BSD sites.
>
   > Guys/girls/fuzzy things - this is 2011; people look at sh= iny
   blog
   > sites with graphs rather than mailing lists. Sorry, we = lost
   that
   > battle. :)
   >
   >
   >
   >= Adrian
   > ___
   &g= t; freebsd-performa...@freebsd.org mailing list
   > http://lists.fre= ebsd.org/mailman/listinfo/freebsd-performance
   > To unsubscribe, se= nd any mail to
   > "freebsd-performance-unsubscr...@freebsd.org"
<= br>

References

   1. 3D"http://Phoronix.com"/
   2. 3D"http://Phoronix.com"/
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server

[Arnaud Lacombe]

Hi,

On Fri, Dec 16, 2011 at 12:16 PM,   wrote:
> Thanks.
>
> My request for the person documenting the tunings also runs the benchmark to
> ensure expected behaviour.
>
Why should you have to tune anything ? Did you tune the Oracle Server
install ? If not, you should not have to tune the FreeBSD install,
that wouldn't be fair. If you tune FreeBSD, you should tune the Oracle
Server install too. It is pretty easy to win at least 30% in
performance for certain workload by choosing the right kernel
configuration.

 - Arnaud

> The installation, execution and comparison against the benchmarks in the
> article is fairly simple.
>
> Note that some tuning may not be relevant or recommended (ie: some of the fs
> benchmarks are sensitive to barriers and other synchronous operations).  I'd
> recommend bowing out of a benchmark with a 'we're going to be slower since
> the default configuration is this way for the following reason' if this is
> the case.
>
> Thanks 'someone'.
>
> Matthew
>
>
>  Dec 16, 2011 8:46 AM, Adrian Chadd  wrote:
>
> Can someone please write up a nice, concise blog post somewhere
> outlining all of this?
>
> Extra bonus points if it's a blog that is picked up by
> blogs.freebsdish.org and/or some of the other BSD sites.
>
> Guys/girls/fuzzy things - this is 2011; people look at shiny blog
> sites with graphs rather than mailing lists. Sorry, we lost that
> battle. :)
>
>
>
> Adrian
> ___
> freebsd-performa...@freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-performance
> To unsubscribe, send any mail to
> "freebsd-performance-unsubscr...@freebsd.org"
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server

[Vincent Hoffman]

On 23/12/2011 20:23, Garrett Cooper wrote:
> On Fri, Dec 23, 2011 at 2:38 AM, Vincent Hoffman  wrote:
>> On 23/12/2011 02:56, Garrett Cooper wrote:

>> There is a wiki page http://wiki.freebsd.org/SystemTuning which is
>> currently more or less tuning(7) with some annotations, the idea being
>> to sort out whats outdated/invalid with an aim of rewriting tuning(7) to
>> be more accurate and useful. I'll grab any info in your pr thats not up
>> there already to keep it updated if thats ok.
> Sure. Please take my suggestions (apart from the networking
> sysctls) with a grain of salt as I didn't look at the sourcecode for
> the filesystem ones (I was going off the top of my head and other
> emails I had seen passed around).
> I'll update the tuning 'wiki' with mention of the new networking
> defaults. If we want to make this manpage 'timeless', should we remove
> mention of defaults and go off basic guidelines (if you set this
> higher, you'll get better performance in scenario, X.Y.Z, etc)?
> Thanks!
> -Garrett

Good point, for tuning the defaults are probably not so important as
they are likely to change at some point (as the current man page will
attest) so maybe its less important to document them.

Happy Christmas  (or holiday of your choice ;) to you all and I hope
everyone has a good new year.


Vince
> ___
> freebsd-curr...@freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-current
> To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server

[Garrett Cooper]

On Fri, Dec 23, 2011 at 2:38 AM, Vincent Hoffman  wrote:
> On 23/12/2011 02:56, Garrett Cooper wrote:
>> On Dec 22, 2011, at 3:58 PM, Jeremy Chadwick  
>> wrote:
>>
>>> On Fri, Dec 23, 2011 at 12:44:14AM +0100, O. Hartmann wrote:
 On 12/21/11 19:41, Alexander Leidinger wrote:
> Hi,
>
> while the discussion continued here, some work started at some other 
> place. Now... in case someone here is willing to help instead of talking, 
> feel free to go to http://wiki.freebsd.org/BenchmarkAdvice and have a 
> look what can be improved. The page is far from perfect and needs some 
> additional people which are willing to improve it.
>
> This is only part of the problem. A tuning page in the wiki - which could 
> be referenced from the benchmark page - would be great too. Any 
> volunteers? A first step would be to take he tuning-man-page and wikify 
> it. Other tuning sources are welcome too.
>
> Every FreeBSD dev with a wiki account can hand out write access to the 
> wiki. The benchmark page gives contributor-access. If someone wants write 
> access create a FirstnameLastname account and ask here for 
> contributor-access.
>
> Don't worry if you think your english is not good enough, even some 
> one-word notes can help (and _my_ english got already corrected by other 
> people on the benchmark page).
>
> Bye,
> Alexander.
>
>
>
>
 Nice to see movement ;-)

 But there seems something unclear:

 man make.conf(5) says, that  MALLOC_PRODUCTION is a knob set in
 /etc/make.conf.
 The WiJi says, MALLOC_PRODUCTION is to be set in /etc/src.conf.

 What's right and what's wrong now?
>>> I can say with certainty that this value belongs in /etc/make.conf
>>> (on RELENG_8 and earlier at least).
>>>
>>> src/share/mk/bsd.own.mk has no framework for MK_MALLOC_PRODUCTION,
>>> so, this is definitely a make.conf variable.
>> Take the advice in tuning(7) with a grain of salt because a number of 
>> suggestions are really outdated. I know because I filed a PR last night 
>> after I saw how out of synch some of the defaults it claimed were with 
>> reality on 9.x+. And I know other suggestions in the manpage are dated as 
>> well ;/.
> There is a wiki page http://wiki.freebsd.org/SystemTuning which is
> currently more or less tuning(7) with some annotations, the idea being
> to sort out whats outdated/invalid with an aim of rewriting tuning(7) to
> be more accurate and useful. I'll grab any info in your pr thats not up
> there already to keep it updated if thats ok.

Sure. Please take my suggestions (apart from the networking
sysctls) with a grain of salt as I didn't look at the sourcecode for
the filesystem ones (I was going off the top of my head and other
emails I had seen passed around).
I'll update the tuning 'wiki' with mention of the new networking
defaults. If we want to make this manpage 'timeless', should we remove
mention of defaults and go off basic guidelines (if you set this
higher, you'll get better performance in scenario, X.Y.Z, etc)?
Thanks!
-Garrett
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server

[O. Hartmann]

On 12/23/11 16:24, Jeremy Chadwick wrote:
> On Fri, Dec 23, 2011 at 10:00:05AM -0500, John Baldwin wrote:
>> On Thursday, December 22, 2011 6:58:46 pm Jeremy Chadwick wrote:
>>> On Fri, Dec 23, 2011 at 12:44:14AM +0100, O. Hartmann wrote:
 On 12/21/11 19:41, Alexander Leidinger wrote:
> Hi,
>
> while the discussion continued here, some work started at some other 
>> place. Now... in case someone here is willing to help instead of talking, 
>> feel 
>> free to go to http://wiki.freebsd.org/BenchmarkAdvice and have a look what 
>> can 
>> be improved. The page is far from perfect and needs some additional people 
>> which are willing to improve it.
>
> This is only part of the problem. A tuning page in the wiki - which 
>> could be referenced from the benchmark page - would be great too. Any 
>> volunteers? A first step would be to take he tuning-man-page and wikify it. 
>> Other tuning sources are welcome too.
>
> Every FreeBSD dev with a wiki account can hand out write access to the 
>> wiki. The benchmark page gives contributor-access. If someone wants write 
>> access create a FirstnameLastname account and ask here for 
>> contributor-access.
>
> Don't worry if you think your english is not good enough, even some one-
>> word notes can help (and _my_ english got already corrected by other people 
>> on 
>> the benchmark page).
>
> Bye,
> Alexander.
>
>
>
>

 Nice to see movement ;-)

 But there seems something unclear:

 man make.conf(5) says, that  MALLOC_PRODUCTION is a knob set in
 /etc/make.conf.
 The WiJi says, MALLOC_PRODUCTION is to be set in /etc/src.conf.

 What's right and what's wrong now?
>>>
>>> I can say with certainty that this value belongs in /etc/make.conf
>>> (on RELENG_8 and earlier at least).
>>>
>>> src/share/mk/bsd.own.mk has no framework for MK_MALLOC_PRODUCTION,
>>> so, this is definitely a make.conf variable.
>>
>> Eh, normal make variables can go in src.conf as well.  They do not have
>> to be listed in bsd.own.mk.  World builds include /etc/src.conf whereas
>> every make invocation includes /etc/make.conf via sys.mk.  The only reason
>> to use /etc/src.conf is to have a place to put variables only affect
>> make buildworld / buildkernel but do not affect other make invocations.
> 
> I was always under the impression src.conf(5) variables had to be
> manually added to bsd.own.mk and similar bits (e.g.
> src/tools/build/options/WITH_xxx which is what's used to create the
> src.conf(5) man page), but upon your comment and manual investigation on
> my part, I found you're indeed right.  Taken from bsd.own.mk:
> 
> 107 .if !defined(_WITHOUT_SRCCONF)
> 108 SRCCONF?=   /etc/src.conf
> 109 .if exists(${SRCCONF})
> 110 .include "${SRCCONF}"
> 111 .endif
> 112 .endif
> 
> As long as third-party software doesn't depend on MALLOC_PRODUCTION for
> something (I don't know why something would, but who knows; maybe
> there's a third-party malloc implementation which might?), then putting
> it in src.conf would be fine (src/lib/libc/stdlib files reference it).
> 

Then the manpage should reflect this. man src.conf does not show up
MALLOC_PRODUCTIOn, but man make.conf does. If the latter is right, then
it should be worth mentioned that make.conf is incorporating src.conf.

Just a suggestion.

Regards,
Oliver



signature.asc
Description: OpenPGP digital signature

Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server

[Jeremy Chadwick]

On Fri, Dec 23, 2011 at 10:00:05AM -0500, John Baldwin wrote:
> On Thursday, December 22, 2011 6:58:46 pm Jeremy Chadwick wrote:
> > On Fri, Dec 23, 2011 at 12:44:14AM +0100, O. Hartmann wrote:
> > > On 12/21/11 19:41, Alexander Leidinger wrote:
> > > > Hi,
> > > > 
> > > > while the discussion continued here, some work started at some other 
> place. Now... in case someone here is willing to help instead of talking, 
> feel 
> free to go to http://wiki.freebsd.org/BenchmarkAdvice and have a look what 
> can 
> be improved. The page is far from perfect and needs some additional people 
> which are willing to improve it.
> > > > 
> > > > This is only part of the problem. A tuning page in the wiki - which 
> could be referenced from the benchmark page - would be great too. Any 
> volunteers? A first step would be to take he tuning-man-page and wikify it. 
> Other tuning sources are welcome too.
> > > > 
> > > > Every FreeBSD dev with a wiki account can hand out write access to the 
> wiki. The benchmark page gives contributor-access. If someone wants write 
> access create a FirstnameLastname account and ask here for contributor-access.
> > > > 
> > > > Don't worry if you think your english is not good enough, even some one-
> word notes can help (and _my_ english got already corrected by other people 
> on 
> the benchmark page).
> > > > 
> > > > Bye,
> > > > Alexander.
> > > > 
> > > > 
> > > > 
> > > > 
> > > 
> > > Nice to see movement ;-)
> > > 
> > > But there seems something unclear:
> > > 
> > > man make.conf(5) says, that  MALLOC_PRODUCTION is a knob set in
> > > /etc/make.conf.
> > > The WiJi says, MALLOC_PRODUCTION is to be set in /etc/src.conf.
> > > 
> > > What's right and what's wrong now?
> > 
> > I can say with certainty that this value belongs in /etc/make.conf
> > (on RELENG_8 and earlier at least).
> > 
> > src/share/mk/bsd.own.mk has no framework for MK_MALLOC_PRODUCTION,
> > so, this is definitely a make.conf variable.
> 
> Eh, normal make variables can go in src.conf as well.  They do not have
> to be listed in bsd.own.mk.  World builds include /etc/src.conf whereas
> every make invocation includes /etc/make.conf via sys.mk.  The only reason
> to use /etc/src.conf is to have a place to put variables only affect
> make buildworld / buildkernel but do not affect other make invocations.

I was always under the impression src.conf(5) variables had to be
manually added to bsd.own.mk and similar bits (e.g.
src/tools/build/options/WITH_xxx which is what's used to create the
src.conf(5) man page), but upon your comment and manual investigation on
my part, I found you're indeed right.  Taken from bsd.own.mk:

107 .if !defined(_WITHOUT_SRCCONF)
108 SRCCONF?=   /etc/src.conf
109 .if exists(${SRCCONF})
110 .include "${SRCCONF}"
111 .endif
112 .endif

As long as third-party software doesn't depend on MALLOC_PRODUCTION for
something (I don't know why something would, but who knows; maybe
there's a third-party malloc implementation which might?), then putting
it in src.conf would be fine (src/lib/libc/stdlib files reference it).

-- 
| Jeremy Chadwickjdc at parodius.com |
| Parodius Networking   http://www.parodius.com/ |
| UNIX Systems Administrator   Mountain View, CA, US |
| Making life hard for others since 1977.   PGP 4BD6C0CB |

___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server

[John Baldwin]

On Thursday, December 22, 2011 6:58:46 pm Jeremy Chadwick wrote:
> On Fri, Dec 23, 2011 at 12:44:14AM +0100, O. Hartmann wrote:
> > On 12/21/11 19:41, Alexander Leidinger wrote:
> > > Hi,
> > > 
> > > while the discussion continued here, some work started at some other 
place. Now... in case someone here is willing to help instead of talking, feel 
free to go to http://wiki.freebsd.org/BenchmarkAdvice and have a look what can 
be improved. The page is far from perfect and needs some additional people 
which are willing to improve it.
> > > 
> > > This is only part of the problem. A tuning page in the wiki - which 
could be referenced from the benchmark page - would be great too. Any 
volunteers? A first step would be to take he tuning-man-page and wikify it. 
Other tuning sources are welcome too.
> > > 
> > > Every FreeBSD dev with a wiki account can hand out write access to the 
wiki. The benchmark page gives contributor-access. If someone wants write 
access create a FirstnameLastname account and ask here for contributor-access.
> > > 
> > > Don't worry if you think your english is not good enough, even some one-
word notes can help (and _my_ english got already corrected by other people on 
the benchmark page).
> > > 
> > > Bye,
> > > Alexander.
> > > 
> > > 
> > > 
> > > 
> > 
> > Nice to see movement ;-)
> > 
> > But there seems something unclear:
> > 
> > man make.conf(5) says, that  MALLOC_PRODUCTION is a knob set in
> > /etc/make.conf.
> > The WiJi says, MALLOC_PRODUCTION is to be set in /etc/src.conf.
> > 
> > What's right and what's wrong now?
> 
> I can say with certainty that this value belongs in /etc/make.conf
> (on RELENG_8 and earlier at least).
> 
> src/share/mk/bsd.own.mk has no framework for MK_MALLOC_PRODUCTION,
> so, this is definitely a make.conf variable.

Eh, normal make variables can go in src.conf as well.  They do not have
to be listed in bsd.own.mk.  World builds include /etc/src.conf whereas
every make invocation includes /etc/make.conf via sys.mk.  The only reason
to use /etc/src.conf is to have a place to put variables only affect
make buildworld / buildkernel but do not affect other make invocations.

Also, MALLOC_PRODUCTION is generally enabled in a stable branch as part of
making the stable branch, there should be no need to set it manually in a 
stable branch.

-- 
John Baldwin
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server

[Vincent Hoffman]

On 23/12/2011 02:56, Garrett Cooper wrote:
> On Dec 22, 2011, at 3:58 PM, Jeremy Chadwick  wrote:
>
>> On Fri, Dec 23, 2011 at 12:44:14AM +0100, O. Hartmann wrote:
>>> On 12/21/11 19:41, Alexander Leidinger wrote:
 Hi,

 while the discussion continued here, some work started at some other 
 place. Now... in case someone here is willing to help instead of talking, 
 feel free to go to http://wiki.freebsd.org/BenchmarkAdvice and have a look 
 what can be improved. The page is far from perfect and needs some 
 additional people which are willing to improve it.

 This is only part of the problem. A tuning page in the wiki - which could 
 be referenced from the benchmark page - would be great too. Any 
 volunteers? A first step would be to take he tuning-man-page and wikify 
 it. Other tuning sources are welcome too.

 Every FreeBSD dev with a wiki account can hand out write access to the 
 wiki. The benchmark page gives contributor-access. If someone wants write 
 access create a FirstnameLastname account and ask here for 
 contributor-access.

 Don't worry if you think your english is not good enough, even some 
 one-word notes can help (and _my_ english got already corrected by other 
 people on the benchmark page).

 Bye,
 Alexander.




>>> Nice to see movement ;-)
>>>
>>> But there seems something unclear:
>>>
>>> man make.conf(5) says, that  MALLOC_PRODUCTION is a knob set in
>>> /etc/make.conf.
>>> The WiJi says, MALLOC_PRODUCTION is to be set in /etc/src.conf.
>>>
>>> What's right and what's wrong now?
>> I can say with certainty that this value belongs in /etc/make.conf
>> (on RELENG_8 and earlier at least).
>>
>> src/share/mk/bsd.own.mk has no framework for MK_MALLOC_PRODUCTION,
>> so, this is definitely a make.conf variable.
> Take the advice in tuning(7) with a grain of salt because a number of 
> suggestions are really outdated. I know because I filed a PR last night after 
> I saw how out of synch some of the defaults it claimed were with reality on 
> 9.x+. And I know other suggestions in the manpage are dated as well ;/.
There is a wiki page http://wiki.freebsd.org/SystemTuning which is
currently more or less tuning(7) with some annotations, the idea being
to sort out whats outdated/invalid with an aim of rewriting tuning(7) to
be more accurate and useful. I'll grab any info in your pr thats not up
there already to keep it updated if thats ok.

Vince

> Thanks,
> -Garrett___
> freebsd-curr...@freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-current
> To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server

[Garrett Cooper]

On Dec 22, 2011, at 3:58 PM, Jeremy Chadwick  wrote:

> On Fri, Dec 23, 2011 at 12:44:14AM +0100, O. Hartmann wrote:
>> On 12/21/11 19:41, Alexander Leidinger wrote:
>>> Hi,
>>> 
>>> while the discussion continued here, some work started at some other place. 
>>> Now... in case someone here is willing to help instead of talking, feel 
>>> free to go to http://wiki.freebsd.org/BenchmarkAdvice and have a look what 
>>> can be improved. The page is far from perfect and needs some additional 
>>> people which are willing to improve it.
>>> 
>>> This is only part of the problem. A tuning page in the wiki - which could 
>>> be referenced from the benchmark page - would be great too. Any volunteers? 
>>> A first step would be to take he tuning-man-page and wikify it. Other 
>>> tuning sources are welcome too.
>>> 
>>> Every FreeBSD dev with a wiki account can hand out write access to the 
>>> wiki. The benchmark page gives contributor-access. If someone wants write 
>>> access create a FirstnameLastname account and ask here for 
>>> contributor-access.
>>> 
>>> Don't worry if you think your english is not good enough, even some 
>>> one-word notes can help (and _my_ english got already corrected by other 
>>> people on the benchmark page).
>>> 
>>> Bye,
>>> Alexander.
>>> 
>>> 
>>> 
>>> 
>> 
>> Nice to see movement ;-)
>> 
>> But there seems something unclear:
>> 
>> man make.conf(5) says, that  MALLOC_PRODUCTION is a knob set in
>> /etc/make.conf.
>> The WiJi says, MALLOC_PRODUCTION is to be set in /etc/src.conf.
>> 
>> What's right and what's wrong now?
> 
> I can say with certainty that this value belongs in /etc/make.conf
> (on RELENG_8 and earlier at least).
> 
> src/share/mk/bsd.own.mk has no framework for MK_MALLOC_PRODUCTION,
> so, this is definitely a make.conf variable.

Take the advice in tuning(7) with a grain of salt because a number of 
suggestions are really outdated. I know because I filed a PR last night after I 
saw how out of synch some of the defaults it claimed were with reality on 9.x+. 
And I know other suggestions in the manpage are dated as well ;/.
Thanks,
-Garrett___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server

[Jeremy Chadwick]

On Fri, Dec 23, 2011 at 12:44:14AM +0100, O. Hartmann wrote:
> On 12/21/11 19:41, Alexander Leidinger wrote:
> > Hi,
> > 
> > while the discussion continued here, some work started at some other place. 
> > Now... in case someone here is willing to help instead of talking, feel 
> > free to go to http://wiki.freebsd.org/BenchmarkAdvice and have a look what 
> > can be improved. The page is far from perfect and needs some additional 
> > people which are willing to improve it.
> > 
> > This is only part of the problem. A tuning page in the wiki - which could 
> > be referenced from the benchmark page - would be great too. Any volunteers? 
> > A first step would be to take he tuning-man-page and wikify it. Other 
> > tuning sources are welcome too.
> > 
> > Every FreeBSD dev with a wiki account can hand out write access to the 
> > wiki. The benchmark page gives contributor-access. If someone wants write 
> > access create a FirstnameLastname account and ask here for 
> > contributor-access.
> > 
> > Don't worry if you think your english is not good enough, even some 
> > one-word notes can help (and _my_ english got already corrected by other 
> > people on the benchmark page).
> > 
> > Bye,
> > Alexander.
> > 
> > 
> > 
> > 
> 
> Nice to see movement ;-)
> 
> But there seems something unclear:
> 
> man make.conf(5) says, that  MALLOC_PRODUCTION is a knob set in
> /etc/make.conf.
> The WiJi says, MALLOC_PRODUCTION is to be set in /etc/src.conf.
> 
> What's right and what's wrong now?

I can say with certainty that this value belongs in /etc/make.conf
(on RELENG_8 and earlier at least).

src/share/mk/bsd.own.mk has no framework for MK_MALLOC_PRODUCTION,
so, this is definitely a make.conf variable.

-- 
| Jeremy Chadwickjdc at parodius.com |
| Parodius Networking   http://www.parodius.com/ |
| UNIX Systems Administrator   Mountain View, CA, US |
| Making life hard for others since 1977.   PGP 4BD6C0CB |

___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server

[O. Hartmann]

On 12/21/11 19:41, Alexander Leidinger wrote:
> Hi,
> 
> while the discussion continued here, some work started at some other place. 
> Now... in case someone here is willing to help instead of talking, feel free 
> to go to http://wiki.freebsd.org/BenchmarkAdvice and have a look what can be 
> improved. The page is far from perfect and needs some additional people which 
> are willing to improve it.
> 
> This is only part of the problem. A tuning page in the wiki - which could be 
> referenced from the benchmark page - would be great too. Any volunteers? A 
> first step would be to take he tuning-man-page and wikify it. Other tuning 
> sources are welcome too.
> 
> Every FreeBSD dev with a wiki account can hand out write access to the wiki. 
> The benchmark page gives contributor-access. If someone wants write access 
> create a FirstnameLastname account and ask here for contributor-access.
> 
> Don't worry if you think your english is not good enough, even some one-word 
> notes can help (and _my_ english got already corrected by other people on the 
> benchmark page).
> 
> Bye,
> Alexander.
> 
> 
> 
> 

Nice to see movement ;-)

But there seems something unclear:

man make.conf(5) says, that  MALLOC_PRODUCTION is a knob set in
/etc/make.conf.
The WiJi says, MALLOC_PRODUCTION is to be set in /etc/src.conf.

What's right and what's wrong now?

Oliver



signature.asc
Description: OpenPGP digital signature

Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server

[Alexander Leidinger]

Hi,

while the discussion continued here, some work started at some other place. 
Now... in case someone here is willing to help instead of talking, feel free to 
go to http://wiki.freebsd.org/BenchmarkAdvice and have a look what can be 
improved. The page is far from perfect and needs some additional people which 
are willing to improve it.

This is only part of the problem. A tuning page in the wiki - which could be 
referenced from the benchmark page - would be great too. Any volunteers? A 
first step would be to take he tuning-man-page and wikify it. Other tuning 
sources are welcome too.

Every FreeBSD dev with a wiki account can hand out write access to the wiki. 
The benchmark page gives contributor-access. If someone wants write access 
create a FirstnameLastname account and ask here for contributor-access.

Don't worry if you think your english is not good enough, even some one-word 
notes can help (and _my_ english got already corrected by other people on the 
benchmark page).

Bye,
Alexander.

-- 
Send via an Android device, please forgive brevity and typographic and spelling 
errors. "O. Hartmann"  hat geschrieben:On 12/20/11 
21:20, Igor Mozolevsky wrote:
> Interestingly, while people seem to be (arguably rightly) focused on
> criticising Phoronix's benchmarking, nobody has offered an alternative
> benchmark; and while (again, arguably rightly) it is important to
> benchmark real world performance, equally, nobody has offered any
> numbers in relation to, for example, HTTP or SMTP, or any other "real
> world"-application torture tests done on the aforementioned two
> platforms... IMO, this just goes to show that "doing is hard" and
> "criticising is much easier" (yes, I am aware of the irony involved in
> making this statement, but someone has to!)
> 
> 
> Cheers,
> Igor M :-)

Unfortunately, M. Larabel is the only one who's performing benchmarks on
FreeBSD, comparing its performance to the Linux-opponents. Adn indeed,
there is a lot of criticism, but no alternative.
I said unfortunately - not offensive - since Larabel and Phoronix are
sadly the only ones who do actually such bechmarking.

It would be much more nicer and kind to support those people.

Well, in January/February we get new hardware. One box is supposed to do
number crunching via 12 cores and a TESLA GPU. My colleague is
developing a high parallelized peice of software for satellite data
transformation. The software package is CPU bound, partially GPU, but
massively memory hungry (96 to 128 GB RAM is needed).
What I can offer is, since I will also work on that machine and I've
free hand to administer, in the spare time of doing my PhD, installing
FreeBSD 9.0/10.0 besides SuSe Linux and looking forward having one ZFS
data storage drive for homes, so both systems can perform on a most
recent ZFS. I'm new to Linux, not a BSD guru, nor I'm a professional
programmer/developer. My skills are sufficient for the daily scientific
work. So, without pressure, I'm willing to perform some HPC benchmarks
under advice if the day comes and those interested in bare numbers of
FreeBSD vs. Linux performance with a real-world-scientific application.

I would appreciate to see some of the developers and/or FreeBSD hackers
to help Phoronix setting up a proper testenvironment instead of bashing
M. Larabel and his fellows.

Regards,
Oliver

___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server

[Francois Tigeot]

On Tue, Dec 20, 2011 at 03:29:25PM -0800, Jeremy Chadwick wrote:
> 
> This also interested me:
> 
> * Linux system crashed
>   http://leaf.dragonflybsd.org/mailarchive/kernel/2011-11/msg8.html
> 
> * OpenIndiana system crashed same way as Linux system
>   http://leaf.dragonflybsd.org/mailarchive/kernel/2011-11/msg00017.html
> 
> I cannot help but wonder if the Linux and OpenIndiana installations were
> more stressful on the hardware -- getting more out of the system, maybe
> resulting in increased power/load, which in turn resulted in the systems
> locking up (shoddy PSU, unstable mainboard, MCH problems, etc.).
> 
> My point is that Francois states these things in such a way to imply
> that "DragonflyBSD was more stable",

Same thing can be said for FreeBSD, only Linux and OpenIndiana crashed
reliably if I remember correctly.

> when in fact I happen to wonder the
> opposite point -- that is to say, Linux and OpenIndiana were trying to
> use the hardware more-so than DragonflyBSD, thus tickled what may be a
> hardware-level problem.

I actually ran the benchmarks on two different machines with the same
hardware -- brand new Supermicro boxes with ECC memory and no cut corners.

Since then, I've found I could stop the Linux crashes by disabling some
options in the BIOS setup:
  - advanced ACPI settings (don't remember exactly which ones)
  - and a new WHEA one.

WHEA means Windows Hardware Error Architecture. For all I know, it may have
been the only culprit but I didn't have time to verify if the machines
also ran fine with only this option disabled.

-- 
Francois Tigeot
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server

[Stefan Esser]

Am 21.12.2011 06:22, schrieb Ian Smith:
> I find the results on this page very strange, but perhaps indicative:
> 
> http://www.phoronix.com/scan.php?page=article&item=debian_kfreebsd_h210&num=1
> 
> Here we see scant difference in results between Debian running FreeBSD 
> 7.3 or 8.0 or Linux 2.6.32 kernels, yet native FreeBSD 7.3 and 8.0 
> installations apparently run far slower, especially on the gzip test!

You did not expect this, since all user space programs were compiled
from identical sources, as were FreeBSD and kFreeBSD (probably with
minimal deviations in kFreeBSD, which should not affect the results)?

> Does this imply that given the similar kernel speed, Debian GNU userland 
> performs so dramatically better than FreeBSD userland?  Or does it 
> perhaps point to the default tuning of the FreeBSD systems compared to 
> (here) Debian, for these particular tests?  Indeed, `which gzip`?

Well, the answer is quite simple: Just run the Linux binaries on FreeBSD
or kFreeBSD (those compiled for testing Linux performance) and I'm
convinced that you'll find that performance significantly improves.

You did notice, that the 7-zip and gzip binaries were built with
gcc-4.4.4 for Linux and with gcc-4.2.1 for FreeBSD?

And another point: The relative advantage between FreeBSD and Linux is
different on R52 and T61. Might it be the case that gcc-4.4.4 has better
knowledge of the newer CPU in the latter (T61, Core 2 Duo) and optimizes
for it, not for the CPU in the R52 (Pentium-M) anymore?

And apparently 7-zip results are less affected by the compiler version
than the gzip results. This also hints at the compiler as the reason for
the better kFreeBSD and Linux results. (7-zip seems to be less dependent
on the better optimization of the newer gcc, or it does not take as much
advantage from it.)

Funny is the finding, that gzip is measured slower on FreeBSD 7.3 than
8.0 on the Pentium-M, while it is faster on 7.3 on the Core 2 Duo. That
does not match my expectations at all ...

There are no technical reasons, that FreeBSD does not come with a newer
GCC, as probably all in this list know. But OTOH, the newer GCC versions
can easily be installed from a port or package, and thus it would not
have been impossible to compare native binaries compiled with the same
compiler version for all test cases.

> And yes, FreeBSD could sure use some sort of tuning 'profiles' mechanism 
> to be able to preconfigure systems for at least several vastly different 
> types of workload.  Nate Lawson used to talk about this, then in respect 
> to simple 'laptop vs desktop' scenarios, but we've since seen volumes 
> written, mostly in lists but some wikis, parts of the Handbook, guides 
> for performance tuning etc, scarcely accessible to J. Random Installer.  
> A set of tunings for these Phoronix benchmarks might be a good start?

I doubt that tuning is responsible, because kFreeBSD performed better
(with the test programs compiled with gcc-4.4.4). The benchmark measured
just that, the better optimization of the newer gcc version.

Install the port (perhaps an even later gcc version, gcc-4.5 is said to
generate even better code than gcc-4.4) and make it the default compiler
for ports, if you want to take advantage of the more advanced compiler.
The FreeBSD ports system makes that very easy.

BTW: Why don't we build binary packages with a later version of gcc than
what is in the system. This should not cause any GPLv3 violation, and we
could have the userland built with the compiler giving best performance ...

Regards, Stefan
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server

[Michael Larabel]

   On 12/20/2011 11:22 PM, Ian Smith wrote:

[performance@ & current@ ccs trimmed, I'm not subscribed.  Feel free ..]

On Mon, 19 Dec 2011, Samuel J. Greear wrote:
 > 2011/12/19 Lev Serebryakov [1]:
 > > Hello, Samuel.
 > > You wrote 15 ÿÿ 2011 ÿÿ., 16:32:47:
 > >
 > >> Other benchmarks in the Phoronix suite and their representations are
 > >> similarly flawed, _ALL_ of these results should be ignored and no time
 > >> should be wasted by any FreeBSD committer further evaluating this
 > >> garbage. (Yes, I have been down this rabbit hole).

I downloaded the sources the other night, poked around a bit trying to
suss out the test environment and FreeBSD dependencies.  Gobs of PHP and
shell scripts for those with time on their hands, but I concentrated on
*BSD installation and such for a couple of hours.  Observations below.

 > >  Here is one problem: we have choice from three items:
 > >
 > > (1) Make FreeBSD looks good on benchmarks by "fixing" FreeBSD

Or use benchmarks and kernel tuning to suit, where FreeBSD can shine :)

 > > (2) Make FreeBSD looks good on benchmarks by "fixing" Phoronix
 > > (communication with them, convincing, that they benchamrks are unfare
 > > / meaningless, ets)

I've no idea whether GPLv3 really allows us to fix it ourselves, but the
general orientation is entirely Linux, with {free,net}bsd as 'distros',
so to speak.  No blame there, just so long as that emphasis is clear.

 > > (3) Lose [potential] userbase.
 > >
 > >  You know, that these benchmarks are bad. I know. But potential (and
 > >  even some current!) user doesn't. And it seems, that these benchmarks
 > >  become popular over Internet.
 > >
 > > --
 > > // Black Lion AKA Lev Serebryakov [2]

Self-selected, like a 'Standard & Poors' of the OS 'market'? :)  People
who choose OS by fan base have already made their choice, and were never
'ours' to lose.  Recall the Benchmark Battles between Windows and Linux?

 > Here is where you completely derail the train, let me paste again what
 > I said before.
 >
 > ...
 > Take the first test as an example, Blogbench read. This doesn't raise
 > any red flags, right? At least not until you realize that Blogbench
 > isn't a read test, it's a read/write test. So what they have done here
 > is run a read/write test and then thrown away the write results for
 > both platforms and reported only the read results. If you dig down
 > into the actual results,
 > [3]http://openbenchmarking.org/result/1112113-AR-ORACLELIN37 -- you will
 > see two Blogbench numbers, one for read and another for write. These
 > were both taken from the same Blogbench run, so FreeBSD optimizes
 > writes over reads, that's probably a good thing for your data but a
 > bad thing when someone totally misrepresents benchmark results.
 > ...
 >
 > FreeBSD actually does _BETTER_ (subjectively) in this test than the
 > Linux system when you look at what is really going on. FreeBSD is
 > favoring writes, which is _GOOD_. FreeBSD does not need to be fixed,
 > the benchmarks need to be fixed to represent reality rather than
 > throwing half of the results in the trash. To be quite frank, "fixing"
 > FreeBSD to look good on this benchmark will make it a worse real-world
 > OS. But you guys go ahead and foot-shoot over these ridiculous
 > benchmarks all you want.
 >
 > Sam

I think the notion that installing FreeBSD with no tuning at all for
particular types of work can give comparable results is flawed, when
optimising for widely varying types of workload is normally expected.
Noone expects a database, file or web server, probably headless, to be
configured anything like the same as, say, a scientific workstation or a
multimedia box or a high-performance router or ..

I've only installed Linux twice, Debian Etch and Lenny.  I soon gave up
trying to install Lenny sans X and Gnome.  I'm sure it can be done, by
fighting the line of least resistance.  My point is that out of the box,
basic configuration and (I suspect) tuning of FreeBSD and Linux systems
has quite a different emphasis, and likely expected workload/s.

One thing I'd like to see is even 'ps -auxww' listings of these setups
while actually running these tests.  Not only PHP and X but all sorts of
stuff gets installed and some are presumed to be running on top of the
benchmarks per se, NetBSD even having a jdk dependency; I was a little
unnerved to see the substantial list of packages to install seen in:

./pts-core/external-test-dependencies/xml/freebsd-packages.xml

presumably, where the listed executables are not found, installed by:

./pts-core/external-test-dependencies/scripts/install-freebsd-packages.sh:
#!/bin/sh
# FreeBSD package installation
echo "Please enter your root password below:" 1>&2
su root -c "PACKAGESITE=\[4]"ftp://ftp.freebsd.org/pub/FreeBSD/ports/i386/packa
ges-7-stable/Latest/\" pkg_add -r $*"
exit

   This file isn't even used at present in the Phoronix Test Suite but if
   there is a needed external dependency the user is promp

Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server

[Ian Smith]

[performance@ & current@ ccs trimmed, I'm not subscribed.  Feel free ..]

On Mon, 19 Dec 2011, Samuel J. Greear wrote:
 > 2011/12/19 Lev Serebryakov :
 > > Hello, Samuel.
 > > You wrote 15 ÿÿ 2011 ÿÿ., 16:32:47:
 > >
 > >> Other benchmarks in the Phoronix suite and their representations are
 > >> similarly flawed, _ALL_ of these results should be ignored and no time
 > >> should be wasted by any FreeBSD committer further evaluating this
 > >> garbage. (Yes, I have been down this rabbit hole).

I downloaded the sources the other night, poked around a bit trying to 
suss out the test environment and FreeBSD dependencies.  Gobs of PHP and 
shell scripts for those with time on their hands, but I concentrated on 
*BSD installation and such for a couple of hours.  Observations below.

 > >  Here is one problem: we have choice from three items:
 > >
 > > (1) Make FreeBSD looks good on benchmarks by "fixing" FreeBSD

Or use benchmarks and kernel tuning to suit, where FreeBSD can shine :)

 > > (2) Make FreeBSD looks good on benchmarks by "fixing" Phoronix
 > > (communication with them, convincing, that they benchamrks are unfare
 > > / meaningless, ets)

I've no idea whether GPLv3 really allows us to fix it ourselves, but the
general orientation is entirely Linux, with {free,net}bsd as 'distros', 
so to speak.  No blame there, just so long as that emphasis is clear.

 > > (3) Lose [potential] userbase.
 > >
 > >  You know, that these benchmarks are bad. I know. But potential (and
 > >  even some current!) user doesn't. And it seems, that these benchmarks
 > >  become popular over Internet.
 > >
 > > --
 > > // Black Lion AKA Lev Serebryakov 

Self-selected, like a 'Standard & Poors' of the OS 'market'? :)  People 
who choose OS by fan base have already made their choice, and were never 
'ours' to lose.  Recall the Benchmark Battles between Windows and Linux?

 > Here is where you completely derail the train, let me paste again what
 > I said before.
 > 
 > ...
 > Take the first test as an example, Blogbench read. This doesn't raise
 > any red flags, right? At least not until you realize that Blogbench
 > isn't a read test, it's a read/write test. So what they have done here
 > is run a read/write test and then thrown away the write results for
 > both platforms and reported only the read results. If you dig down
 > into the actual results,
 > http://openbenchmarking.org/result/1112113-AR-ORACLELIN37 -- you will
 > see two Blogbench numbers, one for read and another for write. These
 > were both taken from the same Blogbench run, so FreeBSD optimizes
 > writes over reads, that's probably a good thing for your data but a
 > bad thing when someone totally misrepresents benchmark results.
 > ...
 > 
 > FreeBSD actually does _BETTER_ (subjectively) in this test than the
 > Linux system when you look at what is really going on. FreeBSD is
 > favoring writes, which is _GOOD_. FreeBSD does not need to be fixed,
 > the benchmarks need to be fixed to represent reality rather than
 > throwing half of the results in the trash. To be quite frank, "fixing"
 > FreeBSD to look good on this benchmark will make it a worse real-world
 > OS. But you guys go ahead and foot-shoot over these ridiculous
 > benchmarks all you want.
 > 
 > Sam

I think the notion that installing FreeBSD with no tuning at all for 
particular types of work can give comparable results is flawed, when 
optimising for widely varying types of workload is normally expected.  
Noone expects a database, file or web server, probably headless, to be 
configured anything like the same as, say, a scientific workstation or a 
multimedia box or a high-performance router or ..

I've only installed Linux twice, Debian Etch and Lenny.  I soon gave up 
trying to install Lenny sans X and Gnome.  I'm sure it can be done, by 
fighting the line of least resistance.  My point is that out of the box, 
basic configuration and (I suspect) tuning of FreeBSD and Linux systems 
has quite a different emphasis, and likely expected workload/s.

One thing I'd like to see is even 'ps -auxww' listings of these setups 
while actually running these tests.  Not only PHP and X but all sorts of 
stuff gets installed and some are presumed to be running on top of the 
benchmarks per se, NetBSD even having a jdk dependency; I was a little 
unnerved to see the substantial list of packages to install seen in:

./pts-core/external-test-dependencies/xml/freebsd-packages.xml

presumably, where the listed executables are not found, installed by:

./pts-core/external-test-dependencies/scripts/install-freebsd-packages.sh:
#!/bin/sh
# FreeBSD package installation
echo "Please enter your root password below:" 1>&2
su root -c 
"PACKAGESITE=\"ftp://ftp.freebsd.org/pub/FreeBSD/ports/i386/packages-7-stable/Latest/\";
 pkg_add -r $*"
exit

Hmm.  Would the 'ordinary user' of this software be expected to notice 
and adjust PACKAGESITE for later versions?  I admit to not having read 
the substantial docs 

Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server

[Michael Larabel]

Any version is fine that's PTS 3.0 or newer in terms of being 
compatible, since the test profiles are versioned separately and 
automatically fetched to match the result file. However, I'd recommended 
the newest (PTS 3.6) as it contains the best FreeBSD support at present 
in terms of hardware/software information parsing (for the automated 
table), etc.


Michael

On 12/20/2011 07:29 PM, Adrian Chadd wrote:

Is there a specific version of the test suite that should be used, to
compare against the published results?


Adrian

On 20 December 2011 17:18, Matthew Tippett  wrote:

For such a system, the greatest immediate value would be to attempt to
reproduce the benchmarks in question.

Install PTS from www.phoronix-test-suite.com or freshports.org.

Run the benchmark against those used in the article

phoronix-test-suite benchmark 1112113-AR-ORACLELIN37

You will be asked to push the comparison up to openbenchmarking at the end.

Matthew


On 12/20/2011 01:39 PM, O. Hartmann wrote:

On 12/20/11 21:20, Igor Mozolevsky wrote:

Interestingly, while people seem to be (arguably rightly) focused on
criticising Phoronix's benchmarking, nobody has offered an alternative
benchmark; and while (again, arguably rightly) it is important to
benchmark real world performance, equally, nobody has offered any
numbers in relation to, for example, HTTP or SMTP, or any other "real
world"-application torture tests done on the aforementioned two
platforms... IMO, this just goes to show that "doing is hard" and
"criticising is much easier" (yes, I am aware of the irony involved in
making this statement, but someone has to!)


Cheers,
Igor M :-)

Unfortunately, M. Larabel is the only one who's performing benchmarks on
FreeBSD, comparing its performance to the Linux-opponents. Adn indeed,
there is a lot of criticism, but no alternative.
I said unfortunately - not offensive - since Larabel and Phoronix are
sadly the only ones who do actually such bechmarking.

It would be much more nicer and kind to support those people.

Well, in January/February we get new hardware. One box is supposed to do
number crunching via 12 cores and a TESLA GPU. My colleague is
developing a high parallelized peice of software for satellite data
transformation. The software package is CPU bound, partially GPU, but
massively memory hungry (96 to 128 GB RAM is needed).
What I can offer is, since I will also work on that machine and I've
free hand to administer, in the spare time of doing my PhD, installing
FreeBSD 9.0/10.0 besides SuSe Linux and looking forward having one ZFS
data storage drive for homes, so both systems can perform on a most
recent ZFS. I'm new to Linux, not a BSD guru, nor I'm a professional
programmer/developer. My skills are sufficient for the daily scientific
work. So, without pressure, I'm willing to perform some HPC benchmarks
under advice if the day comes and those interested in bare numbers of
FreeBSD vs. Linux performance with a real-world-scientific application.

I would appreciate to see some of the developers and/or FreeBSD hackers
to help Phoronix setting up a proper testenvironment instead of bashing
M. Larabel and his fellows.

Regards,
Oliver


___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"



___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server

[matthew]

   The benchmarks themselves are versioned.  So in general most of the
   av= ailable versions of PTS itself should be fine.  PTS can be
   considered = an execution shell that doesn't affect the benchmark
   itself.
   Note th= at you'll download a pile of the benchmarks, build and
   install them.  = Then you run about 49 individual steps.
   Matthew

   -- Sent from my HP Pre3
 _

   On Dec 20, 2011 5:30 PM, Adrian Chadd = ; wrote:
   Is there a specific version of the test suite that = should be used,
   to
   compare against the published results?
   

   Adrian
   
   On 20 December 2011 17:18, Matthew Tippett <= ;matt...@phoronix.com>
   wrote:
   > For such a system, the greatest= immediate value would be to attempt
   to
   > reproduce the benchmarks= in question.
   >
   > Install PTS from www.phoronix-test-suit= e.com or freshports.org.
   >
   > Run the benchmark against th= ose used in the article
   >
   >    phoronix-test-su= ite benchmark
   1112113-AR-ORACLELIN37
   >
   > You will be aske= d to push the comparison up to openbenchmarking at
   the end.
   >
> Matthew
   >
   >
   > On 12/20/2011 01:39 PM, O. = Hartmann wrote:
   >>
   >> On 12/20/11 21:20, Igor Mozol= evsky wrote:
   >>>
   >>> Interestingly, while peo= ple seem to be (arguably rightly)
   focused on
   >>> criticising= Phoronix's benchmarking, nobody has offered an
   alternative
   >>&= gt; benchmark; and while (again, arguably rightly) it is
   important to
   >>> benchmark real world performance, equally, nobody has offered   any
   >>> numbers in relation to, for example, HTTP or SMTP, = or any other
   "real
   >>> world"-application torture tests done= on the aforementioned
   two
   >>> platforms... IMO, this just g= oes to show that "doing is hard"
   and
   >>> "criticising is muc= h easier" (yes, I am aware of the irony
   involved in
   >>> maki= ng this statement, but someone has to!)
   >>>
   >>&g= t;
   >>> Cheers,
   >>> Igor M :-)
   >>= 
   >> Unfortunately, M. Larabel is the only one who's performingbenchmarks 
on
   >> FreeBSD, comparing its performance to the Linu= x-opponents. Adn
   indeed,
   >> there is a lot of criticism, but no= alternative.
   >> I said unfortunately - not offensive - since L= arabel and Phoronix
   are
   >> sadly the only ones who do actually = such bechmarking.
   >>
   >> It would be much more nicer= and kind to support those people.
   >>
   >> Well, in J= anuary/February we get new hardware. One box is
   supposed to do
   >&g= t; number crunching via 12 cores and a TESLA GPU. My colleague
   is
   >= ;> developing a high parallelized peice of software for satellite
   data= 
   >> transformation. The software package is CPU bound, partiall= y GPU,
   but
   >> massively memory hungry (96 to 128 GB RAM is need= ed).
   >> What I can offer is, since I will also work on that mac= hine and
   I've
   >> free hand to administer, in the spare time of = doing my PhD,
   installing
   >> FreeBSD 9.0/10.0 besides SuSe Linux= and looking forward having one
   ZFS
   >> data storage drive for h= omes, so both systems can perform on a
   most
   >> recent ZFS. I'm = new to Linux, not a BSD guru, nor I'm a
   professional
   >> program= mer/developer. My skills are sufficient for the daily
   scientific
   >= > work. So, without pressure, I'm willing to perform some HPC
   benchmarks= 
   >> under advice if the day comes and those interested in barenumbers of
   >> FreeBSD vs. Linux performance with a real-world-s= cientific
   application.
   >>
   >> I would appreciate to = see some of the developers and/or FreeBSD
   hackers
   >> to help Ph= oronix setting up a proper testenvironment instead of
   bashing
   >>= ; M. Larabel and his fellows.
   >>
   >> Regards,
   = >> Oliver
   >>
   >
   > __= _
   > freebsd-stable@freebsd.org mailing lis= t
   > http://lists.freebsd.org/mailman/listinfo/freebsd-stable
   > To unsubscribe, send any mail to
   "freebsd-stable-unsubscribe@freebsd.= org"
   ___
   freebsd-pe= rforma...@freebsd.org mailing list
   http://lists.freebsd.org/mailman/l= istinfo/freebsd-performance
   To unsubscribe, send any mail to "freebsd   
-performance-unsubscr...@freebsd.org"

___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server

[Adrian Chadd]

Is there a specific version of the test suite that should be used, to
compare against the published results?


Adrian

On 20 December 2011 17:18, Matthew Tippett  wrote:
> For such a system, the greatest immediate value would be to attempt to
> reproduce the benchmarks in question.
>
> Install PTS from www.phoronix-test-suite.com or freshports.org.
>
> Run the benchmark against those used in the article
>
>    phoronix-test-suite benchmark 1112113-AR-ORACLELIN37
>
> You will be asked to push the comparison up to openbenchmarking at the end.
>
> Matthew
>
>
> On 12/20/2011 01:39 PM, O. Hartmann wrote:
>>
>> On 12/20/11 21:20, Igor Mozolevsky wrote:
>>>
>>> Interestingly, while people seem to be (arguably rightly) focused on
>>> criticising Phoronix's benchmarking, nobody has offered an alternative
>>> benchmark; and while (again, arguably rightly) it is important to
>>> benchmark real world performance, equally, nobody has offered any
>>> numbers in relation to, for example, HTTP or SMTP, or any other "real
>>> world"-application torture tests done on the aforementioned two
>>> platforms... IMO, this just goes to show that "doing is hard" and
>>> "criticising is much easier" (yes, I am aware of the irony involved in
>>> making this statement, but someone has to!)
>>>
>>>
>>> Cheers,
>>> Igor M :-)
>>
>> Unfortunately, M. Larabel is the only one who's performing benchmarks on
>> FreeBSD, comparing its performance to the Linux-opponents. Adn indeed,
>> there is a lot of criticism, but no alternative.
>> I said unfortunately - not offensive - since Larabel and Phoronix are
>> sadly the only ones who do actually such bechmarking.
>>
>> It would be much more nicer and kind to support those people.
>>
>> Well, in January/February we get new hardware. One box is supposed to do
>> number crunching via 12 cores and a TESLA GPU. My colleague is
>> developing a high parallelized peice of software for satellite data
>> transformation. The software package is CPU bound, partially GPU, but
>> massively memory hungry (96 to 128 GB RAM is needed).
>> What I can offer is, since I will also work on that machine and I've
>> free hand to administer, in the spare time of doing my PhD, installing
>> FreeBSD 9.0/10.0 besides SuSe Linux and looking forward having one ZFS
>> data storage drive for homes, so both systems can perform on a most
>> recent ZFS. I'm new to Linux, not a BSD guru, nor I'm a professional
>> programmer/developer. My skills are sufficient for the daily scientific
>> work. So, without pressure, I'm willing to perform some HPC benchmarks
>> under advice if the day comes and those interested in bare numbers of
>> FreeBSD vs. Linux performance with a real-world-scientific application.
>>
>> I would appreciate to see some of the developers and/or FreeBSD hackers
>> to help Phoronix setting up a proper testenvironment instead of bashing
>> M. Larabel and his fellows.
>>
>> Regards,
>> Oliver
>>
>
> ___
> freebsd-stable@freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-stable
> To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server

[Matthew Tippett]

For such a system, the greatest immediate value would be to attempt to 
reproduce the benchmarks in question.


Install PTS from www.phoronix-test-suite.com or freshports.org.

Run the benchmark against those used in the article

phoronix-test-suite benchmark 1112113-AR-ORACLELIN37

You will be asked to push the comparison up to openbenchmarking at the end.

Matthew

On 12/20/2011 01:39 PM, O. Hartmann wrote:

On 12/20/11 21:20, Igor Mozolevsky wrote:

Interestingly, while people seem to be (arguably rightly) focused on
criticising Phoronix's benchmarking, nobody has offered an alternative
benchmark; and while (again, arguably rightly) it is important to
benchmark real world performance, equally, nobody has offered any
numbers in relation to, for example, HTTP or SMTP, or any other "real
world"-application torture tests done on the aforementioned two
platforms... IMO, this just goes to show that "doing is hard" and
"criticising is much easier" (yes, I am aware of the irony involved in
making this statement, but someone has to!)


Cheers,
Igor M :-)

Unfortunately, M. Larabel is the only one who's performing benchmarks on
FreeBSD, comparing its performance to the Linux-opponents. Adn indeed,
there is a lot of criticism, but no alternative.
I said unfortunately - not offensive - since Larabel and Phoronix are
sadly the only ones who do actually such bechmarking.

It would be much more nicer and kind to support those people.

Well, in January/February we get new hardware. One box is supposed to do
number crunching via 12 cores and a TESLA GPU. My colleague is
developing a high parallelized peice of software for satellite data
transformation. The software package is CPU bound, partially GPU, but
massively memory hungry (96 to 128 GB RAM is needed).
What I can offer is, since I will also work on that machine and I've
free hand to administer, in the spare time of doing my PhD, installing
FreeBSD 9.0/10.0 besides SuSe Linux and looking forward having one ZFS
data storage drive for homes, so both systems can perform on a most
recent ZFS. I'm new to Linux, not a BSD guru, nor I'm a professional
programmer/developer. My skills are sufficient for the daily scientific
work. So, without pressure, I'm willing to perform some HPC benchmarks
under advice if the day comes and those interested in bare numbers of
FreeBSD vs. Linux performance with a real-world-scientific application.

I would appreciate to see some of the developers and/or FreeBSD hackers
to help Phoronix setting up a proper testenvironment instead of bashing
M. Larabel and his fellows.

Regards,
Oliver



___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server

[O. Hartmann]

On 12/21/11 00:29, Jeremy Chadwick wrote:
> On Tue, Dec 20, 2011 at 11:54:23PM +0100, O. Hartmann wrote:
>> On 12/20/11 22:45, Samuel J. Greear wrote:
>>> http://www.osnews.com/story/25334/DragonFly_BSD_MP_Performance_Significantly_Improved
>>>
>>> PostgreSQL tests, see the linked PDF for #'s on FreeBSD, DragonFly, Linux
>>> and Solaris. Steps to reproduce these benchmarks provided.
>>>
>>> Sam
>>>
>>> On Tue, Dec 20, 2011 at 1:20 PM, Igor Mozolevsky 
>>> wrote:
>>>
 Interestingly, while people seem to be (arguably rightly) focused on
 criticising Phoronix's benchmarking, nobody has offered an alternative
 benchmark; and while (again, arguably rightly) it is important to
 benchmark real world performance, equally, nobody has offered any
 numbers in relation to, for example, HTTP or SMTP, or any other "real
 world"-application torture tests done on the aforementioned two
 platforms... IMO, this just goes to show that "doing is hard" and
 "criticising is much easier" (yes, I am aware of the irony involved in
 making this statement, but someone has to!)


 Cheers,
 Igor M :-)
 ___
 freebsd-curr...@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/freebsd-current
 To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

>>
>> Thanks for those numbers.
>> Impressive how Matthew Dillon's project jumps forward now. And it is
>> still impressive to see that the picture is still in the right place
>> when it comes to a comparison to Linux.
>> Also, OpenIndiana shows an impressive performance.
> 
> Preface to my long post below:
> 
> The things being discussed here are benchmarks, as in "how much work
> can you get out of Thing".  This is VERY DIFFERENT from testing
> interactivity in a scheduler, which is more of a test that says "when
> Thing X is executed while heavier-Thing Y is also being executed, how
> much interaction is lost in Thing X".
> 
> The reason people notice this when using Xorg is because it's visual,
> in an environment where responsiveness is absolutely mandatory above all
> else.  Nobody is going to put up with a system where during a buildworld
> they go to move a window or click a mouse button or type a key and find
> that the window doesn't move, the mouse click is lost, or the key typed
> has gone into the bit bucket -- or, that those things are SEVERELY
> delayed, to the point where interactivity is crap.

I whitnessed sticky, jumpy and non-responsive-for seconds FreeBSD
servers (serving homes, NFS/SAMBA and PostgreSQL database (small)).
Those "seconds" where enough to cut a ssh line. Not funny. Network
traffic droped significantly. X/Desktop makes the problem visible,
indeed. But not seeing it does not mean it isn't there.
This might be the reason why FreeBSD is so much behind when it comes to X?

> 
> I just want to make that clear to folks.  This immense thread has been
> with regards to the latter -- bad interactivity/responsiveness on a
> system which was undergoing load that SHOULD be distributed "more
> evenly" across the system *while* keeping interactivity/responsiveness
> high.  Historically nice/renice has been used for this task, but that
> was when kernels were a little less complex and I/O subsystems were less
> complex.  Remember: we've now got schedulers for each type of thing,
> and who gets what priority?  You get my point I'm sure.
> 
> So remember: this was to discuss that aspect, with regards to ULE vs.
> 4BSD schedulers.
> 
> Now, back to the benchmarks:
> 
> This also interested me:
> 
> * Linux system crashed
>   http://leaf.dragonflybsd.org/mailarchive/kernel/2011-11/msg8.html
> 
> * OpenIndiana system crashed same way as Linux system
>   http://leaf.dragonflybsd.org/mailarchive/kernel/2011-11/msg00017.html
> 
> I cannot help but wonder if the Linux and OpenIndiana installations were
> more stressful on the hardware -- getting more out of the system, maybe
> resulting in increased power/load, which in turn resulted in the systems
> locking up (shoddy PSU, unstable mainboard, MCH problems, etc.).

Is FreeBSD supposed to run on dumpyard equipment? In former times,
freeBSD was used on high value hardware, not the decomissioned crap with
shoddy PSUs or whatsoever.
If I need a server, I care about quality hardware as I do for my lab's
box and my own box at home. I expect a "server garde" hardware to act
like that and I expect the operating system to get the maximum out of
that hardware. Otherwise it is not worth one shot.

> 
> My point is that Francois states these things in such a way to imply
> that "DragonflyBSD was more stable", when in fact I happen to wonder the
> opposite point -- that is to say, Linux and OpenIndiana were trying to
> use the hardware more-so than DragonflyBSD, thus tickled what may be a
> hardware-level problem.
> 
>> But this is only one suite of testing. Scientific Linux is supposed to
>>

Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server

[Matthew Tippett]

Bottom post this time to follow Oliver :).

On 12/20/2011 02:54 PM, O. Hartmann wrote:

On 12/20/11 22:45, Samuel J. Greear wrote:

http://www.osnews.com/story/25334/DragonFly_BSD_MP_Performance_Significantly_Improved

PostgreSQL tests, see the linked PDF for #'s on FreeBSD, DragonFly, Linux
and Solaris. Steps to reproduce these benchmarks provided.

Sam
There are still possible issues with those benchmarks.  The Xeon has 
known problems scaling from 6 to 12 cores (well enabling the 
hyperthreading), so you may find that some platforms are penalized in 
performance if HT is turned on.  See the scaling that Phoronix has done in


http://openbenchmarking.org/result/1112166-AR-1112153AR03

Most systems are good with scaling on real cores, the hyperthreading 
(and for that matter the Bulldozer thread affinity) can really break 
performance.   Different platforms have different behaviours.  
Benchmarking is a mucky business..


Note that the benchmarks with Phoronix test suite are repeatable, once 
installed, you can just run "./phoronix-test-suite benchmark 
1112113-AR-ORACLELIN37" to repeat (as close as the system allows) the 
benchmarks that started this thread.

Is the postgresql benchmark the only way to benchmark?
pgbench is already included in the Phoronix Test Suite (at least 9.0.1 
TPC-B benchmark.




Well, this inspires me to gather together all the benchmarks someone
could find. There were lots of compalins about FreeBSD's poor
performance with BIND - once a domain of FreeBSD. Network performance
seems also to be an issue if it comes to scalability.
It would be nice to see what portion of the raw CPU/GPU power the OS
(FreeBSD, Linux ...)  delivers to scientific applications.

I only know some kind of benchmarks, BYTE UNIX benchmark, LINPACK test
... Does someone know a site to look for a couple of benchmarks to test

a) memory system
b) scalability (apart from pgbench)
c) network performance/throughput/network scalability
d) portion of CPU performance the system delivers for numerical
applications to the user apart from the system's own consumption
e) disk I/O performance and scalability
The majority of these benchmarks are already in Phoronix Test Suite.  
There is monitoring capability (temp, load, CPU states, etc).  The 
question is the mapping from system attribute to benchmark, as well as 
determine what the ambigious terms mean (scaling can mean on increasing 
workloads, as memory is increased, as cpus are increased).




it would also be nice to discuss some nice settings and performance
tunings for FreeBSD for several scenarios. I guess, starting developing
benchmarking test scenarios for several purposes would lead faster to
real numbers and non polemic than weird discussions ...

This is what Michael and I are wanting to see.  Adrian Chadd has 
offerered to help facilitate within the FreeBSD community.  As mentioned 
before, what I'd like to see is


1) Recommendations for more rounded benchmarks from the FreeBSD 
perspective

2) Tuning guide documented somewhere within the community
3) Comparative results based on the communities testing.

All concrete, and all achievable.

Regards,

Matthew
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server

[Jeremy Chadwick]

On Tue, Dec 20, 2011 at 11:54:23PM +0100, O. Hartmann wrote:
> On 12/20/11 22:45, Samuel J. Greear wrote:
> > http://www.osnews.com/story/25334/DragonFly_BSD_MP_Performance_Significantly_Improved
> > 
> > PostgreSQL tests, see the linked PDF for #'s on FreeBSD, DragonFly, Linux
> > and Solaris. Steps to reproduce these benchmarks provided.
> > 
> > Sam
> > 
> > On Tue, Dec 20, 2011 at 1:20 PM, Igor Mozolevsky 
> > wrote:
> > 
> >> Interestingly, while people seem to be (arguably rightly) focused on
> >> criticising Phoronix's benchmarking, nobody has offered an alternative
> >> benchmark; and while (again, arguably rightly) it is important to
> >> benchmark real world performance, equally, nobody has offered any
> >> numbers in relation to, for example, HTTP or SMTP, or any other "real
> >> world"-application torture tests done on the aforementioned two
> >> platforms... IMO, this just goes to show that "doing is hard" and
> >> "criticising is much easier" (yes, I am aware of the irony involved in
> >> making this statement, but someone has to!)
> >>
> >>
> >> Cheers,
> >> Igor M :-)
> >> ___
> >> freebsd-curr...@freebsd.org mailing list
> >> http://lists.freebsd.org/mailman/listinfo/freebsd-current
> >> To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
> >>
> 
> Thanks for those numbers.
> Impressive how Matthew Dillon's project jumps forward now. And it is
> still impressive to see that the picture is still in the right place
> when it comes to a comparison to Linux.
> Also, OpenIndiana shows an impressive performance.

Preface to my long post below:

The things being discussed here are benchmarks, as in "how much work
can you get out of Thing".  This is VERY DIFFERENT from testing
interactivity in a scheduler, which is more of a test that says "when
Thing X is executed while heavier-Thing Y is also being executed, how
much interaction is lost in Thing X".

The reason people notice this when using Xorg is because it's visual,
in an environment where responsiveness is absolutely mandatory above all
else.  Nobody is going to put up with a system where during a buildworld
they go to move a window or click a mouse button or type a key and find
that the window doesn't move, the mouse click is lost, or the key typed
has gone into the bit bucket -- or, that those things are SEVERELY
delayed, to the point where interactivity is crap.

I just want to make that clear to folks.  This immense thread has been
with regards to the latter -- bad interactivity/responsiveness on a
system which was undergoing load that SHOULD be distributed "more
evenly" across the system *while* keeping interactivity/responsiveness
high.  Historically nice/renice has been used for this task, but that
was when kernels were a little less complex and I/O subsystems were less
complex.  Remember: we've now got schedulers for each type of thing,
and who gets what priority?  You get my point I'm sure.

So remember: this was to discuss that aspect, with regards to ULE vs.
4BSD schedulers.

Now, back to the benchmarks:

This also interested me:

* Linux system crashed
  http://leaf.dragonflybsd.org/mailarchive/kernel/2011-11/msg8.html

* OpenIndiana system crashed same way as Linux system
  http://leaf.dragonflybsd.org/mailarchive/kernel/2011-11/msg00017.html

I cannot help but wonder if the Linux and OpenIndiana installations were
more stressful on the hardware -- getting more out of the system, maybe
resulting in increased power/load, which in turn resulted in the systems
locking up (shoddy PSU, unstable mainboard, MCH problems, etc.).

My point is that Francois states these things in such a way to imply
that "DragonflyBSD was more stable", when in fact I happen to wonder the
opposite point -- that is to say, Linux and OpenIndiana were trying to
use the hardware more-so than DragonflyBSD, thus tickled what may be a
hardware-level problem.

> But this is only one suite of testing. Scientific Linux is supposed to
> give the best performance for scientifi purposes, i.e. for longhaul
> calculations, much numerical stuff. It outperforms in a typical server
> application FreeBSd, were "FreeBSD shoulkd have the power to serve".
> 
> Is the postgresql benchmark the only way to benchmark?

I sure hope not.  But you know what's equally as interesting?  This:

http://people.freebsd.org/~kris/scaling/

Specifically circa 2008:

http://people.freebsd.org/~kris/scaling/4cpu-pgsql.png
http://people.freebsd.org/~kris/scaling/pgsql-16cpu-2.png
http://people.freebsd.org/~kris/scaling/pgsql-16cpu.png

Now, I don't know if what was used in those ("pgsql sysbench") was the
same thing as "pg_bench" in the DragonflyBSD tests, but if so, the
numbers are different to a point that is preposterous.

There's also this:

http://people.freebsd.org/~kris/scaling/pgsql-ncpu.png

Now, compare those numbers to the TPS numbers shown here:

http://dl.wolfpond.org/Pg-benchmarks.pdf

So um... yeah.  Now, if som

Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server

[O. Hartmann]

On 12/20/11 22:45, Samuel J. Greear wrote:
> http://www.osnews.com/story/25334/DragonFly_BSD_MP_Performance_Significantly_Improved
> 
> PostgreSQL tests, see the linked PDF for #'s on FreeBSD, DragonFly, Linux
> and Solaris. Steps to reproduce these benchmarks provided.
> 
> Sam
> 
> On Tue, Dec 20, 2011 at 1:20 PM, Igor Mozolevsky wrote:
> 
>> Interestingly, while people seem to be (arguably rightly) focused on
>> criticising Phoronix's benchmarking, nobody has offered an alternative
>> benchmark; and while (again, arguably rightly) it is important to
>> benchmark real world performance, equally, nobody has offered any
>> numbers in relation to, for example, HTTP or SMTP, or any other "real
>> world"-application torture tests done on the aforementioned two
>> platforms... IMO, this just goes to show that "doing is hard" and
>> "criticising is much easier" (yes, I am aware of the irony involved in
>> making this statement, but someone has to!)
>>
>>
>> Cheers,
>> Igor M :-)
>> ___
>> freebsd-curr...@freebsd.org mailing list
>> http://lists.freebsd.org/mailman/listinfo/freebsd-current
>> To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
>>

Thanks for those numbers.
Impressive how Matthew Dillon's project jumps forward now. And it is
still impressive to see that the picture is still in the right place
when it comes to a comparison to Linux.
Also, OpenIndiana shows an impressive performance.
But this is only one suite of testing. Scientific Linux is supposed to
give the best performance for scientifi purposes, i.e. for longhaul
calculations, much numerical stuff. It outperforms in a typical server
application FreeBSd, were "FreeBSD shoulkd have the power to serve".

Is the postgresql benchmark the only way to benchmark?

Well, this inspires me to gather together all the benchmarks someone
could find. There were lots of compalins about FreeBSD's poor
performance with BIND - once a domain of FreeBSD. Network performance
seems also to be an issue if it comes to scalability.
It would be nice to see what portion of the raw CPU/GPU power the OS
(FreeBSD, Linux ...)  delivers to scientific applications.

I only know some kind of benchmarks, BYTE UNIX benchmark, LINPACK test
... Does someone know a site to look for a couple of benchmarks to test

a) memory system
b) scalability (apart from pgbench)
c) network performance/throughput/network scalability
d) portion of CPU performance the system delivers for numerical
applications to the user apart from the system's own consumption
e) disk I/O performance and scalability

it would also be nice to discuss some nice settings and performance
tunings for FreeBSD for several scenarios. I guess, starting developing
benchmarking test scenarios for several purposes would lead faster to
real numbers and non polemic than weird discussions ...



signature.asc
Description: OpenPGP digital signature

Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server

[Samuel J. Greear]

http://www.osnews.com/story/25334/DragonFly_BSD_MP_Performance_Significantly_Improved

PostgreSQL tests, see the linked PDF for #'s on FreeBSD, DragonFly, Linux
and Solaris. Steps to reproduce these benchmarks provided.

Sam

On Tue, Dec 20, 2011 at 1:20 PM, Igor Mozolevsky wrote:

> Interestingly, while people seem to be (arguably rightly) focused on
> criticising Phoronix's benchmarking, nobody has offered an alternative
> benchmark; and while (again, arguably rightly) it is important to
> benchmark real world performance, equally, nobody has offered any
> numbers in relation to, for example, HTTP or SMTP, or any other "real
> world"-application torture tests done on the aforementioned two
> platforms... IMO, this just goes to show that "doing is hard" and
> "criticising is much easier" (yes, I am aware of the irony involved in
> making this statement, but someone has to!)
>
>
> Cheers,
> Igor M :-)
> ___
> freebsd-curr...@freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-current
> To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
>
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server

[O. Hartmann]

On 12/20/11 21:20, Igor Mozolevsky wrote:
> Interestingly, while people seem to be (arguably rightly) focused on
> criticising Phoronix's benchmarking, nobody has offered an alternative
> benchmark; and while (again, arguably rightly) it is important to
> benchmark real world performance, equally, nobody has offered any
> numbers in relation to, for example, HTTP or SMTP, or any other "real
> world"-application torture tests done on the aforementioned two
> platforms... IMO, this just goes to show that "doing is hard" and
> "criticising is much easier" (yes, I am aware of the irony involved in
> making this statement, but someone has to!)
> 
> 
> Cheers,
> Igor M :-)

Unfortunately, M. Larabel is the only one who's performing benchmarks on
FreeBSD, comparing its performance to the Linux-opponents. Adn indeed,
there is a lot of criticism, but no alternative.
I said unfortunately - not offensive - since Larabel and Phoronix are
sadly the only ones who do actually such bechmarking.

It would be much more nicer and kind to support those people.

Well, in January/February we get new hardware. One box is supposed to do
number crunching via 12 cores and a TESLA GPU. My colleague is
developing a high parallelized peice of software for satellite data
transformation. The software package is CPU bound, partially GPU, but
massively memory hungry (96 to 128 GB RAM is needed).
What I can offer is, since I will also work on that machine and I've
free hand to administer, in the spare time of doing my PhD, installing
FreeBSD 9.0/10.0 besides SuSe Linux and looking forward having one ZFS
data storage drive for homes, so both systems can perform on a most
recent ZFS. I'm new to Linux, not a BSD guru, nor I'm a professional
programmer/developer. My skills are sufficient for the daily scientific
work. So, without pressure, I'm willing to perform some HPC benchmarks
under advice if the day comes and those interested in bare numbers of
FreeBSD vs. Linux performance with a real-world-scientific application.

I would appreciate to see some of the developers and/or FreeBSD hackers
to help Phoronix setting up a proper testenvironment instead of bashing
M. Larabel and his fellows.

Regards,
Oliver



signature.asc
Description: OpenPGP digital signature

Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server

[Igor Mozolevsky]

Interestingly, while people seem to be (arguably rightly) focused on
criticising Phoronix's benchmarking, nobody has offered an alternative
benchmark; and while (again, arguably rightly) it is important to
benchmark real world performance, equally, nobody has offered any
numbers in relation to, for example, HTTP or SMTP, or any other "real
world"-application torture tests done on the aforementioned two
platforms... IMO, this just goes to show that "doing is hard" and
"criticising is much easier" (yes, I am aware of the irony involved in
making this statement, but someone has to!)


Cheers,
Igor M :-)
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server

[Garrett Cooper]

On Dec 20, 2011, at 1:01 AM, Christer Solskogen wrote:

> On Mon, Dec 19, 2011 at 2:16 PM, Alexander Yerenkow  
> wrote:
>> FreeBSD currently have very obscure, closed community. To get in touch, you
>> need to subscribe to several mail lists, constantly read them, I've just
>> found recently (my shame of course) in mail list that there is service (
>> pub.allbsd.org) which constantly building current versions. This is great,
>> but at homepage of freebsd.org there is no word about it :)
> 
> That's because it's not official. Do you take the risk? Would a
> multi-milion-dollar company do that?
> For your private server, sure it's probably fine. But how do you know
> that those files are not contaminated?
> (That being said, the purpose of that service is good. And the files
> there a most probably 100% fine. But if it's not official... then..)

As long as I have reliable checksums that match the what the upstream source 
says is the real thing, it doesn't practically matter where I get my images 
from.
-Garrett___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server

[Christer Solskogen]

On Tue, Dec 20, 2011 at 10:55 AM, Garrett Cooper  wrote:
>        Release engineering for FreeBSD produces SHA256 checksums for all 
> official releases. AFAIK though they're only in the announcement emails and 
> not stored anywhere else.
>        I can't speak for OpenBSD's release process.
> Thanks,


So why do you want to download from a non-official site then? What do
you gain with that?


-- 
chs,
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server

[Garrett Cooper]

On Dec 20, 2011, at 1:51 AM, Christer Solskogen wrote:

> On Tue, Dec 20, 2011 at 10:42 AM, Garrett Cooper  wrote:
>> 
>> As long as I have reliable checksums that match the what the upstream source 
>> says is the real thing, it doesn't practically matter where I get my images 
>> from.
> 
> Checksums compared to what? How would you know what the correct
> checksums for OpenBSD-current is, if it's not built by Theo?

Release engineering for FreeBSD produces SHA256 checksums for all 
official releases. AFAIK though they're only in the announcement emails and not 
stored anywhere else.
I can't speak for OpenBSD's release process.
Thanks,
-Garrett___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server

[Christer Solskogen]

On Tue, Dec 20, 2011 at 10:42 AM, Garrett Cooper  wrote:
>
> As long as I have reliable checksums that match the what the upstream source 
> says is the real thing, it doesn't practically matter where I get my images 
> from.

Checksums compared to what? How would you know what the correct
checksums for OpenBSD-current is, if it's not built by Theo?


-- 
chs,
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server

[Christer Solskogen]

On Mon, Dec 19, 2011 at 2:16 PM, Alexander Yerenkow  wrote:
> FreeBSD currently have very obscure, closed community. To get in touch, you
> need to subscribe to several mail lists, constantly read them, I've just
> found recently (my shame of course) in mail list that there is service (
> pub.allbsd.org) which constantly building current versions. This is great,
> but at homepage of freebsd.org there is no word about it :)

That's because it's not official. Do you take the risk? Would a
multi-milion-dollar company do that?
For your private server, sure it's probably fine. But how do you know
that those files are not contaminated?
(That being said, the purpose of that service is good. And the files
there a most probably 100% fine. But if it's not official... then..)

-- 
chs, if there is only one candiate, there is one one choice!
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server

[O. Hartmann]

On 12/19/11 13:21, Andreas Nilsson wrote:
> On 19 dec 2011, at 12:50, "Samuel J. Greear"  wrote:
> 
>> 2011/12/19 Lev Serebryakov :
>>> Hello, Samuel.
>>> You wrote 15 декабря 2011 г., 16:32:47:
>>>
 Other benchmarks in the Phoronix suite and their representations are
 similarly flawed, _ALL_ of these results should be ignored and no time
 should be wasted by any FreeBSD committer further evaluating this
 garbage. (Yes, I have been down this rabbit hole).
>>>  Here is one problem: we have choice from three items:
>>>
>>> (1) Make FreeBSD looks good on benchmarks by "fixing" FreeBSD
>>>
>>> (2) Make FreeBSD looks good on benchmarks by "fixing" Phoronix
>>> (communication with them, convincing, that they benchamrks are unfare
>>> / meaningless, ets)
>>>
>>> (3) Lose [potential] userbase.
>>>
>>>  You know, that these benchmarks are bad. I know. But potential (and
>>>  even some current!) user doesn't. And it seems, that these benchmarks
>>>  become popular over Internet.
>>>
>>> --
>>> // Black Lion AKA Lev Serebryakov 
>>>
>>
>> Here is where you completely derail the train, let me paste again what
>> I said before.
>>
>> ...
>> Take the first test as an example, Blogbench read. This doesn't raise
>> any red flags, right? At least not until you realize that Blogbench
>> isn't a read test, it's a read/write test. So what they have done here
>> is run a read/write test and then thrown away the write results for
>> both platforms and reported only the read results. If you dig down
>> into the actual results,
>> http://openbenchmarking.org/result/1112113-AR-ORACLELIN37 -- you will
>> see two Blogbench numbers, one for read and another for write. These
>> were both taken from the same Blogbench run, so FreeBSD optimizes
>> writes over reads, that's probably a good thing for your data but a
>> bad thing when someone totally misrepresents benchmark results.
>> ...
>>
>> FreeBSD actually does _BETTER_ (subjectively) in this test than the
>> Linux system when you look at what is really going on. FreeBSD is
>> favoring writes, which is _GOOD_. FreeBSD does not need to be fixed,
>> the benchmarks need to be fixed to represent reality rather than
>> throwing half of the results in the trash. To be quite frank, "fixing"
>> FreeBSD to look good on this benchmark will make it a worse real-world
>> OS. But you guys go ahead and foot-shoot over these ridiculous
>> benchmarks all you want.
>>
>> Sam
>>
> 
> I seem to remember that before ULE people were fleeing to Linux as the
> os to run apache on since 4BSD didn't scale all too well. That may
> have changed over time though.
> 
> However ULE could perhaps be made aware technologies like turbo-boost,
> ie with few threads higher performance might be gained by utilizing
> all virtual cores on a physical core before spreading tasks to too
> different cores.
> 
> Just my speculations though :)
> 
> Regards
> Andreas Nilsson

Such a scheduling stratey is definitely necessary on AMDs new
"Bulldozer" architecture, which seems to be very pitty about threads
locked on the same module.
Microsoft just offered a patch for Windows 7 to implant such a
"Bulldozer" awarenes but they withdraw the patch as invalid two days
after the release. The seults seem to favour FPU performance over
integer performance.

As Samuel Greear wrote, FreeBSD looks not that bad in some of the
benchmarks but there are obviosly issues, at least the fact that
Phoronix/openbenchmark.org are the only sites offering benchmarks at all.

People outside the FreeBSD realm looking for opportunities, what do you
think they will look first after?
Phoronix/Openbenchmark.org made the first step and they seem to make
FreeBSD look bad (in my opinion), whether righteous or not. Compared to
several subjective impressions I have in our heterogeneous environment
at the lab, Linux on the same hardware looks in several aspects much better.

Oliver



signature.asc
Description: OpenPGP digital signature

Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server

[Alexander Yerenkow]

IMHO, no offence, as always.

As were told, Phoronix used "default" setup, not tuned.
So? Is average user will tune it after setup? No, he'll get same defaults,
and would expect same performance as in tests, and he probably get it.
The problem of FreeBSD is not it's default settings, some kind of very-safe
defaults really should be there.
But problem really is lacking of choosing them (defaults) during install,
for average users.
For example, few checkboxes with common sysctl tuning would be perfect,
even if they would be marked as "Experimental", or not recommended.
I'm thinking it's better way to make something in one place (like in
installer) rather than require make almost same actions in many (hundreds
of thousands?... more?...) places (end-users forced to read
mail-lists/handbooks/forums over and over for same solutions).
Simple example - many connections for PostgreSQL is not available on
FreeBSD out-of-box. Just google "postgresql freebsd max connection" and
you'll see how many there bikesheds requested and same solutions posted
again and again :)

FreeBSD currently have very obscure, closed community. To get in touch, you
need to subscribe to several mail lists, constantly read them, I've just
found recently (my shame of course) in mail list that there is service (
pub.allbsd.org) which constantly building current versions. This is great,
but at homepage of freebsd.org there is no word about it :)

I hope we all do something good about this, and things will going to change.

-- 
Regards,
Alexander Yerenkow
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server

[Andreas Nilsson]

On 19 dec 2011, at 12:50, "Samuel J. Greear"  wrote:

> 2011/12/19 Lev Serebryakov :
>> Hello, Samuel.
>> You wrote 15 декабря 2011 г., 16:32:47:
>>
>>> Other benchmarks in the Phoronix suite and their representations are
>>> similarly flawed, _ALL_ of these results should be ignored and no time
>>> should be wasted by any FreeBSD committer further evaluating this
>>> garbage. (Yes, I have been down this rabbit hole).
>>  Here is one problem: we have choice from three items:
>>
>> (1) Make FreeBSD looks good on benchmarks by "fixing" FreeBSD
>>
>> (2) Make FreeBSD looks good on benchmarks by "fixing" Phoronix
>> (communication with them, convincing, that they benchamrks are unfare
>> / meaningless, ets)
>>
>> (3) Lose [potential] userbase.
>>
>>  You know, that these benchmarks are bad. I know. But potential (and
>>  even some current!) user doesn't. And it seems, that these benchmarks
>>  become popular over Internet.
>>
>> --
>> // Black Lion AKA Lev Serebryakov 
>>
>
> Here is where you completely derail the train, let me paste again what
> I said before.
>
> ...
> Take the first test as an example, Blogbench read. This doesn't raise
> any red flags, right? At least not until you realize that Blogbench
> isn't a read test, it's a read/write test. So what they have done here
> is run a read/write test and then thrown away the write results for
> both platforms and reported only the read results. If you dig down
> into the actual results,
> http://openbenchmarking.org/result/1112113-AR-ORACLELIN37 -- you will
> see two Blogbench numbers, one for read and another for write. These
> were both taken from the same Blogbench run, so FreeBSD optimizes
> writes over reads, that's probably a good thing for your data but a
> bad thing when someone totally misrepresents benchmark results.
> ...
>
> FreeBSD actually does _BETTER_ (subjectively) in this test than the
> Linux system when you look at what is really going on. FreeBSD is
> favoring writes, which is _GOOD_. FreeBSD does not need to be fixed,
> the benchmarks need to be fixed to represent reality rather than
> throwing half of the results in the trash. To be quite frank, "fixing"
> FreeBSD to look good on this benchmark will make it a worse real-world
> OS. But you guys go ahead and foot-shoot over these ridiculous
> benchmarks all you want.
>
> Sam
>

I seem to remember that before ULE people were fleeing to Linux as the
os to run apache on since 4BSD didn't scale all too well. That may
have changed over time though.

However ULE could perhaps be made aware technologies like turbo-boost,
ie with few threads higher performance might be gained by utilizing
all virtual cores on a physical core before spreading tasks to too
different cores.

Just my speculations though :)

Regards
Andreas Nilsson
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server

[Edho Arief]

On Mon, Dec 19, 2011 at 6:49 PM, Samuel J. Greear  wrote:
> FreeBSD actually does _BETTER_ (subjectively) in this test than the
> Linux system when you look at what is really going on. FreeBSD is
> favoring writes, which is _GOOD_. FreeBSD does not need to be fixed,
> the benchmarks need to be fixed to represent reality rather than
> throwing half of the results in the trash. To be quite frank, "fixing"
> FreeBSD to look good on this benchmark will make it a worse real-world
> OS. But you guys go ahead and foot-shoot over these ridiculous
> benchmarks all you want.
>

Would you prefer a blog which allows you to:

A:
- create/write 100 posts/s
- serve/read 1000 posts/s

or

B:
- create/write 80 posts/s
- serve/read 3000 posts/s

?

I would personally choose B.

-- 
O< ascii ribbon campaign - stop html mail - www.asciiribbon.org
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server

[Samuel J. Greear]

2011/12/19 Lev Serebryakov :
> Hello, Samuel.
> You wrote 15 декабря 2011 г., 16:32:47:
>
>> Other benchmarks in the Phoronix suite and their representations are
>> similarly flawed, _ALL_ of these results should be ignored and no time
>> should be wasted by any FreeBSD committer further evaluating this
>> garbage. (Yes, I have been down this rabbit hole).
>  Here is one problem: we have choice from three items:
>
> (1) Make FreeBSD looks good on benchmarks by "fixing" FreeBSD
>
> (2) Make FreeBSD looks good on benchmarks by "fixing" Phoronix
> (communication with them, convincing, that they benchamrks are unfare
> / meaningless, ets)
>
> (3) Lose [potential] userbase.
>
>  You know, that these benchmarks are bad. I know. But potential (and
>  even some current!) user doesn't. And it seems, that these benchmarks
>  become popular over Internet.
>
> --
> // Black Lion AKA Lev Serebryakov 
>

Here is where you completely derail the train, let me paste again what
I said before.

...
Take the first test as an example, Blogbench read. This doesn't raise
any red flags, right? At least not until you realize that Blogbench
isn't a read test, it's a read/write test. So what they have done here
is run a read/write test and then thrown away the write results for
both platforms and reported only the read results. If you dig down
into the actual results,
http://openbenchmarking.org/result/1112113-AR-ORACLELIN37 -- you will
see two Blogbench numbers, one for read and another for write. These
were both taken from the same Blogbench run, so FreeBSD optimizes
writes over reads, that's probably a good thing for your data but a
bad thing when someone totally misrepresents benchmark results.
...

FreeBSD actually does _BETTER_ (subjectively) in this test than the
Linux system when you look at what is really going on. FreeBSD is
favoring writes, which is _GOOD_. FreeBSD does not need to be fixed,
the benchmarks need to be fixed to represent reality rather than
throwing half of the results in the trash. To be quite frank, "fixing"
FreeBSD to look good on this benchmark will make it a worse real-world
OS. But you guys go ahead and foot-shoot over these ridiculous
benchmarks all you want.

Sam
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server

[O. Hartmann]

On 12/19/11 09:27, Lev Serebryakov wrote:
> Hello, Samuel.
> You wrote 15 декабря 2011 г., 16:32:47:
> 
>> Other benchmarks in the Phoronix suite and their representations are
>> similarly flawed, _ALL_ of these results should be ignored and no time
>> should be wasted by any FreeBSD committer further evaluating this
>> garbage. (Yes, I have been down this rabbit hole).
>   Here is one problem: we have choice from three items:
> 
> (1) Make FreeBSD looks good on benchmarks by "fixing" FreeBSD
> 
> (2) Make FreeBSD looks good on benchmarks by "fixing" Phoronix
> (communication with them, convincing, that they benchamrks are unfare
> / meaningless, ets)
> 
> (3) Lose [potential] userbase.
> 
>   You know, that these benchmarks are bad. I know. But potential (and
>  even some current!) user doesn't. And it seems, that these benchmarks
>  become popular over Internet.
> 
 +1

It is not about a faky way to let a specific OS look good by any means.
I'M afraid of (3), which also implies pushing more towards beeing
meaningless and not anymore a alternative with a unique, remarkable
criteria to be choosen as __the__ operating system of the first choice
for several purposes. By the way, how such a development could look
alaike is very clear when it comes to GPGPU/HPC, highly related to the
availability of proper graphics card drivers, X11 development and the
necessary libraries, APIs and even compilers.

None of those "professionals" out here, none of those pushing the
eyewhitness of bad performance into very deep-insight-talks about what
could cause the problem has obviously ever negotiated with people of the
"upper floor" when it comes to the choice of the OS.
Within my department, the *BSD aren't even considered an option, even if
they would perform best for the specified purpose (which, I regeret,
is a shrinking basis now since also Linux will have ZFS).

Sometimes I feel like Don Quixote, fighting against windmills. Sorry
having brought up this thread and I beg for pardon for putting another
scrtach into the autoerotic world of the "core".



signature.asc
Description: OpenPGP digital signature

Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server

[Lev Serebryakov]

Hello, Adrian.
You wrote 16 декабря 2011 г., 20:43:27:

> Guys/girls/fuzzy things - this is 2011; people look at shiny blog
> sites with graphs rather than mailing lists. Sorry, we lost that
> battle. :)
  My thoughts exactly.

-- 
// Black Lion AKA Lev Serebryakov 

___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server

[Lev Serebryakov]

Hello, Samuel.
You wrote 15 декабря 2011 г., 16:32:47:

> Other benchmarks in the Phoronix suite and their representations are
> similarly flawed, _ALL_ of these results should be ignored and no time
> should be wasted by any FreeBSD committer further evaluating this
> garbage. (Yes, I have been down this rabbit hole).
  Here is one problem: we have choice from three items:

(1) Make FreeBSD looks good on benchmarks by "fixing" FreeBSD

(2) Make FreeBSD looks good on benchmarks by "fixing" Phoronix
(communication with them, convincing, that they benchamrks are unfare
/ meaningless, ets)

(3) Lose [potential] userbase.

  You know, that these benchmarks are bad. I know. But potential (and
 even some current!) user doesn't. And it seems, that these benchmarks
 become popular over Internet.

-- 
// Black Lion AKA Lev Serebryakov 

___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server

[matthew]

   Thanks.
   My request for the person documenting the tunings also runs = the
   benchmark to ensure expected behaviour.
   The installation, execut= ion and comparison against the benchmarks in
   the article is fairly simple.<= br>
   Note that some tuning may not be relevant or recommended (ie: some o   f the 
fs benchmarks are sensitive to barriers and other synchronous
   operati= ons).  I'd recommend bowing out of a benchmark with a 'we're
   going to = be slower since the default configuration is this way for
   the following rea= son' if this is the case.
   Thanks 'someone'.
   Matthew
Dec 16, 2011 8:46 AM,= Adrian Chadd  wrote:
   Can someone please write up a nice, concise blog post somewhere
   outlining all of this?
   
   Extra bonus points if it's a blog t= hat is picked up by
   blogs.freebsdish.org and/or some of the other BSD= sites.
   
   Guys/girls/fuzzy things - this is 2011; people look at= shiny blog
   sites with graphs rather than mailing lists. Sorry, we lo= st that
   battle. :)
   
   
   
   Adrian
   __= _
   freebsd-performa...@freebsd.org= mailing list
   http://lists.freebsd.org/mailman/listinfo/freebsd-perfo= rmance
   To unsubscribe, send any mail to "freebsd-performance-unsubscr   
i...@freebsd.org"
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server

[Adrian Chadd]

Can someone please write up a nice, concise blog post somewhere
outlining all of this?

Extra bonus points if it's a blog that is picked up by
blogs.freebsdish.org and/or some of the other BSD sites.

Guys/girls/fuzzy things - this is 2011; people look at shiny blog
sites with graphs rather than mailing lists. Sorry, we lost that
battle. :)



Adrian
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server

[Arnaud Lacombe]

Hi,

[resend on the ml, my bad]

On Fri, Dec 16, 2011 at 5:54 AM, Attilio Rao  wrote:
> 2011/12/16 Arnaud Lacombe :
>> Hi,
>>
>> On Thu, Dec 15, 2011 at 2:32 AM, O. Hartmann
>>  wrote:
>>> Just saw this shot benchmark on Phoronix dot com today:
>>>
>>> http://www.phoronix.com/scan.php?page=news_item&px=MTAyNzA
>>>
>> it might be worth highlighting that despite Oracle Linux 6.1 Server is
>> using a kernel + compiler almost 2 years old, it still manages to
>> out-perform the bleeding edge FreeBSD :-)
>>
>> Now, from what I've read so far in this thread, it seems that a lot of
>> people are still in abnegation...
>>
>> my 0.2c,
>>  - Arnaud
>
> Said by someone which really thinks passing __FILE__ and __LINE__ to
> kernel function is going to give a mesaurable performance penalty is
> really hilarious however :)
>
You are right, the rest of the kernel's subsystem are so sluggish,
fragile and half baked that this would barely improve anything...

That will be my last word in this thread.

 - Arnaud
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server

[Stefan Esser]

Am 16.12.2011 08:06, schrieb O. Hartmann:
> For the underlying OS, as far as I know, the compiler hasn't as much
> impact as on userland software since autovectorization and other neat
> things are not used during system build.
> 
> From my experience using gcc 4.2 or 4.4/4.5 does not have an impact
> beyond 3% when SSE isn't explicetly enforced.

Well, but the compute intensive tests showed performance variance of a
few percents only, IIRC. The big differences were in the parts that
heavily depend on file system and buffer cache concepts (i.e. the low
limit on dirty buffers in FreeBSD, which is very beneficial in real
world situations; do you remember the first few releases of SunOS-4,
which heavily suffered in interactive performance due to a naive unified
buffer cache VM system that did not limit the amount of dirty buffers?
It caused interactive shells to be swapped out within seconds on systems
with background jobs writing to disk).

> More interesting is the performance gain due to the architecture. I
> think it would be very easy for M. Larabel to repeat this benchmark with
> a "bleeding edge"  Ubuntu or Suse as well. And since FreeBSD 9.0 can be
> compiled with CLANG, it should be possible to compare both also with
> "bleeding edge" compilers, say FreeBSD 9/CLANG, Ubuntu 12/gcc 4.6.2.

Clang may be considered "bleeding edge", but in quite a different way
than gcc-4.6.2. While the latter can look back on 2 decades of
development, clang is still in a state where feature completeness (and
bug-to-bug compatibility with GCC ;-) is much more important than
performance. there is much promise of powerful optimizations becoming
available in clang once it is mature, but just now expect GCC 4.6.2 to
deliver 5% to 10% higher performance than clang.

But as stated before: To exclude compiler dependencies just run the
Linux binaries on FreeBSD. There is slight emulation overhead and Glibc
is not particularly optimized for FreeBSD, but this will still provide
more useful results.

And the tests should be selected to represent reasonable real-world
scenarios. Server programs tested on otherwise idle systems and running
for just a few seconds (not reaching equilibrium during the majority of
the test period) are not representative at all (again: if your goal is
to compare server performance).

Regards, STefan
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server

[Attilio Rao]

2011/12/16 Arnaud Lacombe :
> Hi,
>
> On Thu, Dec 15, 2011 at 2:32 AM, O. Hartmann
>  wrote:
>> Just saw this shot benchmark on Phoronix dot com today:
>>
>> http://www.phoronix.com/scan.php?page=news_item&px=MTAyNzA
>>
> it might be worth highlighting that despite Oracle Linux 6.1 Server is
> using a kernel + compiler almost 2 years old, it still manages to
> out-perform the bleeding edge FreeBSD :-)
>
> Now, from what I've read so far in this thread, it seems that a lot of
> people are still in abnegation...
>
> my 0.2c,
>  - Arnaud

Said by someone which really thinks passing __FILE__ and __LINE__ to
kernel function is going to give a mesaurable performance penalty is
really hilarious however :)

It is crystal clear you really don't understand how to make reliable
benchmarks (and likely you don't really have a grasp of nowaday's
machine contention points), so why you keep talking about it? It would
be more valuable for you and whatever project you follow if you spend
your time coding and making real benchmarking.

Attilio


-- 
Peace can only be achieved by understanding - A. Einstein
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server

[O. Hartmann]

On 12/16/11 07:44, Joe Holden wrote:
> Arnaud Lacombe wrote:
>> Hi,
>>
>> On Thu, Dec 15, 2011 at 2:32 AM, O. Hartmann
>>  wrote:
>>> Just saw this shot benchmark on Phoronix dot com today:
>>>
>>> http://www.phoronix.com/scan.php?page=news_item&px=MTAyNzA
>>>
>> it might be worth highlighting that despite Oracle Linux 6.1 Server is
>> using a kernel + compiler almost 2 years old, it still manages to
>> out-perform the bleeding edge FreeBSD :-)
>>
> serenity# gcc --version
> gcc (GCC) 4.2.1 20070831 patched [FreeBSD]
> 
> serenity# uname -r
> 9.0-RC3
> 

For the underlying OS, as far as I know, the compiler hasn't as much
impact as on userland software since autovectorization and other neat
things are not used during system build.

From my experience using gcc 4.2 or 4.4/4.5 does not have an impact
beyond 3% when SSE isn't explicetly enforced.

More interesting is the performance gain due to the architecture. I
think it would be very easy for M. Larabel to repeat this benchmark with
a "bleeding edge"  Ubuntu or Suse as well. And since FreeBSD 9.0 can be
compiled with CLANG, it should be possible to compare both also with
"bleeding edge" compilers, say FreeBSD 9/CLANG, Ubuntu 12/gcc 4.6.2.


>> Now, from what I've read so far in this thread, it seems that a lot of
>> people are still in abnegation...
>>
>> my 0.2c,
>>  - Arnaud
>>
>>> It may be worth to discuss the sad performance of FBSD in some parts of
>>> the benchmark. A difference of a factor 10 or 100 is simply far beyond
>>> disapointing, it is more than inacceptable and by just reading those
>>> benchmarks, I'd like to drop thinking of using FreeBSD even as a backend
>>> server in scientific and business environments. In detail, some of the
>>> SciMark benches look disappointing. The overall image can't help over
>>> the fact that in C-Ray FreeBSD is better performing.
>>>
>>> From the compiler, I'd like say there couldn't be a drop of more than 10
>>> - 15% in performance - but not 10 or 100 times.
>>>
>>> I'm just thinking about the discussion of SCHED_ULE and all the saur
>>> spots we discussed when I stumbled over the test.
>>>
>>> Regards,
>>> Oliver



signature.asc
Description: OpenPGP digital signature

Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server

[Joe Holden]

Arnaud Lacombe wrote:

Hi,

On Thu, Dec 15, 2011 at 2:32 AM, O. Hartmann
 wrote:

Just saw this shot benchmark on Phoronix dot com today:

http://www.phoronix.com/scan.php?page=news_item&px=MTAyNzA


it might be worth highlighting that despite Oracle Linux 6.1 Server is
using a kernel + compiler almost 2 years old, it still manages to
out-perform the bleeding edge FreeBSD :-)


serenity# gcc --version
gcc (GCC) 4.2.1 20070831 patched [FreeBSD]

serenity# uname -r
9.0-RC3


Now, from what I've read so far in this thread, it seems that a lot of
people are still in abnegation...

my 0.2c,
 - Arnaud


It may be worth to discuss the sad performance of FBSD in some parts of
the benchmark. A difference of a factor 10 or 100 is simply far beyond
disapointing, it is more than inacceptable and by just reading those
benchmarks, I'd like to drop thinking of using FreeBSD even as a backend
server in scientific and business environments. In detail, some of the
SciMark benches look disappointing. The overall image can't help over
the fact that in C-Ray FreeBSD is better performing.

From the compiler, I'd like say there couldn't be a drop of more than 10
- 15% in performance - but not 10 or 100 times.

I'm just thinking about the discussion of SCHED_ULE and all the saur
spots we discussed when I stumbled over the test.

Regards,
Oliver


___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"


___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server

[Arnaud Lacombe]

Hi,

On Thu, Dec 15, 2011 at 2:32 AM, O. Hartmann
 wrote:
> Just saw this shot benchmark on Phoronix dot com today:
>
> http://www.phoronix.com/scan.php?page=news_item&px=MTAyNzA
>
it might be worth highlighting that despite Oracle Linux 6.1 Server is
using a kernel + compiler almost 2 years old, it still manages to
out-perform the bleeding edge FreeBSD :-)

Now, from what I've read so far in this thread, it seems that a lot of
people are still in abnegation...

my 0.2c,
 - Arnaud

> It may be worth to discuss the sad performance of FBSD in some parts of
> the benchmark. A difference of a factor 10 or 100 is simply far beyond
> disapointing, it is more than inacceptable and by just reading those
> benchmarks, I'd like to drop thinking of using FreeBSD even as a backend
> server in scientific and business environments. In detail, some of the
> SciMark benches look disappointing. The overall image can't help over
> the fact that in C-Ray FreeBSD is better performing.
>
> From the compiler, I'd like say there couldn't be a drop of more than 10
> - 15% in performance - but not 10 or 100 times.
>
> I'm just thinking about the discussion of SCHED_ULE and all the saur
> spots we discussed when I stumbled over the test.
>
> Regards,
> Oliver
>
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server

[Chris Rees]

On 15 Dec 2011 21:25, "Kevin Oberman"  wrote:
>
> On Thu, Dec 15, 2011 at 10:46 AM, Chris Rees  wrote:
> > On 15 December 2011 17:58, O. Hartmann 
wrote:
> >> Since ZFS in Linux can only be achieved via FUSE (ad far as I know), it
> >> is legitimate to compare ZFS and ext4. It would be much more
competetive
> >> to compare Linux BTRFS and FreeBSD ZFS.
> >>
> >
> >
> > Er... does ext4 guarantee data integrity?
> >
> > You're not comparing like with like; please do some research on the
> > point of ZFS before asserting that they're fair comparisons.
> >
> > A fair(er) comparison could be ext4 with UFS+soft-updates.
>
> Wouldn't UFS+SUJ be the closest atch?

Yup. Thanks.

Chris
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server

[Kevin Oberman]

On Thu, Dec 15, 2011 at 10:46 AM, Chris Rees  wrote:
> On 15 December 2011 17:58, O. Hartmann  wrote:
>> Since ZFS in Linux can only be achieved via FUSE (ad far as I know), it
>> is legitimate to compare ZFS and ext4. It would be much more competetive
>> to compare Linux BTRFS and FreeBSD ZFS.
>>
>
>
> Er... does ext4 guarantee data integrity?
>
> You're not comparing like with like; please do some research on the
> point of ZFS before asserting that they're fair comparisons.
>
> A fair(er) comparison could be ext4 with UFS+soft-updates.

Wouldn't UFS+SUJ be the closest atch?
-- 
R. Kevin Oberman, Network Engineer
E-mail: kob6...@gmail.com
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server

[Chris Rees]

On 15 December 2011 17:58, O. Hartmann  wrote:
> Since ZFS in Linux can only be achieved via FUSE (ad far as I know), it
> is legitimate to compare ZFS and ext4. It would be much more competetive
> to compare Linux BTRFS and FreeBSD ZFS.
>


Er... does ext4 guarantee data integrity?

You're not comparing like with like; please do some research on the
point of ZFS before asserting that they're fair comparisons.

A fair(er) comparison could be ext4 with UFS+soft-updates.

Chris
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server

[Freddie Cash]

On Thu, Dec 15, 2011 at 9:58 AM, O. Hartmann
 wrote:
> Am 12/15/11 14:51, schrieb Daniel Kalchev:
>>
>> On Dec 15, 2011, at 3:25 PM, Stefan Esser wrote:
>>
>>> Am 15.12.2011 11:10, schrieb Michael Larabel:
 No, the same hardware was used for each OS.

 In terms of the software, the stock software stack for each OS was used.
>>>
>>> Just curious: Why did you choose ZFS on FreeBSD, while UFS2 (with
>>> journaling enabled) should be an obvious choice since it is more similar
>>> in concept to ext4 and since that is what most FreeBSD users will use
>>> with FreeBSD?
>>
>>
>> Or perhaps, since it is "server" Linux distribution, use ZFS on Linux as 
>> well. With identical tuning on both Linux and FreeBSD. Having the same FS 
>> used by both OS will help make the comparison more sensible for FS I/O.
>>
>> Daniel___
>
> Since ZFS in Linux can only be achieved via FUSE (ad far as I know), it
> is legitimate to compare ZFS and ext4. It would be much more competetive
> to compare Linux BTRFS and FreeBSD ZFS.

There is a separate kernel module for ZFS that can be installed,
giving you proper kernel-level support for ZFS on Linux.

-- 
Freddie Cash
fjwc...@gmail.com
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server

[O. Hartmann]

Am 12/15/11 14:58, schrieb Daniel Kalchev:
> 
> On Dec 15, 2011, at 3:48 PM, Jeremy Chadwick wrote:
> 
> […]
>> That said: thrown out, data ignored, done.
>>
>> Now what?  Where are we?  We're right back where we were a day or two
>> ago; meaning no closer to solving the dilemma reported by users and
>> SCHED_ULE.  Heck, we're not even sure if there is an issue, other than
>> some folks confirming that SCHED_4BSD performs better for them (that's
>> what started this whole thread), and there are at least a couple which
>> have stated this.
> 
> But, are any of these benchmarks really engaging the 4BSD/ULE scheduler 
> differences? Most such benchmarks are run on a system with no other load 
> whatsoever and in no way represent real world experience.
> 
> What is more, I believe in such benchmarks "the system feels sluggish" is not 
> measured at all. Even if it is measured, if in such case the benchmark 
> finishes "better" - that is, faster, or say, makes the system freeze for the 
> user for the duration of the test -- it will be considered "win", because the 
> benchmark suite ran faster on that particular system -- whereas a system 
> which ran the benchmark fast, provided good interactive response etc would be 
> considered "loser".

I guess you have some proofs on that "feeling"?

> 
> I think it is not good idea to hijack this thread, but instead focusing on 
> the other SCHED_ULE bashing thread to define an reasonable benchmark or a set 
> of benchmarks rather -- so that many would run it and provide feedback.
> 
> 
> Daniel___



signature.asc
Description: OpenPGP digital signature

Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server

[O. Hartmann]

Am 12/15/11 14:51, schrieb Daniel Kalchev:
> 
> On Dec 15, 2011, at 3:25 PM, Stefan Esser wrote:
> 
>> Am 15.12.2011 11:10, schrieb Michael Larabel:
>>> No, the same hardware was used for each OS.
>>>
>>> In terms of the software, the stock software stack for each OS was used.
>>
>> Just curious: Why did you choose ZFS on FreeBSD, while UFS2 (with
>> journaling enabled) should be an obvious choice since it is more similar
>> in concept to ext4 and since that is what most FreeBSD users will use
>> with FreeBSD?
> 
> 
> Or perhaps, since it is "server" Linux distribution, use ZFS on Linux as 
> well. With identical tuning on both Linux and FreeBSD. Having the same FS 
> used by both OS will help make the comparison more sensible for FS I/O.
> 
> Daniel___

Since ZFS in Linux can only be achieved via FUSE (ad far as I know), it
is legitimate to compare ZFS and ext4. It would be much more competetive
to compare Linux BTRFS and FreeBSD ZFS.

Each OS does optimize on different filesystems and a user/manager can
assume that the vendor offers the best performance available by turning
on the default FS by a standard stock installation.

Using ZFS on Linux would be a great disadvantage and the benchmark would
turn out the same bullsh... as comparing Linux-domain only with FreeBSD
weknesses only ...

Linux distributions offer setups for desktop and server. The FreeBSD
folks have the choice to do it themselfes. And maybe I'm one of those
puritain people appreciating this. "Out of the box" OS is Windooze, with
all its consequences.

Oliver

Post scriptum:
It seems to be hard to follow the benchmark environment on Phoronix
since the URL refers to a setup of different systems.



signature.asc
Description: OpenPGP digital signature

Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server

[Sergey Matveychuk]

15.12.2011 17:36, Michael Larabel пишет:

On 12/15/2011 07:25 AM, Stefan Esser wrote:

Am 15.12.2011 11:10, schrieb Michael Larabel:

No, the same hardware was used for each OS.

In terms of the software, the stock software stack for each OS was used.

Just curious: Why did you choose ZFS on FreeBSD, while UFS2 (with
journaling enabled) should be an obvious choice since it is more similar
in concept to ext4 and since that is what most FreeBSD users will use
with FreeBSD?


I was running some ZFS vs. UFS tests as well and this happened to have
ZFS on when I was running some other tests.



Can we look at the tests?
My opinion is ZFS without tuning is much slower than UFS2.
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server

[Volodymyr Kostyrko]

15.12.2011 15:48, Jeremy Chadwick wrote:

I'm getting to the point where I'm considering formulating a private
mail to Jeff Roberson, requesting that he be aware of the discussion
that's happening (not that he necessarily follow or read it), and that
based on what I can tell we're at a roadblock -- nobody so far is
absolutely certain how to "benchmark" and compare ULE vs. 4BSD in
multiple ways, so that those of us involved here can run such utilities
and provide the data somewhere central for devs to review.  I only
mention this because so far I haven't seen anyone really say "okay, this
is what we should be using for these kinds of tests".  Yay nature of the
beast.


I'll try to summarize and propose a test scenario. I don't know whether 
this helps or not.


We should have two different task types for this one. The first would be 
Super Affine tasks. They should use few to none syscalls, use medium 
math, have low memory footprint. No syscalls means this tasks will never 
stop for memory/disk or other activity so each time the queue is looked 
upon this task will be ready to run. Medium math means this shouldn't be 
just a simple big loop so that processor will really compute something 
with this data. Low memory footprint means this task can reside with 
data on CPU L1 cache for eons. I'm not sure about branch prediction, 
should it be distorted or not...


The other task type would be Worker. It doesn't matter what it does but 
it agressively uses syscalls like working with files/directories.


There should be at least one SA-task per core and at least 10 (?) 
W-tasks per core.


--
Sphinx of black quartz judge my vow.
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server

[Michael Larabel]

On 12/15/2011 08:26 AM, Sergey Matveychuk wrote:

15.12.2011 17:36, Michael Larabel пишет:

On 12/15/2011 07:25 AM, Stefan Esser wrote:

Am 15.12.2011 11:10, schrieb Michael Larabel:

No, the same hardware was used for each OS.

In terms of the software, the stock software stack for each OS was 
used.

Just curious: Why did you choose ZFS on FreeBSD, while UFS2 (with
journaling enabled) should be an obvious choice since it is more 
similar

in concept to ext4 and since that is what most FreeBSD users will use
with FreeBSD?


I was running some ZFS vs. UFS tests as well and this happened to have
ZFS on when I was running some other tests.



Can we look at the tests?
My opinion is ZFS without tuning is much slower than UFS2.



http://www.phoronix.com/scan.php?page=news_item&px=MTAyNjg
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server

[Daniel Kalchev]

On Dec 15, 2011, at 3:48 PM, Jeremy Chadwick wrote:

[…]
> That said: thrown out, data ignored, done.
> 
> Now what?  Where are we?  We're right back where we were a day or two
> ago; meaning no closer to solving the dilemma reported by users and
> SCHED_ULE.  Heck, we're not even sure if there is an issue, other than
> some folks confirming that SCHED_4BSD performs better for them (that's
> what started this whole thread), and there are at least a couple which
> have stated this.

But, are any of these benchmarks really engaging the 4BSD/ULE scheduler 
differences? Most such benchmarks are run on a system with no other load 
whatsoever and in no way represent real world experience.

What is more, I believe in such benchmarks "the system feels sluggish" is not 
measured at all. Even if it is measured, if in such case the benchmark finishes 
"better" - that is, faster, or say, makes the system freeze for the user for 
the duration of the test -- it will be considered "win", because the benchmark 
suite ran faster on that particular system -- whereas a system which ran the 
benchmark fast, provided good interactive response etc would be considered 
"loser".

I think it is not good idea to hijack this thread, but instead focusing on the 
other SCHED_ULE bashing thread to define an reasonable benchmark or a set of 
benchmarks rather -- so that many would run it and provide feedback.


Daniel___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server

[Daniel Kalchev]

On Dec 15, 2011, at 3:25 PM, Stefan Esser wrote:

> Am 15.12.2011 11:10, schrieb Michael Larabel:
>> No, the same hardware was used for each OS.
>> 
>> In terms of the software, the stock software stack for each OS was used.
> 
> Just curious: Why did you choose ZFS on FreeBSD, while UFS2 (with
> journaling enabled) should be an obvious choice since it is more similar
> in concept to ext4 and since that is what most FreeBSD users will use
> with FreeBSD?


Or perhaps, since it is "server" Linux distribution, use ZFS on Linux as well. 
With identical tuning on both Linux and FreeBSD. Having the same FS used by 
both OS will help make the comparison more sensible for FS I/O.

Daniel___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server

[Jeremy Chadwick]

On Thu, Dec 15, 2011 at 05:32:47AM -0700, Samuel J. Greear wrote:
> > Well, the only way it's going to get fixed is if someone sits down,
> > replicates it, and starts to document exactly what it is that these
> > benchmarks are/aren't doing.
> >
> 
> I think you will find that investigation is largely a waste of time,
> because not only are some of these benchmarks just downright silly,
> there are huge differences in the environments (compiler versions),
> etc., etc. leading to a largely apples/oranges comparison. But also
> the the analysis and reporting of the results by Phoronix is simply
> moronic to the point of being worse than useful, they are spreading
> misinformation.
> 
> Take the first test as an example, Blogbench read. This doesn't raise
> any red flags, right? At least not until you realize that Blogbench
> isn't a read test, it's a read/write test. So what they have done here
> is run a read/write test and then thrown away the write results for
> both platforms and reported only the read results. If you dig down
> into the actual results,
> http://openbenchmarking.org/result/1112113-AR-ORACLELIN37 -- you will
> see two Blogbench numbers, one for read and another for write. These
> were both taken from the same Blogbench run, so FreeBSD optimizes
> writes over reads, that's probably a good thing for your data but a
> bad thing when someone totally misrepresents benchmark results.
> 
> Other benchmarks in the Phoronix suite and their representations are
> similarly flawed, _ALL_ of these results should be ignored and no time
> should be wasted by any FreeBSD committer further evaluating this
> garbage. (Yes, I have been down this rabbit hole).

For sake of argument, let's say we throw out the Phoronix benchmarks as
a data source (I don't think the benchmark specifically implied or
stated "this is all because of SCHED_ULE" though; remember, that's what
we're supposed to be focusing on.  There may not be a direct correlation
between the Phoronix benchmarks and the ULE issue reported here...).
That said: thrown out, data ignored, done.

Now what?  Where are we?  We're right back where we were a day or two
ago; meaning no closer to solving the dilemma reported by users and
SCHED_ULE.  Heck, we're not even sure if there is an issue, other than
some folks confirming that SCHED_4BSD performs better for them (that's
what started this whole thread), and there are at least a couple which
have stated this.

So given the above semi-devil's-advocate response -- Sam, do you have
something positive or progressive to offer so we can move forward on the
ULE vs. 4BSD debacle?  :-)  The smiley is meant to be sincere, not
sarcastic.

I'm getting to the point where I'm considering formulating a private
mail to Jeff Roberson, requesting that he be aware of the discussion
that's happening (not that he necessarily follow or read it), and that
based on what I can tell we're at a roadblock -- nobody so far is
absolutely certain how to "benchmark" and compare ULE vs. 4BSD in
multiple ways, so that those of us involved here can run such utilities
and provide the data somewhere central for devs to review.  I only
mention this because so far I haven't seen anyone really say "okay, this
is what we should be using for these kinds of tests".  Yay nature of the
beast.

-- 
| Jeremy Chadwickjdc at parodius.com |
| Parodius Networking   http://www.parodius.com/ |
| UNIX Systems Administrator   Mountain View, CA, US |
| Making life hard for others since 1977.   PGP 4BD6C0CB |

___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server

[Stefan Esser]

Am 15.12.2011 11:10, schrieb Michael Larabel:
> No, the same hardware was used for each OS.
> 
> In terms of the software, the stock software stack for each OS was used.

Just curious: Why did you choose ZFS on FreeBSD, while UFS2 (with
journaling enabled) should be an obvious choice since it is more similar
in concept to ext4 and since that is what most FreeBSD users will use
with FreeBSD?

Did you tune the ZFS ARC (e.g. vfs.zfs.arc_max="6G") for the tests?

And BTW: Did your measured run times account for the effect, that Linux
keeps much more dirty data in the buffer cache (FreeBSD has a low limit
on dirty buffers since under realistic load the already cached data is
much more likely to be reused and thus more valuable than freshly
written data; aggressively caching dirty data would significantly reduce
throughput and responsiveness under high load). Given the hardware specs
of the test system, I guess that Linux accepts at least 100 times the
dirty data in the buffer cache, compared to FreeBSD (where this number
is at most in the tens of megabyte range).

If you did not, then your results do not represent a server load (which
I'd expect relevant, if you are testing against Oracle Linux 6.1
server), where continuous performance is required. Tests that run on an
idle system starting in a clean state and ignoring background flushing
of the buffer cache after the timed program has stopped are perhaps
useful for a very lowly loaded PC, but not for a system with high load
average as the default.

I bet that if you compared the systems under higher load (which
admittedly makes it much harder to get sensible numbers for the program
under test) or with reduced buffer cache size (or raise the dirty buffer
limit in FreeBSD accordingly, which ought to be possible with sysctl
and/or boot time tuneables, e.g. "vfs.hidirtybuffers").

And a last remark: Single benchmark runs do not provide reliable data.
FreeBSD comes with "ministat" to check the significance of benchmark
results. Each test should be repeated at least 5 times for meaningful
averages with acceptable confidence level.

Regards, STefan
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server

[Michael Larabel]

On 12/15/2011 07:25 AM, Stefan Esser wrote:

Am 15.12.2011 11:10, schrieb Michael Larabel:

No, the same hardware was used for each OS.

In terms of the software, the stock software stack for each OS was used.

Just curious: Why did you choose ZFS on FreeBSD, while UFS2 (with
journaling enabled) should be an obvious choice since it is more similar
in concept to ext4 and since that is what most FreeBSD users will use
with FreeBSD?


I was running some ZFS vs. UFS tests as well and this happened to have 
ZFS on when I was running some other tests.




Did you tune the ZFS ARC (e.g. vfs.zfs.arc_max="6G") for the tests?


The OS was left in its stock configuration.



And BTW: Did your measured run times account for the effect, that Linux
keeps much more dirty data in the buffer cache (FreeBSD has a low limit
on dirty buffers since under realistic load the already cached data is
much more likely to be reused and thus more valuable than freshly
written data; aggressively caching dirty data would significantly reduce
throughput and responsiveness under high load). Given the hardware specs
of the test system, I guess that Linux accepts at least 100 times the
dirty data in the buffer cache, compared to FreeBSD (where this number
is at most in the tens of megabyte range).

If you did not, then your results do not represent a server load (which
I'd expect relevant, if you are testing against Oracle Linux 6.1
server), where continuous performance is required. Tests that run on an
idle system starting in a clean state and ignoring background flushing
of the buffer cache after the timed program has stopped are perhaps
useful for a very lowly loaded PC, but not for a system with high load
average as the default.

I bet that if you compared the systems under higher load (which
admittedly makes it much harder to get sensible numbers for the program
under test) or with reduced buffer cache size (or raise the dirty buffer
limit in FreeBSD accordingly, which ought to be possible with sysctl
and/or boot time tuneables, e.g. "vfs.hidirtybuffers").

And a last remark: Single benchmark runs do not provide reliable data.
FreeBSD comes with "ministat" to check the significance of benchmark
results. Each test should be repeated at least 5 times for meaningful
averages with acceptable confidence level.


The Phoronix Test Suite runs most tests a minimum of three times and if 
the standard deviation exceeds 3.5% the run count is dynamically 
increased, among other safeguards.


-- Michael



Regards, STefan



___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server

[Samuel J. Greear]

> Well, the only way it's going to get fixed is if someone sits down,
> replicates it, and starts to document exactly what it is that these
> benchmarks are/aren't doing.
>

I think you will find that investigation is largely a waste of time,
because not only are some of these benchmarks just downright silly,
there are huge differences in the environments (compiler versions),
etc., etc. leading to a largely apples/oranges comparison. But also
the the analysis and reporting of the results by Phoronix is simply
moronic to the point of being worse than useful, they are spreading
misinformation.

Take the first test as an example, Blogbench read. This doesn't raise
any red flags, right? At least not until you realize that Blogbench
isn't a read test, it's a read/write test. So what they have done here
is run a read/write test and then thrown away the write results for
both platforms and reported only the read results. If you dig down
into the actual results,
http://openbenchmarking.org/result/1112113-AR-ORACLELIN37 -- you will
see two Blogbench numbers, one for read and another for write. These
were both taken from the same Blogbench run, so FreeBSD optimizes
writes over reads, that's probably a good thing for your data but a
bad thing when someone totally misrepresents benchmark results.

Other benchmarks in the Phoronix suite and their representations are
similarly flawed, _ALL_ of these results should be ignored and no time
should be wasted by any FreeBSD committer further evaluating this
garbage. (Yes, I have been down this rabbit hole).

Best,
Sam
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server

[Jeremy Chadwick]

On Thu, Dec 15, 2011 at 04:55:16AM -0600, Michael Larabel wrote:
> On 12/15/2011 04:41 AM, Michael Ross wrote:
> >Am 15.12.2011, 11:10 Uhr, schrieb Michael Larabel
> >:
> >
> >>On 12/15/2011 02:48 AM, Michael Ross wrote:
> >
> >>>Anyway these tests were performed on different hardware, FWIW.
> >>>And with different filesystems, different compilers, different GUIs...
> >>>
> >>>
> >>
> >>No, the same hardware was used for each OS.
> >>
> >
> >The picture under the heading "System Hardware / Software" does
> >not reflect that.
> >
> >Motherboard description differs, Chipset description for FreeBSD
> >is empty.
> >
> 
> I was the on that carried out the testing and know that it was on
> the same system.
> 
> All of the testing, including the system tables, is fully automated.
> Under FreeBSD sometimes the parsing of some component strings isn't
> as nice as Linux and other supported operating systems by the
> Phoronix Test Suite. For the BSD motherboard string parsing it's
> grabbing hw.vendor/hw.product from sysctl.
>
> Is there a better place to read the motherboard DMI information from?

I *think* what you're referring to is SMBIOS strings -- and these are
available from kenv(1) / kenv(2), not sysctl.  But keep reading for why
SMBIOS data is not 100% reliable (greatly depends on the hardware).  For
actual device strings/etc. for all devices on busses (PCI, AGP, etc.)
you can use pciconf -lvcb.

That's about as good as it's going to get via software.  SMBIOS data
(e.g. smbios.{bios,chassis,planar,system}) is never going to give you
fully-identifiable data; I can point you to tons of systems where the
data inserted there is nonsense, sometimes even just ASCII spaces (and
that is the fault of the system vendor/BIOS manufacturer, not FreeBSD).
Sometimes identical strings are used across completely different
systems/boards (sometimes even server-class boards like ones from
Supermicro).  And PCI vendor strings don't give you things like speeds,
frequency/voltages, etc..  Sometimes this matters.  For example (just
making something up): "the video benchmark was horrible on FreeBSD",
when in fact it turned out that a run of "pciconf -lvcb" showed your
PCIe card was running at x4 link speed instead of x16.

The best place to get your specifications from are:

* The box
* The physical hardware (by physically inspecting it)
* The user manual / product documentation/
* Purchase orders from whoever bought the hardware
* And, of course, operational speed (if possible) from the OS/userland
  utilities

When I read a benchmark/review, I have to assume the person is doing
them on a system they have 100% control over, all the way down to the
hardware.  Thus, they should know what exact hardware they have.

Also, when publishing results online, you should take the time to
proofread everything (with a 2nd set of eyes if possible) and be patient
and thorough.  People like accuracy, especially when there's hard
data/evidence to back it up that can be made available for download.

Try to understand: so many review-esque sites consist of individuals who
do not understand even remotely what they're doing.

I'm going to give you two examples -- one personal, one word-of-mouth
but from someone I trust dearly.

I have a "reverse analysis" of Anantech's Intel 510 SSD review that has been
sitting in my "draft" folder on my blog for a month now because I'm
downright afraid to publish how their data seems completely and totally
wrong (with evidence to prove it).  I'm afraid/stalling because I want
to make absolutely damn sure I'm not missing some key piece of evidence
that explains it, and I've had multiple people read it and go "...wow, I
didn't notice that, that benchmark data makes no sense", but I'm STILL
reluctant.  The last thing I want to do is "publish" something that
sparks a controversy where it turns out I'm wrong (and I AM wrong, quite
often!).

As for the other:

http://www.overclockers.com/bulldozer-architecture-explained/

The author of this "review" talks about CPU arch and is praised for
writing a "wonderful article that speaks the truth".  But sadly that
doesn't appear to be the case.  A colleague of mine is long-time friends
with another individual who is getting his Ph.D in computer architecture
and recently submit a paper to a journal (and was published/accepted)
which has published papers on things like RAID (when it was first
introduced as a concept/method), and hardware watchpoints.  Said
individual read the above "review" and described it as, quote, "the
worst article on computer architecture on the entire Internet".  One of
the amusing quotes (that got me laughing since I did understand it; my
understanding of CPUs on a silicon level is limited, I'm just an old
65xxx assembly programmer...) was how the article states "this is the
first time AMD has implemented branch prediction".  Sigh.

Here's the kicker: said individual immediately recognised that the
article was a near dry cut-and-paste from one of two commonl

Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server

[Patrick M. Hausen]

Hi, all,

Am 15.12.2011 um 12:18 schrieb Michael Ross:
> Following Steven Hartlands' suggestion,
> from one of my machines:
> 
> /usr/ports/sysutils/dmidecode/#sysctl -a | egrep "hw.vendor|hw.product"
> 
> /usr/ports/sysutils/dmidecode/#dmidecode -t 2
> # dmidecode 2.11
> SMBIOS 2.6 present.
> 
> Handle 0x0002, DMI type 2, 15 bytes
> Base Board Information
>Manufacturer: FUJITSU
>Product Name: D2759
>Version: S26361-D2759-A13 WGS04 GS02
>Serial Number: 35838599
>Asset Tag: -
>Features:
>Board is a hosting board
>Board is removable
>Location In Chassis: -
>Chassis Handle: 0x0003
>Type: Motherboard
>Contained Object Handles: 0


Without the need to install an additional port:

datatomb2# kenv
…
smbios.bios.reldate="11/03/2011"
smbios.bios.vendor="FUJITSU // American Megatrends Inc."
smbios.bios.version="V4.6.4.1 R1.18.0 for D3034-A1x"
smbios.chassis.maker="FUJITSU"
smbios.chassis.serial="YLAP004857"
smbios.chassis.tag="System Asset Tag"
smbios.chassis.version="RX100S7R2"
smbios.memory.enabled="8388608"
smbios.planar.maker="FUJITSU"
smbios.planar.product="D3034-A1"
smbios.planar.serial="LJ1B-P00996"
smbios.planar.version="S26361-D3034-A100 WGS01 GS02"
smbios.socket.enabled="1"
smbios.socket.populated="1"
smbios.system.maker="FUJITSU"
smbios.system.product="PRIMERGY RX100 S7"
smbios.system.serial="YLAP004857"
smbios.system.uuid="f0493081-f5ca-e011-b8a5-a1c4d143da5f"
smbios.system.version="GS02"
smbios.version="2.7"
…

Kind regards,
Patrick
-- 
punkt.de GmbH * Kaiserallee 13a * 76133 Karlsruhe
Tel. 0721 9109 0 * Fax 0721 9109 100
i...@punkt.de   http://www.punkt.de
Gf: Jürgen Egeling  AG Mannheim 108285



___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server

[Michael Ross]

Am 15.12.2011, 11:55 Uhr, schrieb Michael Larabel  
:



On 12/15/2011 04:41 AM, Michael Ross wrote:
Am 15.12.2011, 11:10 Uhr, schrieb Michael Larabel  
:



On 12/15/2011 02:48 AM, Michael Ross wrote:



Anyway these tests were performed on different hardware, FWIW.
And with different filesystems, different compilers, different GUIs...




No, the same hardware was used for each OS.



The picture under the heading "System Hardware / Software" does not  
reflect that.


Motherboard description differs, Chipset description for FreeBSD is  
empty.




I was the on that carried out the testing and know that it was on the  
same system.


No offense. I'm not doubting you.

But I didn't know this:

All of the testing, including the system tables, is fully automated.  
Under FreeBSD sometimes the parsing of some component strings isn't as  
nice as Linux and other supported operating systems by the Phoronix Test  
Suite. For the BSD motherboard string parsing it's grabbing  
hw.vendor/hw.product from sysctl.


so maybe you can understand how I got my impression.
NVidia Audio and Realtek Audio.
Looks different to me :-)


Is there a better place to read the motherboard DMI information from?



Following Steven Hartlands' suggestion,
from one of my machines:

/usr/ports/sysutils/dmidecode/#sysctl -a | egrep "hw.vendor|hw.product"

/usr/ports/sysutils/dmidecode/#dmidecode -t 2
# dmidecode 2.11
SMBIOS 2.6 present.

Handle 0x0002, DMI type 2, 15 bytes
Base Board Information
Manufacturer: FUJITSU
Product Name: D2759
Version: S26361-D2759-A13 WGS04 GS02
Serial Number: 35838599
Asset Tag: -
Features:
Board is a hosting board
Board is removable
Location In Chassis: -
Chassis Handle: 0x0003
Type: Motherboard
Contained Object Handles: 0


Nice. Didn't know about that.

Regards,

Michael
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server

[Michael Larabel]

On 12/15/2011 02:48 AM, Michael Ross wrote:
Am 15.12.2011, 08:32 Uhr, schrieb O. Hartmann 
:



Just saw this shot benchmark on Phoronix dot com today:

http://www.phoronix.com/scan.php?page=news_item&px=MTAyNzA

It may be worth to discuss the sad performance of FBSD in some parts of
the benchmark. A difference of a factor 10 or 100 is simply far beyond
disapointing, it is more than inacceptable and by just reading those
benchmarks, I'd like to drop thinking of using FreeBSD even as a backend
server in scientific and business environments. In detail, some of the
SciMark benches look disappointing.


Why SciMark?

SciMark FreeBSD : Oracle, Mflops

Composite   884.79 :  844.03 (Faster: FreeBSD)
FFT 236.17 :  213.65 (Faster: FreeBSD)
Jacobi  970.76 :  974.84 (Faster: Oracle)
Monte Carlo 443.00 :  246.27 (Faster: FreeBSD)
Sparse Matrix  1213.64 : 1228.22 (Faster: Oracle)
Dense LU   1560.39 : 1557.18(Faster: FreeBSD)


The threaded I/O results (Oracle outperforms FreeBSD by x10 on one, by 
x100 on another test)
or the disc TPS ( 486 : 3526 ) sure look worse and are worth looking 
into.



Anyway these tests were performed on different hardware, FWIW.
And with different filesystems, different compilers, different GUIs...




No, the same hardware was used for each OS.

In terms of the software, the stock software stack for each OS was used.

-- Michael




Regards,

Michael
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"



___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server

[Michael Larabel]

On 12/15/2011 04:41 AM, Michael Ross wrote:
Am 15.12.2011, 11:10 Uhr, schrieb Michael Larabel 
:



On 12/15/2011 02:48 AM, Michael Ross wrote:



Anyway these tests were performed on different hardware, FWIW.
And with different filesystems, different compilers, different GUIs...




No, the same hardware was used for each OS.



The picture under the heading "System Hardware / Software" does not 
reflect that.


Motherboard description differs, Chipset description for FreeBSD is 
empty.




I was the on that carried out the testing and know that it was on the 
same system.


All of the testing, including the system tables, is fully automated. 
Under FreeBSD sometimes the parsing of some component strings isn't as 
nice as Linux and other supported operating systems by the Phoronix Test 
Suite. For the BSD motherboard string parsing it's grabbing 
hw.vendor/hw.product from sysctl. Is there a better place to read the 
motherboard DMI information from?


-- Michael





Regards,

Michael



In terms of the software, the stock software stack for each OS was used.

-- Michael

___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"



___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server

[Michael Ross]

Am 15.12.2011, 11:10 Uhr, schrieb Michael Larabel  
:



On 12/15/2011 02:48 AM, Michael Ross wrote:



Anyway these tests were performed on different hardware, FWIW.
And with different filesystems, different compilers, different GUIs...




No, the same hardware was used for each OS.



The picture under the heading "System Hardware / Software" does not  
reflect that.


Motherboard description differs, Chipset description for FreeBSD is empty.


Regards,

Michael



In terms of the software, the stock software stack for each OS was used.

-- Michael

___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server

[Michael Ross]

Am 15.12.2011, 08:32 Uhr, schrieb O. Hartmann  
:



Just saw this shot benchmark on Phoronix dot com today:

http://www.phoronix.com/scan.php?page=news_item&px=MTAyNzA

It may be worth to discuss the sad performance of FBSD in some parts of
the benchmark. A difference of a factor 10 or 100 is simply far beyond
disapointing, it is more than inacceptable and by just reading those
benchmarks, I'd like to drop thinking of using FreeBSD even as a backend
server in scientific and business environments. In detail, some of the
SciMark benches look disappointing.


Why SciMark?

SciMark FreeBSD : Oracle, Mflops

Composite   884.79 :  844.03 (Faster: FreeBSD)
FFT 236.17 :  213.65 (Faster: FreeBSD)
Jacobi  970.76 :  974.84 (Faster: Oracle)
Monte Carlo 443.00 :  246.27 (Faster: FreeBSD)
Sparse Matrix  1213.64 : 1228.22 (Faster: Oracle)
Dense LU   1560.39 : 1557.18(Faster: FreeBSD)


The threaded I/O results (Oracle outperforms FreeBSD by x10 on one, by  
x100 on another test)

or the disc TPS ( 486 : 3526 ) sure look worse and are worth looking into.


Anyway these tests were performed on different hardware, FWIW.
And with different filesystems, different compilers, different GUIs...



Regards,

Michael
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server

[Adrian Chadd]

On 14 December 2011 23:32, O. Hartmann  wrote:
> Just saw this shot benchmark on Phoronix dot com today:
>
> http://www.phoronix.com/scan.php?page=news_item&px=MTAyNzA
>
> It may be worth to discuss the sad performance of FBSD in some parts of
> the benchmark. A difference of a factor 10 or 100 is simply far beyond

Well, the only way it's going to get fixed is if someone sits down,
replicates it, and starts to document exactly what it is that these
benchmarks are/aren't doing.

Sometimes it's because the benchmark is very much tickling things
incorrectly. In a lot of cases though, the benchmark is testing
something synthetic that Linux just happens to have micro-optimised.

So if you care about this a lot, someone needs to stand up, work with
Phronix to get some actual feedback about what's going on, and see if
it can be fixed. Maybe you'll find ULE is broken in some instances; I
bet you'll find something like "the disk driver is suboptimal." For
example, I remember seeing someone mess up a test because they split
their filesystems across raid5 boundaries, and this was hidden by the
choice of raid controller and stripe size. This made FreeBSD look
worse; when this was corrected for, it sped up far past Linux.



Adrian
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server

[O. Hartmann]

Just saw this shot benchmark on Phoronix dot com today:

http://www.phoronix.com/scan.php?page=news_item&px=MTAyNzA

It may be worth to discuss the sad performance of FBSD in some parts of
the benchmark. A difference of a factor 10 or 100 is simply far beyond
disapointing, it is more than inacceptable and by just reading those
benchmarks, I'd like to drop thinking of using FreeBSD even as a backend
server in scientific and business environments. In detail, some of the
SciMark benches look disappointing. The overall image can't help over
the fact that in C-Ray FreeBSD is better performing.

From the compiler, I'd like say there couldn't be a drop of more than 10
- 15% in performance - but not 10 or 100 times.

I'm just thinking about the discussion of SCHED_ULE and all the saur
spots we discussed when I stumbled over the test.

Regards,
Oliver



signature.asc
Description: OpenPGP digital signature