Re: Benchmarks results for FreeBSD 11

[Eric A. Borisch]

Plans for usage of LLVM's linker were just discussed on BSDNow
(from toolchain mailing list):

http://www.bsdnow.tv/episodes/2016_08_24-the_fresh_bsd_experience

https://lists.freebsd.org/pipermail/freebsd-toolchain/2016-August/002240.html

  - Eric

On Wednesday, August 31, 2016, Fernando Herrero Carrón 
wrote:

> That's great news! Thanks a lot!
>
> Excuse me if I drift off-topic, how about -flto? LTO requires both
> compiler and linker support, however clang3? from ports still calls the
> system linker 'ld'. LLVM has its own linker, 'lld', but it is not
> straightforward, AFAIK, to make clang3? call it.
>
> My trick up to now is to create a symbolic link in the current directory
> and have '.' in the PATH before anything else.
>
> ln -s /usr/local/llvm37/bin/lld ./ld
> clang37 -flto ...
>
> Am I missing something or is there a chance for improvement here?
>
> 2016-08-31 3:17 GMT+02:00 K. Macy  >:
>
>> On Tue, Aug 30, 2016 at 10:39 AM, Eric A. Borisch > > wrote:
>> > FWIW, in MacPorts, we patch clang such that it can find the (MacPorts
>> > provided) libomp headers and library. This lets -fopenmp "just work,"
>> > and configure scripts can do their job. The libomp headers and lib in
>> > dedicated sub-directories to minimize the impact of -fopenmp adding
>> > them to the include and link paths.
>> >
>> > It is a fairly minor patch, and shouldn't (tm) have any impact on
>> > clang executions without an openmp flag:
>> > https://trac.macports.org/browser/trunk/dports/lang/llvm-3.
>> 8/files/openmp-locations.patch
>> >
>> > To get a simple OpenMP test script to compile on FreeBSD, I currently
>> > need to pass (note I'm not the one using -lm):
>> >
>> >   clang38 -fopenmp -o test test.c -L /usr/local/llvm38/lib  -lm
>> >
>> > instead of (~ what configure will try)
>> >
>> >   clang38 -fopenmp -o test test.c
>> >
>> > (11.0RC2 w/ llvm38 installed via pkg)
>> >
>> > I'd love to see base include llvm's OpenMP support, but failing that,
>> > the one from ports should be made to work as configure scripts expect.
>> >
>> > And who knows, if it is there, maybe some items in base will start to
>> > use it. We've got a chicken-and-egg problem right now.
>>
>> Thanks for the patch. Johannes Dieterich hacked the llvm38 port in my
>> graphics branch and:
>>
>>
>> mmacy@armageddon [~|18:13|81] clang++ omp.cpp -fopenmp
>> mmacy@armageddon [~|18:13|81] cat omp.cpp
>> #include 
>> #include 
>> using namespace std;
>> int main(int argc, char** argv) {
>>
>>
>> #pragma omp parallel for default(shared)
>> for(int i = 0; i < 100; ++i){
>> #ifdef _OPENMP
>> cout << "WITH OPENMP " << i << endl;
>> #else
>> cout << "WITHOUT OPENMP " << i << endl;
>> #endif
>> }
>> return 0;
>> }
>> mmacy@armageddon [~|18:13|82] clang++38 omp.cpp -fopenmp
>> mmacy@armageddon [~|18:13|83] ./a.out| head -5
>> WITH OPENMP WITH OPENMP 84WITH OPENMP 92WITH OPENMP
>> 44
>> WITH OPENMP WITH OPENMP 9352WITH OPENMP
>> 76WITH OPENMP WITH OPENMP 0
>> WITH OPENMP
>>
>>
>>
>> Fingers crossed that this can get propagated to master and the
>> defaults for openblas, FFTW, and others can be made more sensible.
>>
>> -M
>> ___
>> freebsd-stable@freebsd.org
>>  mailing list
>> https://lists.freebsd.org/mailman/listinfo/freebsd-stable
>> To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org
>> "
>>
>
>
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: Benchmarks results for FreeBSD 11

[Fernando Herrero Carrón]

That's great news! Thanks a lot!

Excuse me if I drift off-topic, how about -flto? LTO requires both compiler
and linker support, however clang3? from ports still calls the system
linker 'ld'. LLVM has its own linker, 'lld', but it is not straightforward,
AFAIK, to make clang3? call it.

My trick up to now is to create a symbolic link in the current directory
and have '.' in the PATH before anything else.

ln -s /usr/local/llvm37/bin/lld ./ld
clang37 -flto ...

Am I missing something or is there a chance for improvement here?

2016-08-31 3:17 GMT+02:00 K. Macy :

> On Tue, Aug 30, 2016 at 10:39 AM, Eric A. Borisch 
> wrote:
> > FWIW, in MacPorts, we patch clang such that it can find the (MacPorts
> > provided) libomp headers and library. This lets -fopenmp "just work,"
> > and configure scripts can do their job. The libomp headers and lib in
> > dedicated sub-directories to minimize the impact of -fopenmp adding
> > them to the include and link paths.
> >
> > It is a fairly minor patch, and shouldn't (tm) have any impact on
> > clang executions without an openmp flag:
> > https://trac.macports.org/browser/trunk/dports/lang/
> llvm-3.8/files/openmp-locations.patch
> >
> > To get a simple OpenMP test script to compile on FreeBSD, I currently
> > need to pass (note I'm not the one using -lm):
> >
> >   clang38 -fopenmp -o test test.c -L /usr/local/llvm38/lib  -lm
> >
> > instead of (~ what configure will try)
> >
> >   clang38 -fopenmp -o test test.c
> >
> > (11.0RC2 w/ llvm38 installed via pkg)
> >
> > I'd love to see base include llvm's OpenMP support, but failing that,
> > the one from ports should be made to work as configure scripts expect.
> >
> > And who knows, if it is there, maybe some items in base will start to
> > use it. We've got a chicken-and-egg problem right now.
>
> Thanks for the patch. Johannes Dieterich hacked the llvm38 port in my
> graphics branch and:
>
>
> mmacy@armageddon [~|18:13|81] clang++ omp.cpp -fopenmp
> mmacy@armageddon [~|18:13|81] cat omp.cpp
> #include 
> #include 
> using namespace std;
> int main(int argc, char** argv) {
>
>
> #pragma omp parallel for default(shared)
> for(int i = 0; i < 100; ++i){
> #ifdef _OPENMP
> cout << "WITH OPENMP " << i << endl;
> #else
> cout << "WITHOUT OPENMP " << i << endl;
> #endif
> }
> return 0;
> }
> mmacy@armageddon [~|18:13|82] clang++38 omp.cpp -fopenmp
> mmacy@armageddon [~|18:13|83] ./a.out| head -5
> WITH OPENMP WITH OPENMP 84WITH OPENMP 92WITH OPENMP
> 44
> WITH OPENMP WITH OPENMP 9352WITH OPENMP
> 76WITH OPENMP WITH OPENMP 0
> WITH OPENMP
>
>
>
> Fingers crossed that this can get propagated to master and the
> defaults for openblas, FFTW, and others can be made more sensible.
>
> -M
> ___
> freebsd-stable@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-stable
> To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
>
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: Benchmarks results for FreeBSD 11

[K. Macy]

On Tue, Aug 30, 2016 at 10:39 AM, Eric A. Borisch  wrote:
> FWIW, in MacPorts, we patch clang such that it can find the (MacPorts
> provided) libomp headers and library. This lets -fopenmp "just work,"
> and configure scripts can do their job. The libomp headers and lib in
> dedicated sub-directories to minimize the impact of -fopenmp adding
> them to the include and link paths.
>
> It is a fairly minor patch, and shouldn't (tm) have any impact on
> clang executions without an openmp flag:
> https://trac.macports.org/browser/trunk/dports/lang/llvm-3.8/files/openmp-locations.patch
>
> To get a simple OpenMP test script to compile on FreeBSD, I currently
> need to pass (note I'm not the one using -lm):
>
>   clang38 -fopenmp -o test test.c -L /usr/local/llvm38/lib  -lm
>
> instead of (~ what configure will try)
>
>   clang38 -fopenmp -o test test.c
>
> (11.0RC2 w/ llvm38 installed via pkg)
>
> I'd love to see base include llvm's OpenMP support, but failing that,
> the one from ports should be made to work as configure scripts expect.
>
> And who knows, if it is there, maybe some items in base will start to
> use it. We've got a chicken-and-egg problem right now.

Thanks for the patch. Johannes Dieterich hacked the llvm38 port in my
graphics branch and:


mmacy@armageddon [~|18:13|81] clang++ omp.cpp -fopenmp
mmacy@armageddon [~|18:13|81] cat omp.cpp
#include 
#include 
using namespace std;
int main(int argc, char** argv) {


#pragma omp parallel for default(shared)
for(int i = 0; i < 100; ++i){
#ifdef _OPENMP
cout << "WITH OPENMP " << i << endl;
#else
cout << "WITHOUT OPENMP " << i << endl;
#endif
}
return 0;
}
mmacy@armageddon [~|18:13|82] clang++38 omp.cpp -fopenmp
mmacy@armageddon [~|18:13|83] ./a.out| head -5
WITH OPENMP WITH OPENMP 84WITH OPENMP 92WITH OPENMP
44
WITH OPENMP WITH OPENMP 9352WITH OPENMP
76WITH OPENMP WITH OPENMP 0
WITH OPENMP



Fingers crossed that this can get propagated to master and the
defaults for openblas, FFTW, and others can be made more sensible.

-M
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: Benchmarks results for FreeBSD 11

[Eric A. Borisch]

FWIW, in MacPorts, we patch clang such that it can find the (MacPorts
provided) libomp headers and library. This lets -fopenmp "just work,"
and configure scripts can do their job. The libomp headers and lib in
dedicated sub-directories to minimize the impact of -fopenmp adding
them to the include and link paths.

It is a fairly minor patch, and shouldn't (tm) have any impact on
clang executions without an openmp flag:
https://trac.macports.org/browser/trunk/dports/lang/llvm-3.8/files/openmp-locations.patch

To get a simple OpenMP test script to compile on FreeBSD, I currently
need to pass (note I'm not the one using -lm):

  clang38 -fopenmp -o test test.c -L /usr/local/llvm38/lib  -lm

instead of (~ what configure will try)

  clang38 -fopenmp -o test test.c

(11.0RC2 w/ llvm38 installed via pkg)

I'd love to see base include llvm's OpenMP support, but failing that,
the one from ports should be made to work as configure scripts expect.

And who knows, if it is there, maybe some items in base will start to
use it. We've got a chicken-and-egg problem right now.

 - Eric
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: Benchmarks results for FreeBSD 11

[Chris]

On 29 August 2016 at 09:30, Fernando Herrero Carrón  wrote:
> 2016-08-29 3:04 GMT+02:00 K. Macy :
>
>> > I'm writing from my cellphone away from my computer, so take this with a
>> > grain of salt:
>> >
>> > -L/usr/local/llvm38/lib
>>
>> You're missing the point. If your webserver crashes every other day,
>> the fact that you can run a batch job to restart it doesn't make it
>> OK.
>>
>>
> Fair enough, I misunderstood the point as "clang, even from ports, cannot
> do omp at all [from a shell]".
>
>
>> [...] Requiring additional tweaking to build on FreeBSD
>> or requiring users to install gcc is kind of underwhelming.
>>
>
> Isn't that precisely what the ports infrastructure is supposed to do? What
> about compiler.mk? If it can provide an extra library-dirs argument to the
> port's configure then we're not that bad. And still I agree, this is far
> from an ideal situation, and we speaking about finding a library, we have
> still not touched having clang use the LLVM linker...
> ___
> freebsd-stable@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-stable
> To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

I have not looked at the article yet. but yes if one compiles FreeBSD
ports using a default make.conf (empty file) then the ports will not
be built optimised.
I dont go all out on optimisation like some people do but I consider
basic optimisation to be a modern version of gcc combined with the cpu
native flag.  Which will auto detect the processor used and add the
right flags correctly.  I dont fiddle with -O2 -O3 etc.

However there is a flipside to this on performance, ubuntu packages by
default are now built with PIE, BIND_NOW, SMACK, RELRO etc.  These
will all slow down performance but are modern expected hardening.
FreeBSD I think still does not do this by default? and as such ubuntu
is managing to beat FreeBSD whilst also adding exploit mitigations
which is impressive.

It is a real shame the default ports compile and base has not been
looked into but instead it all has been kept on conservative approach
which is no exploit protection and to compile for the lowest
denominator like a 486 cpu.  Hardened bsd has done some good work on
this but I see they were forced to fork away because their changes
were rejected on the base system.

As a final note I assumed clang no longer has a noticeable
disadvantage vs gcc, if it does then that's bad news for the base
files.

If I Am wrong on the above please feel free to correct me.

Chris
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: Benchmarks results for FreeBSD 11

[Fernando Herrero Carrón]

2016-08-29 3:04 GMT+02:00 K. Macy :

> > I'm writing from my cellphone away from my computer, so take this with a
> > grain of salt:
> >
> > -L/usr/local/llvm38/lib
>
> You're missing the point. If your webserver crashes every other day,
> the fact that you can run a batch job to restart it doesn't make it
> OK.
>
>
Fair enough, I misunderstood the point as "clang, even from ports, cannot
do omp at all [from a shell]".


> [...] Requiring additional tweaking to build on FreeBSD
> or requiring users to install gcc is kind of underwhelming.
>

Isn't that precisely what the ports infrastructure is supposed to do? What
about compiler.mk? If it can provide an extra library-dirs argument to the
port's configure then we're not that bad. And still I agree, this is far
from an ideal situation, and we speaking about finding a library, we have
still not touched having clang use the LLVM linker...
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: Benchmarks results for FreeBSD 11

[K. Macy]

> I'm writing from my cellphone away from my computer, so take this with a
> grain of salt:
>
> -L/usr/local/llvm38/lib

You're missing the point. If your webserver crashes every other day,
the fact that you can run a batch job to restart it doesn't make it
OK.

No software written to date assumes the need to find libomp. It will
either configure itself to not use openmp or not build without
additional tweaking. Requiring additional tweaking to build on FreeBSD
or requiring users to install gcc is kind of underwhelming.

-M
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: Benchmarks results for FreeBSD 11

[Fernando Herrero Carrón]

El 29 ago. 2016 12:36 a. m., "K. Macy"  escribió:
>
> On Sun, Aug 28, 2016 at 2:01 PM, K. Macy  wrote:
> >>>
> >>
> >> With 11, one can even simply install devel/openmp which will only
install the libopenmp bits from llvm, and after that, base cc can do openmp.
> >
> > This isn't really useful unless the clang in base knows where to find
> > libomp. Considering that even the devel/llvm ports aren't configured
> > properly for that I don't see -fopenmp working. Without patching
> > makefiles or configure. So although this is better than having to
> > installing a complete copy of the compiler for the sake of a single
> > library it's still forcing FreeBSD users to jump through extra hoops
> > that they don't have to with gcc or on other platforms.
>
> I just tried this on Ubuntu and it looks like even there clang doesn't
> invoke ld correctly to make -fopenmp work. Both icc and gcc do this
> automatically. Thus I wonder if anyone is actually using openmp in
> earnest with clang.
>
>
> mmacy@pandemonium:~$ clang++ -fopenmp omp.cpp
> /usr/bin/ld: cannot find -liomp5
> clang-3.5: error: linker command failed with exit code 1 (use -v to
> see invocation)
> mmacy@pandemonium:~$ clang++-3.8 -fopenmp omp.cpp
> /usr/bin/ld: cannot find -lomp
> clang: error: linker command failed with exit code 1 (use -v to see
invocation)
> mmacy@pandemonium:~$ g++ -fopenmp omp.cpp
> mmacy@pandemonium:~$ g++-4.9 -fopenmp omp.cpp
> mmacy@pandemonium:~$
>

I'm writing from my cellphone away from my computer, so take this with a
grain of salt:

-L/usr/local/llvm38/lib
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: Benchmarks results for FreeBSD 11

[K. Macy]

On Sun, Aug 28, 2016 at 2:01 PM, K. Macy  wrote:
>>>
>>
>> With 11, one can even simply install devel/openmp which will only install 
>> the libopenmp bits from llvm, and after that, base cc can do openmp.
>
> This isn't really useful unless the clang in base knows where to find
> libomp. Considering that even the devel/llvm ports aren't configured
> properly for that I don't see -fopenmp working. Without patching
> makefiles or configure. So although this is better than having to
> installing a complete copy of the compiler for the sake of a single
> library it's still forcing FreeBSD users to jump through extra hoops
> that they don't have to with gcc or on other platforms.

I just tried this on Ubuntu and it looks like even there clang doesn't
invoke ld correctly to make -fopenmp work. Both icc and gcc do this
automatically. Thus I wonder if anyone is actually using openmp in
earnest with clang.


mmacy@pandemonium:~$ clang++ -fopenmp omp.cpp
/usr/bin/ld: cannot find -liomp5
clang-3.5: error: linker command failed with exit code 1 (use -v to
see invocation)
mmacy@pandemonium:~$ clang++-3.8 -fopenmp omp.cpp
/usr/bin/ld: cannot find -lomp
clang: error: linker command failed with exit code 1 (use -v to see invocation)
mmacy@pandemonium:~$ g++ -fopenmp omp.cpp
mmacy@pandemonium:~$ g++-4.9 -fopenmp omp.cpp
mmacy@pandemonium:~$

-M
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: Benchmarks results for FreeBSD 11

[K. Macy]

>>
>
> With 11, one can even simply install devel/openmp which will only install the 
> libopenmp bits from llvm, and after that, base cc can do openmp.

This isn't really useful unless the clang in base knows where to find
libomp. Considering that even the devel/llvm ports aren't configured
properly for that I don't see -fopenmp working. Without patching
makefiles or configure. So although this is better than having to
installing a complete copy of the compiler for the sake of a single
library it's still forcing FreeBSD users to jump through extra hoops
that they don't have to with gcc or on other platforms.
-M
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: Benchmarks results for FreeBSD 11

[K. Macy]

On Sunday, August 28, 2016, Brandon Allbery  wrote:

> On Sun, Aug 28, 2016 at 1:57 PM, K. Macy  > wrote:
>
>> Can you point to other platforms where the default system compiler has
>> disabled functionality?
>>
>
> You have to install LLVM from elsewhere to get full functionality on OS X:
> Apple only ships the parts that Xcode cares about. OTOH, this pretty much
> only impacts things that want to use LLVM IR. (On the gripping hand, for
> some people that is about as relevant as OpenMP.)
>
> There were some late SunOS 4 releases where you had (p)cc in the base,
> with various tools that people expect missing, and had to install SunSoft C
> to get a decent compiler and all the tools. (They removed almost all of it
> from Solaris 2, of course.)
>
>
Interesting. The primary OSX user doesn't use a compiler. The SunOS example
strikes me as being the closest. And it's because they wanted to sell you
extra software. Here it's the notion that there is a compiler for "base"
and then ports are "everything else".

Ultimately, from the user's perspective "base" will just be a particularly well
integrated set of packages - including, rather idiosyncratically, a
compiler that only has the features required by that package set.

-M

> --
> brandon s allbery kf8nh   sine nomine
> associates
> allber...@gmail.com 
>  ballb...@sinenomine.net
> 
> unix, openafs, kerberos, infrastructure, xmonad
> http://sinenomine.net
>
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: Benchmarks results for FreeBSD 11

[Brandon Allbery]

On Sun, Aug 28, 2016 at 1:57 PM, K. Macy  wrote:

> Can you point to other platforms where the default system compiler has
> disabled functionality?
>

You have to install LLVM from elsewhere to get full functionality on OS X:
Apple only ships the parts that Xcode cares about. OTOH, this pretty much
only impacts things that want to use LLVM IR. (On the gripping hand, for
some people that is about as relevant as OpenMP.)

There were some late SunOS 4 releases where you had (p)cc in the base, with
various tools that people expect missing, and had to install SunSoft C to
get a decent compiler and all the tools. (They removed almost all of it
from Solaris 2, of course.)

-- 
brandon s allbery kf8nh   sine nomine associates
allber...@gmail.com  ballb...@sinenomine.net
unix, openafs, kerberos, infrastructure, xmonadhttp://sinenomine.net
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: Benchmarks results for FreeBSD 11

[K. Macy]

On Sunday, August 28, 2016, Dimitry Andric  wrote:

> On 28 Aug 2016, at 02:10, K. Macy >
> wrote:
> >
> >> The problem here is that Phoronix took a Beta version of FreeBSD 11.
> >> Beta versions have a lot of debugging (malloc, invariants, witness)
> >> options enabled which make it significantly slower than release
> >> versions. This is even obviously when you run a Beta as a desktop. It
> >> just feels much slower.
> >
> >
> > I don't know what was going on in these particular tests, but in a
> > more recent benchmarking run
> > -https://www.phoronix.com/scan.php?page=article=
> freebsd11-clang-gcc=1
> > - you're seeing the result of openmp being disabled in base. The clang
> > maintainer for src refuses to include libomp as required for -fopenmp
> > because nothing in base requires it.
>
> Come on, this is nonsense.  I have indicated earlier that I would have
> liked to import openmp into base, but this was shot down precisely for
> that reason: nothing in base uses it.
>
>
So for now, the solution is simply: install one of the llvm ports, and
> use it.  These have configuration setting to install every optional
> component from the LLVM project.


It's hardly nonsense. I didn't say that it couldn't be made to work by
installing the right ports, with the right options, if you pass the right
extra library paths (-fopenmp is sufficient on other platforms). I said
that it is not present in base. Thus out of the box performance is what we
see.

Can you point to other platforms where the default system compiler has
disabled functionality? I think the whole concept of "base" may be
confusing things. Users don't care about what is base and what isn't. The
simple fact is you install a compiler with missing functionality, requiring
extra steps by the user that only make sense if you're a committer.


-M


>
> -Dimitry
>
>
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: Benchmarks results for FreeBSD 11

[Matthieu Volat]

On Sun, 28 Aug 2016 14:55:37 +0200
Dimitry Andric  wrote:

> On 28 Aug 2016, at 02:10, K. Macy  wrote:
> >   
> >> The problem here is that Phoronix took a Beta version of FreeBSD 11.
> >> Beta versions have a lot of debugging (malloc, invariants, witness)
> >> options enabled which make it significantly slower than release
> >> versions. This is even obviously when you run a Beta as a desktop. It
> >> just feels much slower.  
> > 
> > 
> > I don't know what was going on in these particular tests, but in a
> > more recent benchmarking run
> > -https://www.phoronix.com/scan.php?page=article=freebsd11-clang-gcc=1
> > - you're seeing the result of openmp being disabled in base. The clang
> > maintainer for src refuses to include libomp as required for -fopenmp
> > because nothing in base requires it.  
> 
> Come on, this is nonsense.  I have indicated earlier that I would have
> liked to import openmp into base, but this was shot down precisely for
> that reason: nothing in base uses it.
> 
> So for now, the solution is simply: install one of the llvm ports, and
> use it.  These have configuration setting to install every optional
> component from the LLVM project.
> 
> -Dimitry
> 

With 11, one can even simply install devel/openmp which will only install the 
libopenmp bits from llvm, and after that, base cc can do openmp.

-- Matthieu Volat 


pgpMac1sRLat2.pgp
Description: OpenPGP digital signature

Re: Benchmarks results for FreeBSD 11

[Johannes Dieterich]

On Sun, Aug 28, 2016 at 11:16 AM, Fernando Herrero Carrón
 wrote:
> El 28/8/2016 14:56, "Dimitry Andric"  escribió:
>>
>> On 28 Aug 2016, at 02:10, K. Macy  wrote:
>> >
>> >> The problem here is that Phoronix took a Beta version of FreeBSD 11.
>> >> Beta versions have a lot of debugging (malloc, invariants, witness)
>> >> options enabled which make it significantly slower than release
>> >> versions. This is even obviously when you run a Beta as a desktop. It
>> >> just feels much slower.
>> >
>> >
>> > I don't know what was going on in these particular tests, but in a
>> > more recent benchmarking run
>> >
>> > -https://www.phoronix.com/scan.php?page=article=freebsd11-clang-gcc=1
>> > - you're seeing the result of openmp being disabled in base. The clang
>> > maintainer for src refuses to include libomp as required for -fopenmp
>> > because nothing in base requires it.
>>
>> Come on, this is nonsense.  I have indicated earlier that I would have
>> liked to import openmp into base, but this was shot down precisely for
>> that reason: nothing in base uses it.
>>
>> So for now, the solution is simply: install one of the llvm ports, and
>> use it.  These have configuration setting to install every optional
>> component from the LLVM project.
>>
>> -Dimitry
>>
>
> How does the port infrastructure handle openmp-enabled ports (those with an
> openmp option) then? Is an omp-capable compiler automatically pulled in or
> is openmp ignored unless the port explicitely requests one from ports?
If you set compiler:openmp, it currently defaults to pull in lang/gcc.
So, even if the port would be perfectly happy to compile with llvm, we
rely on gcc. If the port is an important library such as openblas or
fftw3, this then in turn causes all the well documented issues for
software using that library. So we should rely on, at the very least,
lang/llvm38 to be used for these cases. However, that does not work
out of the box as those ports do not find their own libraries (libomp
is the important one here) by themselves, requiring alterations to
every single make system (as the rest of the world correctly assumes
that -fopenmp enables OpenMP) to add the link path. If that vaguely
reminds you of the -Wl,-rpath=/usr/local/lib/gcc48 insanity, you'd be
right.

All of this may be why important ports (openblas, fftw3, ImageMagick)
do not enable OpenMP by default, leaving us with sub-par performance
not only in benchmarks such as the one cited above.

It is a dreadful situation in my opinion and is not helped by the fact
that we do not even acknowledge there to be an issue. I happen to
require working OpenMP support on a day-to-day basis, are required to
jump through hoops to get it, and it seems decisions are made without
assessing the impact of them for people like me.

Sorry

Johannes
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: Benchmarks results for FreeBSD 11

[Fernando Herrero Carrón]

El 28/8/2016 14:56, "Dimitry Andric"  escribió:
>
> On 28 Aug 2016, at 02:10, K. Macy  wrote:
> >
> >> The problem here is that Phoronix took a Beta version of FreeBSD 11.
> >> Beta versions have a lot of debugging (malloc, invariants, witness)
> >> options enabled which make it significantly slower than release
> >> versions. This is even obviously when you run a Beta as a desktop. It
> >> just feels much slower.
> >
> >
> > I don't know what was going on in these particular tests, but in a
> > more recent benchmarking run
> > -
https://www.phoronix.com/scan.php?page=article=freebsd11-clang-gcc=1
> > - you're seeing the result of openmp being disabled in base. The clang
> > maintainer for src refuses to include libomp as required for -fopenmp
> > because nothing in base requires it.
>
> Come on, this is nonsense.  I have indicated earlier that I would have
> liked to import openmp into base, but this was shot down precisely for
> that reason: nothing in base uses it.
>
> So for now, the solution is simply: install one of the llvm ports, and
> use it.  These have configuration setting to install every optional
> component from the LLVM project.
>
> -Dimitry
>

How does the port infrastructure handle openmp-enabled ports (those with an
openmp option) then? Is an omp-capable compiler automatically pulled in or
is openmp ignored unless the port explicitely requests one from ports?

Fernando
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: Benchmarks results for FreeBSD 11

[Dimitry Andric]

On 28 Aug 2016, at 02:10, K. Macy  wrote:
> 
>> The problem here is that Phoronix took a Beta version of FreeBSD 11.
>> Beta versions have a lot of debugging (malloc, invariants, witness)
>> options enabled which make it significantly slower than release
>> versions. This is even obviously when you run a Beta as a desktop. It
>> just feels much slower.
> 
> 
> I don't know what was going on in these particular tests, but in a
> more recent benchmarking run
> -https://www.phoronix.com/scan.php?page=article=freebsd11-clang-gcc=1
> - you're seeing the result of openmp being disabled in base. The clang
> maintainer for src refuses to include libomp as required for -fopenmp
> because nothing in base requires it.

Come on, this is nonsense.  I have indicated earlier that I would have
liked to import openmp into base, but this was shot down precisely for
that reason: nothing in base uses it.

So for now, the solution is simply: install one of the llvm ports, and
use it.  These have configuration setting to install every optional
component from the LLVM project.

-Dimitry



signature.asc
Description: Message signed with OpenPGP using GPGMail

Re: Benchmarks results for FreeBSD 11

[K. Macy]

> The problem here is that Phoronix took a Beta version of FreeBSD 11.
> Beta versions have a lot of debugging (malloc, invariants, witness)
> options enabled which make it significantly slower than release
> versions. This is even obviously when you run a Beta as a desktop. It
> just feels much slower.


I don't know what was going on in these particular tests, but in a
more recent benchmarking run
-https://www.phoronix.com/scan.php?page=article=freebsd11-clang-gcc=1
- you're seeing the result of openmp being disabled in base. The clang
maintainer for src refuses to include libomp as required for -fopenmp
because nothing in base requires it. I certainly can't speak for the
community as a whole - but based on my experience when discussing new
features that adversely impact networking performance my impression is
that out of the box performance is generally not a priority.

-M
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: Benchmarks results for FreeBSD 11

[Jakub Lach]

Yes, when going from FreeBSD 6 to 7 on a UP system. The latter was
a little slower.



--
View this message in context: 
http://freebsd.1045724.n5.nabble.com/Benchmarks-results-for-FreeBSD-11-tp6123994p6124925.html
Sent from the freebsd-stable mailing list archive at Nabble.com.
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: Benchmarks results for FreeBSD 11

[Matthieu Volat]

On Fri, 26 Aug 2016 13:20:59 +0800
Erich Dollansky  wrote:

> Hi,
> 
> On Wed, 24 Aug 2016 13:12:24 +0200
> Fernando Herrero Carrón  wrote:
> 
> > Many ports offer an option to compile with optimized cflags. See for
> > instance http://www.freshports.org/multimedia/ffmpeg:
> > 
> >  OPTIMIZED_CFLAGS=off: Use extra compiler optimizations
> > 
> > though:
> > 
> >  SSE=on: Use SSE optimized routines
> > 
> > It turns out that optimization options are usually off by default, so  
> 
> if we assume Micheal has built the test suite from ports, it is all to
> the defaults. I also do not see a compiler option. With the compiler
> option, the same compiler could be used on all platforms.

I think at some point, he mentionned that PC-BSD could/would be the basis for 
the FreeBSD benchmarks at phoronix (due to deployment considerations).

In that case, the options used in PC-BSD packages should be taken into 
account...

> 
> I am not even sure if the same compiler was used on the Linux platforms.

No, when he does OSes benchmarks, he make them compete with the compiler the OS 
provides. But he sometimes do compiler benchmarks.

> 
> Erich
> ___
> freebsd-stable@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-stable
> To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
> 


-- 
Matthieu Volat 
tel: 06 84 54 39 43
www: 


pgpNHS4wfN_Vr.pgp
Description: OpenPGP digital signature

Re: Benchmarks results for FreeBSD 11

[Erich Dollansky]

Hi,

On Wed, 24 Aug 2016 13:12:24 +0200
Fernando Herrero Carrón  wrote:

> Many ports offer an option to compile with optimized cflags. See for
> instance http://www.freshports.org/multimedia/ffmpeg:
> 
>  OPTIMIZED_CFLAGS=off: Use extra compiler optimizations
> 
> though:
> 
>  SSE=on: Use SSE optimized routines
> 
> It turns out that optimization options are usually off by default, so

if we assume Micheal has built the test suite from ports, it is all to
the defaults. I also do not see a compiler option. With the compiler
option, the same compiler could be used on all platforms.

I am not even sure if the same compiler was used on the Linux platforms.

Erich
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: Benchmarks results for FreeBSD 11

[Fernando Herrero Carrón]

Many ports offer an option to compile with optimized cflags. See for
instance http://www.freshports.org/multimedia/ffmpeg:

 OPTIMIZED_CFLAGS=off: Use extra compiler optimizations

though:

 SSE=on: Use SSE optimized routines

It turns out that optimization options are usually off by default, so
binary packages will be built without them. It is an interesting question
whether they should be enabled by default. On the other hand, a generic
distribution cannot assume specific processor capabilities, for instance
the newest SSE or AVX, so binary distributed packages will, in general,
never be as performant as specifically built ones (some software packages
do runtime CPU detection though). That is one of the reasons why I use
FreeBSD and why I always build my own ports.

Cheers,
Fernando

2016-08-24 12:07 GMT+02:00 Andrea Brancatelli :

> Il 2016-08-23 22:55 Erich Dollansky ha scritto:
>
> > The problem here is that Phoronix took a Beta version of FreeBSD
> > 11. Beta versions have a lot of debugging (malloc, invariants,
> > witness) options enabled which make it significantly slower than
> > release versions. This is even obviously when you run a Beta as a
> > desktop. It just feels much slower.
> > No.
> > All debugs in amd64 is off at time of BETA.
>
> Ok, then FreeBSD is just slower...
> you missed the main point. The test was done with applications compiled
> with unknown options. If you want to find out the impact of an
> operating system on you application you have to use the same compiler
> with the same options on all platforms. Basically, it was a compiler
> test named operating system benchmark.
>
> OK, now I get your point. You mean he's doing an Operative System test
> using an application, that is something that involve an additional
> complexity.
>
> I get your point, but, er... I suppose he's using pkg to install the
> ports or apt-get to install the packages. I mean I suppose he's lazy
> enough to compare binaries vs binaries.
>
> Now what you're telling me is "it's not a kernel comparison, it's an
> application over a kernel comparison" and I agree with you, but why
> should a casual reader be interested in a kernel comparison? Or, in
> general, what would you do with a kernel without an application? :)
>
> Sorry, I don't want to sound polemic, neither start a flame or anything,
> I'm just curious... And I understand clearly the philosophical
> differences between a Base BSD system and a Linux Distribution, but
> again I'm a simple person, I just stick the FreeBSD cd in, install and
> then do pkg install unzip. If it's slower than my colleague using Ubuntu
> and doing the same thing I just feel sad :)
>
> So the first outcome everyone thought about was "The RC is slower
> because of debugging?". No. Then the next question that pops into my
> mind is "The ports are slow because the binaries are compiled with the
> wrong options?".
>
> Thanks for your clarifications.
> ___
> freebsd-stable@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-stable
> To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
>
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: Benchmarks results for FreeBSD 11

[Andrea Brancatelli]

Il 2016-08-23 22:55 Erich Dollansky ha scritto:

> The problem here is that Phoronix took a Beta version of FreeBSD
> 11. Beta versions have a lot of debugging (malloc, invariants,
> witness) options enabled which make it significantly slower than
> release versions. This is even obviously when you run a Beta as a
> desktop. It just feels much slower.   
> No.
> All debugs in amd64 is off at time of BETA.

Ok, then FreeBSD is just slower... 
you missed the main point. The test was done with applications compiled
with unknown options. If you want to find out the impact of an
operating system on you application you have to use the same compiler
with the same options on all platforms. Basically, it was a compiler
test named operating system benchmark.

OK, now I get your point. You mean he's doing an Operative System test
using an application, that is something that involve an additional
complexity. 

I get your point, but, er... I suppose he's using pkg to install the
ports or apt-get to install the packages. I mean I suppose he's lazy
enough to compare binaries vs binaries. 

Now what you're telling me is "it's not a kernel comparison, it's an
application over a kernel comparison" and I agree with you, but why
should a casual reader be interested in a kernel comparison? Or, in
general, what would you do with a kernel without an application? :) 

Sorry, I don't want to sound polemic, neither start a flame or anything,
I'm just curious... And I understand clearly the philosophical
differences between a Base BSD system and a Linux Distribution, but
again I'm a simple person, I just stick the FreeBSD cd in, install and
then do pkg install unzip. If it's slower than my colleague using Ubuntu
and doing the same thing I just feel sad :) 

So the first outcome everyone thought about was "The RC is slower
because of debugging?". No. Then the next question that pops into my
mind is "The ports are slow because the binaries are compiled with the
wrong options?". 

Thanks for your clarifications.
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: Benchmarks results for FreeBSD 11

[Erich Dollansky]

Hi,

On Tue, 23 Aug 2016 13:01:59 +0200
Lars Engels  wrote:

> On Mon, Aug 22, 2016 at 03:02:15PM +0300, Slawa Olhovchenkov wrote:
> > On Mon, Aug 22, 2016 at 12:14:23PM +0200, Lars Engels wrote:
> >   
> > > On Mon, Aug 22, 2016 at 11:48:46AM +0200, Andrea Brancatelli
> > > wrote:  
> > > > Il 2016-08-21 08:45 Erich Dollansky ha scritto:
> > > >   
> > > > > I am sure that some know of this site:
> > > > > 
> > > > > http://www.phoronix.com/scan.php?page=article=2bsd-7linux-bench=4
> > > > > 
> > > > > I wonder about the results for FreeBSD. As I do not have 11
> > > > > on my machines, a stupid question. Are there still some
> > > > > debugging aids enabled in 11?   
> > > > > They're off in those versions, but did note compiler (and
> > > > > compiler args) differences between within most tests (See
> > > > > attachments) as you mentioned.  
> > > >  the benchmark then compares the off-the-shelve distributions. 
> > > > 
> > > > Excuse me, as a casual reader of the list, I don't get this
> > > > "critique". 
> > > > 
> > > > I never recompile my installations, I just use them from the
> > > > installation CD (as probably 90% of the rest of the world), so
> > > > I don't get what is wrong with the approach of comparing an
> > > > out-of-the-box FreeBSD 11 with an out-of-the-box Ubuntu
> > > > whatever. 
> > > > 
> > > > If FreeBSD 11 "out-the-box" performs slow because the standard
> > > > compilers options aren't good it's not a problem with the
> > > > benchmarking platform but with the default CD compiling
> > > > options. 
> > > > 
> > > > Am I getting it wrong?   
> > > 
> > > The problem here is that Phoronix took a Beta version of FreeBSD
> > > 11. Beta versions have a lot of debugging (malloc, invariants,
> > > witness) options enabled which make it significantly slower than
> > > release versions. This is even obviously when you run a Beta as a
> > > desktop. It just feels much slower.  
> > 
> > No.
> > All debugs in amd64 is off at time of BETA.  
> 
> Ok, then FreeBSD is just slower...

you missed the main point. The test was done with applications compiled
with unknown options. If you want to find out the impact of an
operating system on you application you have to use the same compiler
with the same options on all platforms. Basically, it was a compiler
test named operating system benchmark.

Erich
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: Benchmarks results for FreeBSD 11

[Lars Engels]

On Mon, Aug 22, 2016 at 03:02:15PM +0300, Slawa Olhovchenkov wrote:
> On Mon, Aug 22, 2016 at 12:14:23PM +0200, Lars Engels wrote:
> 
> > On Mon, Aug 22, 2016 at 11:48:46AM +0200, Andrea Brancatelli wrote:
> > > Il 2016-08-21 08:45 Erich Dollansky ha scritto:
> > > 
> > > > I am sure that some know of this site:
> > > > 
> > > > http://www.phoronix.com/scan.php?page=article=2bsd-7linux-bench=4
> > > > 
> > > > I wonder about the results for FreeBSD. As I do not have 11 on my 
> > > > machines, a stupid question. Are there still some debugging aids 
> > > > enabled in 11?   
> > > > They're off in those versions, but did note compiler (and compiler
> > > > args) differences between within most tests (See attachments) as you
> > > > mentioned.
> > >  the benchmark then compares the off-the-shelve distributions. 
> > > 
> > > Excuse me, as a casual reader of the list, I don't get this "critique". 
> > > 
> > > I never recompile my installations, I just use them from the
> > > installation CD (as probably 90% of the rest of the world), so I don't
> > > get what is wrong with the approach of comparing an out-of-the-box
> > > FreeBSD 11 with an out-of-the-box Ubuntu whatever. 
> > > 
> > > If FreeBSD 11 "out-the-box" performs slow because the standard compilers
> > > options aren't good it's not a problem with the benchmarking platform
> > > but with the default CD compiling options. 
> > > 
> > > Am I getting it wrong? 
> > 
> > The problem here is that Phoronix took a Beta version of FreeBSD 11.
> > Beta versions have a lot of debugging (malloc, invariants, witness)
> > options enabled which make it significantly slower than release
> > versions. This is even obviously when you run a Beta as a desktop. It
> > just feels much slower.
> 
> No.
> All debugs in amd64 is off at time of BETA.

Ok, then FreeBSD is just slower...


pgpHkCIbmimuX.pgp
Description: PGP signature

Re: Benchmarks results for FreeBSD 11

[George Mitchell]

On 08/23/16 02:32, Erich Dollansky wrote:
> Hi,
> 
> On Mon, 22 Aug 2016 03:22:35 +
> Kubilay Kocak  wrote:
> 
>> On Mon, 22 Aug 2016, 11:31 AM Mark Linimon 
>> wrote:
>>
>>> On Mon, Aug 22, 2016 at 09:57:24AM +1000, Dewayne Geraghty wrote:  
 unless knowledgable people respond publicly and/or in the phoronix
 forums [...] this interpretation of reality will be fixed in
 decision- makers' minds and consequently the uptake (and support)
 of FreeBSD.  
>>>
>>> IIRC this has been done before and hasn't really been productive.
>>> OTOH I don't recall the details.
>>>
>>> FreeBSD hasn't had a benchmarking guru since Kris Kennaway retired
>>> from working on FreeBSD.  
>>
>>
>> Michael has reached out off-list (thanks!) If anyone else is
>> interested, I'd be happy to create a dedicated IRC channel on
>> freenode to widen and focus the freebsd performance discussion net
>> and doc any outcomes/notes in the FreeBSD Wiki.
>>
> this is good. So, how can we help him to get things straight the next
> time?
> 
> Erich
> ___
> freebsd-stable@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-stable
> To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
> 
Off-the-wall note: has anyone tried comparing SCHED_ULE to SCHED_BSD?
-- George
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: Benchmarks results for FreeBSD 11

[Erich Dollansky]

Hi,

On Mon, 22 Aug 2016 03:22:35 +
Kubilay Kocak  wrote:

> On Mon, 22 Aug 2016, 11:31 AM Mark Linimon 
> wrote:
> 
> > On Mon, Aug 22, 2016 at 09:57:24AM +1000, Dewayne Geraghty wrote:  
> > > unless knowledgable people respond publicly and/or in the phoronix
> > > forums [...] this interpretation of reality will be fixed in
> > > decision- makers' minds and consequently the uptake (and support)
> > > of FreeBSD.  
> >
> > IIRC this has been done before and hasn't really been productive.
> > OTOH I don't recall the details.
> >
> > FreeBSD hasn't had a benchmarking guru since Kris Kennaway retired
> > from working on FreeBSD.  
> 
> 
> Michael has reached out off-list (thanks!) If anyone else is
> interested, I'd be happy to create a dedicated IRC channel on
> freenode to widen and focus the freebsd performance discussion net
> and doc any outcomes/notes in the FreeBSD Wiki.
> 
this is good. So, how can we help him to get things straight the next
time?

Erich
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: Benchmarks results for FreeBSD 11

[Jakub Lach]

Unixbench is slower for me since switching to 11-STABLE from 10-STABLE, 
due to triple drop of 'Pipe Throughput'. Rest looks the same or better. 
That's all I know.



--
View this message in context: 
http://freebsd.1045724.n5.nabble.com/Benchmarks-results-for-FreeBSD-11-tp6123994p6124304.html
Sent from the freebsd-stable mailing list archive at Nabble.com.
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: Benchmarks results for FreeBSD 11

[Slawa Olhovchenkov]

On Mon, Aug 22, 2016 at 12:14:23PM +0200, Lars Engels wrote:

> On Mon, Aug 22, 2016 at 11:48:46AM +0200, Andrea Brancatelli wrote:
> > Il 2016-08-21 08:45 Erich Dollansky ha scritto:
> > 
> > > I am sure that some know of this site:
> > > 
> > > http://www.phoronix.com/scan.php?page=article=2bsd-7linux-bench=4
> > > 
> > > I wonder about the results for FreeBSD. As I do not have 11 on my 
> > > machines, a stupid question. Are there still some debugging aids 
> > > enabled in 11?   
> > > They're off in those versions, but did note compiler (and compiler
> > > args) differences between within most tests (See attachments) as you
> > > mentioned.
> >  the benchmark then compares the off-the-shelve distributions. 
> > 
> > Excuse me, as a casual reader of the list, I don't get this "critique". 
> > 
> > I never recompile my installations, I just use them from the
> > installation CD (as probably 90% of the rest of the world), so I don't
> > get what is wrong with the approach of comparing an out-of-the-box
> > FreeBSD 11 with an out-of-the-box Ubuntu whatever. 
> > 
> > If FreeBSD 11 "out-the-box" performs slow because the standard compilers
> > options aren't good it's not a problem with the benchmarking platform
> > but with the default CD compiling options. 
> > 
> > Am I getting it wrong? 
> 
> The problem here is that Phoronix took a Beta version of FreeBSD 11.
> Beta versions have a lot of debugging (malloc, invariants, witness)
> options enabled which make it significantly slower than release
> versions. This is even obviously when you run a Beta as a desktop. It
> just feels much slower.

No.
All debugs in amd64 is off at time of BETA.
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: Benchmarks results for FreeBSD 11

[Lars Engels]

On Mon, Aug 22, 2016 at 11:48:46AM +0200, Andrea Brancatelli wrote:
> Il 2016-08-21 08:45 Erich Dollansky ha scritto:
> 
> > I am sure that some know of this site:
> > 
> > http://www.phoronix.com/scan.php?page=article=2bsd-7linux-bench=4
> > 
> > I wonder about the results for FreeBSD. As I do not have 11 on my 
> > machines, a stupid question. Are there still some debugging aids 
> > enabled in 11?   
> > They're off in those versions, but did note compiler (and compiler
> > args) differences between within most tests (See attachments) as you
> > mentioned.
>  the benchmark then compares the off-the-shelve distributions. 
> 
> Excuse me, as a casual reader of the list, I don't get this "critique". 
> 
> I never recompile my installations, I just use them from the
> installation CD (as probably 90% of the rest of the world), so I don't
> get what is wrong with the approach of comparing an out-of-the-box
> FreeBSD 11 with an out-of-the-box Ubuntu whatever. 
> 
> If FreeBSD 11 "out-the-box" performs slow because the standard compilers
> options aren't good it's not a problem with the benchmarking platform
> but with the default CD compiling options. 
> 
> Am I getting it wrong? 

The problem here is that Phoronix took a Beta version of FreeBSD 11.
Beta versions have a lot of debugging (malloc, invariants, witness)
options enabled which make it significantly slower than release
versions. This is even obviously when you run a Beta as a desktop. It
just feels much slower.


pgpOsKyxPVeSN.pgp
Description: PGP signature

Re: Benchmarks results for FreeBSD 11

[Andrea Brancatelli]

Il 2016-08-21 08:45 Erich Dollansky ha scritto:

> I am sure that some know of this site:
> 
> http://www.phoronix.com/scan.php?page=article=2bsd-7linux-bench=4
> 
> I wonder about the results for FreeBSD. As I do not have 11 on my 
> machines, a stupid question. Are there still some debugging aids 
> enabled in 11?   
> They're off in those versions, but did note compiler (and compiler
> args) differences between within most tests (See attachments) as you
> mentioned.
 the benchmark then compares the off-the-shelve distributions. 

Excuse me, as a casual reader of the list, I don't get this "critique". 

I never recompile my installations, I just use them from the
installation CD (as probably 90% of the rest of the world), so I don't
get what is wrong with the approach of comparing an out-of-the-box
FreeBSD 11 with an out-of-the-box Ubuntu whatever. 

If FreeBSD 11 "out-the-box" performs slow because the standard compilers
options aren't good it's not a problem with the benchmarking platform
but with the default CD compiling options. 

Am I getting it wrong? 

Thanks :) 

---

Andrea Brancatelli
Schema31 S.p.a.
Responsabile IT

ROMA - BO - FI - PA 
ITALY
Tel: +39.06.98.358.472
Cell: +39.331.2488468
Fax: +39.055.71.880.466
Società del Gruppo SC31 ITALIA
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: Benchmarks results for FreeBSD 11

[Slawa Olhovchenkov]

On Mon, Aug 22, 2016 at 03:22:35AM +, Kubilay Kocak wrote:

> On Mon, 22 Aug 2016, 11:31 AM Mark Linimon  wrote:
> 
> > On Mon, Aug 22, 2016 at 09:57:24AM +1000, Dewayne Geraghty wrote:
> > > unless knowledgable people respond publicly and/or in the phoronix
> > > forums [...] this interpretation of reality will be fixed in decision-
> > > makers' minds and consequently the uptake (and support) of FreeBSD.
> >
> > IIRC this has been done before and hasn't really been productive.  OTOH
> > I don't recall the details.
> >
> > FreeBSD hasn't had a benchmarking guru since Kris Kennaway retired from
> > working on FreeBSD.
> 
> 
> Michael has reached out off-list (thanks!) If anyone else is interested,
> I'd be happy to create a dedicated IRC channel on freenode to widen and
> focus the freebsd performance discussion net and doc any outcomes/notes in
> the FreeBSD Wiki.

What about support by hardware?
For good benchmarking report need:

1. Hardware (with KVM access)
2. People(s) do benchmark
3. Advocacy
4. Time for this
5. Reputation (only time create this).

I am have some expirense with performance advocacing and improvement,
but don't have hardware for test and advocacy skils.
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: Benchmarks results for FreeBSD 11

[Kubilay Kocak]

On Mon, 22 Aug 2016, 11:31 AM Mark Linimon  wrote:

> On Mon, Aug 22, 2016 at 09:57:24AM +1000, Dewayne Geraghty wrote:
> > unless knowledgable people respond publicly and/or in the phoronix
> > forums [...] this interpretation of reality will be fixed in decision-
> > makers' minds and consequently the uptake (and support) of FreeBSD.
>
> IIRC this has been done before and hasn't really been productive.  OTOH
> I don't recall the details.
>
> FreeBSD hasn't had a benchmarking guru since Kris Kennaway retired from
> working on FreeBSD.


Michael has reached out off-list (thanks!) If anyone else is interested,
I'd be happy to create a dedicated IRC channel on freenode to widen and
focus the freebsd performance discussion net and doc any outcomes/notes in
the FreeBSD Wiki.

./koobs
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: Benchmarks results for FreeBSD 11

[Mark Linimon]

On Mon, Aug 22, 2016 at 09:57:24AM +1000, Dewayne Geraghty wrote:
> unless knowledgable people respond publicly and/or in the phoronix
> forums [...] this interpretation of reality will be fixed in decision-
> makers' minds and consequently the uptake (and support) of FreeBSD.

IIRC this has been done before and hasn't really been productive.  OTOH
I don't recall the details.

FreeBSD hasn't had a benchmarking guru since Kris Kennaway retired from
working on FreeBSD.  It would be really nice if Someone(TM) took up the
position.

I would have to educate myself on the whole topic, so I'm not a likely
nominee.

mcl
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: Benchmarks results for FreeBSD 11

[Dewayne Geraghty]

Unfortunately people (customers, developers, hardward vendors) make
decisions on the basis of bang-for-buck.  FreeBSD is consistently
underperforming on benchmarks.  And regardless of real-world similarity,
the contrived benchmarks are the best that is used.

If clang (v3.4.1 on 10.3 Stable)  really is the cause, then there should be
an option to use "whatever" will let the source perform at its best and
reflect the stability that we have with FreeBSD.

We can criticise the test results or the test regime, but unless
knowledgable people respond publicly and/or in the phoronix forums, such as
https://www.phoronix.com/forums/forum/software/distributions/891660-benchmarks-2-bsds-vs-7-linux-distributions
Then this interpretation of reality will be fixed in decision-makers' minds
and consequently the uptake (and support) of FreeBSD.

As I recall Michael's testing regime has been consistently criticised (by
FreeBSD folks) since he started his website, unfortunately when someone is
making public statements and comparisons we really need to have a
relationship with them to ensure that FreeBSD's best position is put
forward.
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: Benchmarks results for FreeBSD 11

[Erich Dollansky]

Hi,

On Sun, 21 Aug 2016 15:21:01 +1000
Kubilay Kocak  wrote:

> On 19/08/2016 9:34 AM, Erich Dollansky wrote:
> > 
> > I am sure that some know of this site:
> > 
> > http://www.phoronix.com/scan.php?page=article=2bsd-7linux-bench=4
> >
> >  I wonder about the results for FreeBSD. As I do not have 11 on my 
> > machines, a stupid question. Are there still some debugging aids 
> > enabled in 11?  
> 
> They're off in those versions, but did note compiler (and compiler
> args) differences between within most tests (See attachments) as you
> mentioned.
> 
the benchmark then compares the off-the-shelve distributions.
> 
> > I know that some of the results are caused by the use of CLang and
> > some of the results test applications/compilers and not operating
> > systems.  
> 
> gcc/clang tests and defaults in upstream build systems are almost
> certainly contributors. At a minimum it would be nice to see an
> attempt to standardise (force) compiler args across all OS runs for
> the same test, even if this doesn't prove to be perfect.

Yes, without, he compares compilers more than operating systems.
> 
> Separating or adding tests for the same tests using non-default
> compilers (in particular latest GCC versions from ports) so they match
> across OS's would also be valuable.
> 
> At a minimum it would be worth Michael highlighting the differences,
> and ideally removing these variables from the tests, even if they
> aren't default configurations. Though a test of out of the box
> configurations is still valuable, it can serve to muddy the
> underlying differences and make them tougher to isolate.

He never does this. I have no idea what he wants to achieve with his
benchmarking all the while as the results are very difficult to compare
with real life workloads.

Erich
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: Benchmarks results for FreeBSD 11

[Kubilay Kocak]

On 19/08/2016 9:34 AM, Erich Dollansky wrote:
> Hi,
> 
> I am sure that some know of this site:
> 
> http://www.phoronix.com/scan.php?page=article=2bsd-7linux-bench=4
>
>  I wonder about the results for FreeBSD. As I do not have 11 on my 
> machines, a stupid question. Are there still some debugging aids 
> enabled in 11?

They're off in those versions, but did note compiler (and compiler args)
differences between within most tests (See attachments) as you mentioned.


> I know that some of the results are caused by the use of CLang and
> some of the results test applications/compilers and not operating
> systems.

gcc/clang tests and defaults in upstream build systems are almost
certainly contributors. At a minimum it would be nice to see an attempt
to standardise (force) compiler args across all OS runs for the same
test, even if this doesn't prove to be perfect.

Separating or adding tests for the same tests using non-default
compilers (in particular latest GCC versions from ports) so they match
across OS's would also be valuable.

At a minimum it would be worth Michael highlighting the differences, and
ideally removing these variables from the tests, even if they aren't
default configurations. Though a test of out of the box configurations
is still valuable, it can serve to muddy the underlying differences and
make them tougher to isolate.
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: Benchmarks results for FreeBSD 11

[Slawa Olhovchenkov]

On Fri, Aug 19, 2016 at 07:34:22AM +0800, Erich Dollansky wrote:

> Hi,
> 
> I am sure that some know of this site:
> 
> http://www.phoronix.com/scan.php?page=article=2bsd-7linux-bench=4
> 
> I wonder about the results for FreeBSD. As I do not have 11 on my
> machines, a stupid question. Are there still some debugging aids
> enabled in 11?
> 
> I know that some of the results are caused by the use of CLang and some
> of the results test applications/compilers and not operating systems.

Most debugging aids off at time of branching stable/11, remain (mostly
for non-amd64 architectures switch off too later).

Some bench result very stranges
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: Benchmarks results for FreeBSD 11

[Erich Dollansky]

Hi,

On Fri, 19 Aug 2016 09:50:16 +0300
Sami Halabi  wrote:

> At the bottom of the article there is link to the source benchmark
> site. From a quick glance the benchmarks made not with the same
> hardware for all. Phoronix is sort of anti-bsd site.. i won't count
> on the truth of the data they present.

leaving this aside, I know while I used what is now 11, there were
several hand breaks on debugging just in case off. I do not know when
these hand breaks are taken off and I also do not know if they still
have been in the binaries used.

Forget about the rest. This site is something the Sun of the IT world.

Erich
> 
> Sami
> 
> בתאריך 19 באוג׳ 2016 02:34 AM,‏ "Erich Dollansky" <
> erichsfreebsdl...@alogt.com> כתב:  
> 
> > Hi,
> >
> > I am sure that some know of this site:
> >
> > http://www.phoronix.com/scan.php?page=article=2bsd-7linux-bench=4
> >
> > I wonder about the results for FreeBSD. As I do not have 11 on my
> > machines, a stupid question. Are there still some debugging aids
> > enabled in 11?
> >
> > I know that some of the results are caused by the use of CLang and
> > some of the results test applications/compilers and not operating
> > systems.
> >
> > Erich
> > ___
> > freebsd-stable@freebsd.org mailing list
> > https://lists.freebsd.org/mailman/listinfo/freebsd-stable
> > To unsubscribe, send any mail to
> > "freebsd-stable-unsubscr...@freebsd.org" 
> ___
> freebsd-stable@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-stable
> To unsubscribe, send any mail to
> "freebsd-stable-unsubscr...@freebsd.org"
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: Benchmarks results for FreeBSD 11

[Sami Halabi]

Hi,
At the bottom of the article there is link to the source benchmark site.
From a quick glance the benchmarks made not with the same hardware for all.
Phoronix is sort of anti-bsd site.. i won't count on the truth of the data
they present.

Sami

בתאריך 19 באוג׳ 2016 02:34 AM,‏ "Erich Dollansky" <
erichsfreebsdl...@alogt.com> כתב:

> Hi,
>
> I am sure that some know of this site:
>
> http://www.phoronix.com/scan.php?page=article=2bsd-7linux-bench=4
>
> I wonder about the results for FreeBSD. As I do not have 11 on my
> machines, a stupid question. Are there still some debugging aids
> enabled in 11?
>
> I know that some of the results are caused by the use of CLang and some
> of the results test applications/compilers and not operating systems.
>
> Erich
> ___
> freebsd-stable@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-stable
> To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
>
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Benchmarks results for FreeBSD 11

[Erich Dollansky]

Hi,

I am sure that some know of this site:

http://www.phoronix.com/scan.php?page=article=2bsd-7linux-bench=4

I wonder about the results for FreeBSD. As I do not have 11 on my
machines, a stupid question. Are there still some debugging aids
enabled in 11?

I know that some of the results are caused by the use of CLang and some
of the results test applications/compilers and not operating systems.

Erich
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"