Re: [gentoo-dev] ICC Profile

2008-07-19 Thread Michael Hammer
Cold you please fix your reply?

-- 

Michael Hammer|[EMAIL PROTECTED] | Graz, AT
Gentoo Developer (Kerberos)  |  http://www.michael-hammer.at

pgpDY8QTPhvzj.pgp
Description: PGP signature


Re: [gentoo-dev] ICC Profile

2008-07-19 Thread Adam Stylinski
Well, it looks like we're not alone on this project.  This email was just sent 
to me:


Adam,
I managed to track down someone who can probably help you with your ICC work, 
if you're nice to him. :)
See the forwarded message:

Have your friend contact bill dot hilliard at intel dot com.  He's on the ICC 
team and can either answer any questions or works next to someone who can.

Scott
-- 
gentoo-dev@lists.gentoo.org mailing list



Re: [gentoo-dev] ICC Profile

2008-07-18 Thread Adam Stylinski
I'm not suggesting that it be sold. Gentoo is
non-profit anyway, the livecd could be available for
download only. The binaries don't have to be
licensed if you're not selling them, however the
compiler does. This is where the non-commercial free
license comes in (with a fetch restriction requiring
the user to register for it). This is completely
free and I don't see it much more of a pain than it
is to go sign up for the IBM developers work to
extract their PPC version of java.
-- 
gentoo-dev@lists.gentoo.org mailing list



Re: [gentoo-dev] ICC Profile

2008-07-18 Thread Doug Goldstein

Robert Bridge wrote:

On Fri, 18 Jul 2008 11:34:11 +0900
Luca Barbato [EMAIL PROTECTED] wrote:

  

Adam Stylinski wrote:


The intel C Compiler (icc)
  

icc, xlc, llvm, sunstudio could be interesting fields of discovery.

Which are the pitfalls of using icc?

lu




If I recall correctly, needing some packages compiled with ICC flags
set one way, and needing others compiled with the same flag set the
oter way.

Can it compile a kernel yet?

Rob.
  
No. It doesn't implement all of GCCs extensions and all GNUisms that the 
kernel relies on.

--
gentoo-dev@lists.gentoo.org mailing list



Re: [gentoo-dev] ICC Profile

2008-07-18 Thread Sébastien Fabbro
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Apologies if you received this mail already, I'm having problems with my
smtp.

On Thursday July 17 Adam Stylinski wrote:

 The intel C Compiler (icc) has an ebuild for gentoo and the wiki has
 a script to integrate it with portage.  This script works will in
 terms of building binaries, however when mixed with gcc environments
 there are massive linking issues.  I propose that an ICC profile is
 made which contains specific versions and default flags for people
 who want to build a mixed icc-gcc environment.  ICC is much faster
 than GCC and although not free, offers a free non-commercial
 license.  I would be very interested in this project and more than
 willing to help to the best of my abilities.  I've already been
 trying to maintain a mixed environment with some luck, while there
 have been a lot of problems using dynamically linked libraries (ld
 from intel and ld from gcc don't always get along), my system is
 substatially faster.  The kernel obviously will still be built under
 gcc as well as bash (unless intel helps submit patches to make the
 code work with their compiler).  There are many tools icc ! has to
 offer for vectorization.  If these were streamlined into Gentoo with
 a fetch restriction for ICC, a bootsrapping boot disk could be made
 and result in a very fast distribution. 


An icc profile would be welcome. I've been the maintainer of icc (and
other Intel tools) for the last year or so more by default than
real interest. I would welcome any input from the Gentoo community.
Re-adding slots and an icc profile was on my mind, but never found the
time to invest in it, and got at the tail of my priority list. So don't
hesitate to contact me (email, irc, bugs) and others.

There was some attempts a few years ago for rolling up a full Gentoo
with icc, but it hit several problems if I recall. Now both icc and gcc
have improved since then.

Also, if you haven't already, check also some of the old bugs [1,2] we
have, and a recurring one [3].

I would like to recall one important issue with the Intel license
concerning the free for non-commercial use [4]. It means
you can't use it for free if you're paid to use it. Yes,
beer is not free for academic scientists too.

[1] http://bugs.gentoo.org/show_bug.cgi?id=26757
[2] http://bugs.gentoo.org/show_bug.cgi?id=53808
[3] http://bugs.gentoo.org/show_bug.cgi?id=201596
[4] http://www.intel.com/cd/software/products/asmo-na/eng/219692.htm#0

- - -- 
Sébastien
-BEGIN PGP SIGNATURE-
Version: GnuPG v2.0.9 (GNU/Linux)

iEYEARECAAYFAkiApboACgkQ1ycZbhPLE2BMHgCggfzfS4SakVyw42r+JnnxYNpL
E9gAoJvRrocinIDlInF6kbeSGF2kvX9t
=M4XK
-END PGP SIGNATURE-


Re: [gentoo-dev] ICC Profile

2008-07-18 Thread Sébastien Fabbro
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1


On Thursday July 17, Adam Stylinski wrote:

 Pro's:
 
 1.) Bloody fast machine code.  Intel obfuscates their architecture
 but they give back to the community as much as possible to make their
 hardware marketable toward the open source sysadmin, developer, etc
 etc.  Their drivers are open and they develop for the kernel
 constantly.  This cooperation leads me to believe that they would
 assist a team of developers in making 100% icc compatible code.  

Gentoo is not supported from Intel, and they had not plans doing so. As
of October 2007, I asked their Software channel whether Gentoo
users have similar support as RedHat or SUSE users and the answer was:

No, we have no current plans to support Gentoo. Also, Gentoo is NOT a
derivative of a Linux we do support.  My understanding is that it is
independently derived from Kernel.org.  Thus it is less likely to work
than a distro which is a derivative of a supported distribution.

Meanwhile, Debian/Ubuntu got support, so things might change if
Gentoo re-becomes/remains popular. Any Intel dev reading this list,
please contact us.

And as Luca mentioned, having sunstudio, xlc (is this one free?) or llvm
would not make Intel a privileged case for Gentoo.


 2.) Bloody fast compilation time.  In my experience the compiler
 works much faster even with heavy optimization.  

I don't experience this that much, but I really don't use it much
either. Would be nice to have benchmarks here.

 
 4.) will project gentoo toward the power user more, helps the gentoo
 image, and overall will make linux a more professional operating
 system (and a quite competitive alternative to something like a
 SPARC+Solaris configuration).  This would also make cluster farms and
 science application more respectful toward the gentoo community.  The
 academic and research world already uses ICC to compile their apps
 for the sake of speed.  The interprocedural optimizations for both
 the fortran and c/c++ compilers make it a must.  

I would be careful about this, and this needs benchmarks, especially
with gcc  4.3. By default icc flags are fairly agressive. For example,
for many scientific applications, you don't want a simple -O2 where you
loose floating point precision. Add -mp or -mp1 to your icc flags, add
some decent gcc flags, and improvement over gcc is much smaller.
 
 5.) It's free, albeit a commercial product.  As gentoo is entirely
 non-profit, there is no restriction when it comes to licensing.  The
 binaries won't be sold for the intel-compiled livecd, and the
 compiler itself with a fetch restriction allows the user to legally
 register for their free non-commercial license.

Again, as long as you're not being compensated for doing it (for Gentoo
I'm not).

In summary, I'm completely in favor of trying projects like this, but
first, this needs a few benchmarks before going further.


- -- 
Sébastien
-BEGIN PGP SIGNATURE-
Version: GnuPG v2.0.9 (GNU/Linux)

iEYEARECAAYFAkiApcoACgkQ1ycZbhPLE2C0TQCfYLD2+mazHKjosK7wiHFU6FGb
d3EAnR1FWZ92O2B9xBJerCj4dj2GRUZ4
=YhT2
-END PGP SIGNATURE-
���^�X�����(��j)b�b�

Re: [gentoo-dev] ICC Profile

2008-07-18 Thread Robert Bridge
On Fri, 18 Jul 2008 15:16:18 +0100
Sébastien Fabbro [EMAIL PROTECTED] wrote:

 There was some attempts a few years ago for rolling up a full Gentoo
 with icc, but it hit several problems if I recall. Now both icc and
 gcc have improved since then.

Including needing package specific CFLAGS because some packages in
@system needed mutually exclusive flag settings. 

I'll see if I can dig up the link for an old blog on the topic later.
--
gentoo-dev@lists.gentoo.org mailing list



Re: [gentoo-dev] ICC Profile

2008-07-18 Thread Adam Stylinski
Also, in the academic environment the grad student/university can pay for the 
license that the student slipstreams into their gentoo installation, making it 
100% legal depending on how many seats he or she buys.
-- 
gentoo-dev@lists.gentoo.org mailing list



Re: [gentoo-dev] ICC Profile

2008-07-17 Thread Luca Barbato

Adam Stylinski wrote:

The intel C Compiler (icc)


icc, xlc, llvm, sunstudio could be interesting fields of discovery.

Which are the pitfalls of using icc?

lu

--

Luca Barbato
Gentoo Council Member
Gentoo/linux Gentoo/PPC
http://dev.gentoo.org/~lu_zero

--
gentoo-dev@lists.gentoo.org mailing list



Re: [gentoo-dev] ICC Profile

2008-07-17 Thread Robert Bridge
On Fri, 18 Jul 2008 11:34:11 +0900
Luca Barbato [EMAIL PROTECTED] wrote:

 Adam Stylinski wrote:
  The intel C Compiler (icc)
 
 icc, xlc, llvm, sunstudio could be interesting fields of discovery.
 
 Which are the pitfalls of using icc?
 
 lu
 

If I recall correctly, needing some packages compiled with ICC flags
set one way, and needing others compiled with the same flag set the
oter way.

Can it compile a kernel yet?

Rob.
-- 
gentoo-dev@lists.gentoo.org mailing list



Re: [gentoo-dev] ICC Profile

2008-07-17 Thread Adam Stylinski
There are very few pitfalls, none of which I see as real killers.  These 
include:

1.) Closed source compiler: Yes this stands against what we believe, and yes by 
closing their source they're protecting the trade secrets of their 
architecture.  It also could be more difficult to debug, although that's highly 
unlikely, they have the idb (intel debugger) which works very much like gdb.

2.) Linking issues: So far it's pretty versatile, but it doesn't always 
cooperate with gcc compiled apps.  It may be a good strategy to make the 
troublesome apps which won't compile with ICC compile with ICC.

Pro's:

1.) Bloody fast machine code.  Intel obfuscates their architecture but they 
give back to the community as much as possible to make their hardware 
marketable toward the open source sysadmin, developer, etc etc.  Their drivers 
are open and they develop for the kernel constantly.  This cooperation leads me 
to believe that they would assist a team of developers in making 100% icc 
compatible code.

2.) Bloody fast compilation time.  In my experience the compiler works much 
faster even with heavy optimization.

3.) Takes full advantage of SSE enabled hardware.  SIMD instructions are quite 
useful, code is extremely vectorized.

4.) will project gentoo toward the power user more, helps the gentoo image, and 
overall will make linux a more professional operating system (and a quite 
competitive alternative to something like a SPARC+Solaris configuration).  This 
would also make cluster farms and science application more respectful toward 
the gentoo community.  The academic and research world already uses ICC to 
compile their apps for the sake of speed.  The interprocedural optimizations 
for both the fortran and c/c++ compilers make it a must.

5.) It's free, albeit a commercial product.  As gentoo is entirely non-profit, 
there is no restriction when it comes to licensing.  The binaries won't be sold 
for the intel-compiled livecd, and the compiler itself with a fetch restriction 
allows the user to legally register for their free non-commercial license.  
-- 
gentoo-dev@lists.gentoo.org mailing list



Re: [gentoo-dev] ICC Profile

2008-07-17 Thread Adam Stylinski
There is some record of a version of the kernel being compiled with some 
patches involved.  It's probably possible, I'd imagine.  Though, this is not 
necessary for the first builds.
-- 
gentoo-dev@lists.gentoo.org mailing list



Re: [gentoo-dev] ICC Profile

2008-07-17 Thread Adam Stylinski
Oh yes, and we can also take advantage of the free support that intel offers 
for all users:

http://www.intel.com/support/performancetools/sb/CS-017156.htm

Not to mention they have forums filled with intel developers/experts which we 
can get involved.  I'm sure intel would benefit from this and be more than 
excited to collaborate:
http://softwarecommunity.intel.com/isn/Community/en-US/forums/default.aspx
 
-- 
gentoo-dev@lists.gentoo.org mailing list



Re: [gentoo-dev] ICC Profile

2008-07-17 Thread Donnie Berkholz
On 14:23 Thu 17 Jul , Adam Stylinski wrote:
 The intel C Compiler (icc) has an ebuild for gentoo and the wiki has a 
 script to integrate it with portage.  This script works will in terms 
 of building binaries, however when mixed with gcc environments there 
 are massive linking issues.  I propose that an ICC profile is made 
 which contains specific versions and default flags for people who want 
 to build a mixed icc-gcc environment.  ICC is much faster than GCC and 
 although not free, offers a free non-commercial license.  I would be 
 very interested in this project and more than willing to help to the 
 best of my abilities.  I've already been trying to maintain a mixed 
 environment with some luck, while there have been a lot of problems 
 using dynamically linked libraries (ld from intel and ld from gcc 
 don't always get along), my system is substatially faster.  The kernel 
 obviously will still be built under gcc as well as bash (unless intel 
 helps submit patches to make the code work with their compiler).  
 There are many tools icc has to offer for vectorization.  If these 
 were streamlined into Gentoo with a fetch restriction for ICC, a 
 bootsrapping boot disk could be made and result in a very fast 
 distribution. -- gentoo-dev@lists.gentoo.org mailing list

Adam,

I've already got a mixed system that works reasonably well. I use 
/etc/portage/env/ to control which packages build with icc and 
icc-specific CFLAGS by having /etc/portage/env/$CATEGORY/$PN be symlinks 
to /etc/portage/env/env.icc with these contents:

  echo  * Exporting: Intel compilers in $EBUILD_PHASE
  CC=icc
  CXX=icc
  FC=ifort
  F77=ifort
  FFLAGS=-openmp -parallel -ipo -xO -O2 -no-prec-div
  FCFLAGS=${FFLAGS}
  
  export CC CXX FC F77 FFLAGS FCFLAGS

Is what you're proposing here an improvement over something like the 
above setup? If so, could you explain how?

-- 
Thanks,
Donnie

Donnie Berkholz
Developer, Gentoo Linux
Blog: http://dberkholz.wordpress.com


pgpRKtYUHvEJk.pgp
Description: PGP signature


Re: [gentoo-dev] ICC Profile

2008-07-17 Thread Donnie Berkholz
On 23:24 Thu 17 Jul , Adam Stylinski wrote:
 There are very few pitfalls, none of which I see as real killers.  
 These include:
 
 1.) Closed source compiler: Yes this stands against what we believe, 
 and yes by closing their source they're protecting the trade secrets 
 of their architecture.  It also could be more difficult to debug, 
 although that's highly unlikely, they have the idb (intel debugger) 
 which works very much like gdb.

Doesn't really matter, although we could never move to it as the default 
without violating our social contract.

 2.) Linking issues: So far it's pretty versatile, but it doesn't 
 always cooperate with gcc compiled apps.  It may be a good strategy to 
 make the troublesome apps which won't compile with ICC compile with 
 ICC.

Pretty sure you didn't mean ICC twice here, but sure.

 Pro's:
 
 1.) Bloody fast machine code.  Intel obfuscates their architecture but 
 they give back to the community as much as possible to make their 
 hardware marketable toward the open source sysadmin, developer, etc 
 etc.  Their drivers are open and they develop for the kernel 
 constantly.  This cooperation leads me to believe that they would 
 assist a team of developers in making 100% icc compatible code.

OK. This involves upstream projects more than us, though.

 2.) Bloody fast compilation time.  In my experience the compiler works 
 much faster even with heavy optimization.
 
 3.) Takes full advantage of SSE enabled hardware.  SIMD instructions 
 are quite useful, code is extremely vectorized.

Sure, sure, speed is nice.

 4.) will project gentoo toward the power user more, helps the gentoo 
 image, and overall will make linux a more professional operating 
 system (and a quite competitive alternative to something like a 
 SPARC+Solaris configuration).

I don't buy any parts of this argument, although the rest of your email 
is pretty good.

 This would also make cluster farms and science application more 
 respectful toward the gentoo community.  The academic and research 
 world already uses ICC to compile their apps for the sake of speed.  
 The interprocedural optimizations for both the fortran and c/c++ 
 compilers make it a must.

Yes, ICC is nice. And people can already use it easily within Gentoo for 
performance-critical apps.

 5.) It's free, albeit a commercial product.  As gentoo is entirely 
 non-profit, there is no restriction when it comes to licensing.  The 
 binaries won't be sold for the intel-compiled livecd, and the compiler 
 itself with a fetch restriction allows the user to legally register 
 for their free non-commercial license.

OK, so we can't sell icc-compiled software in our Gentoo store, so the 
releases would always remain built with gcc.

-- 
Thanks,
Donnie

Donnie Berkholz
Developer, Gentoo Linux
Blog: http://dberkholz.wordpress.com


pgpyuC3bLpdh5.pgp
Description: PGP signature