Re: [gentoo-dev] ICC Profile
Cold you please fix your reply? -- Michael Hammer|[EMAIL PROTECTED] | Graz, AT Gentoo Developer (Kerberos) | http://www.michael-hammer.at pgpDY8QTPhvzj.pgp Description: PGP signature
Re: [gentoo-dev] ICC Profile
Well, it looks like we're not alone on this project. This email was just sent to me: Adam, I managed to track down someone who can probably help you with your ICC work, if you're nice to him. :) See the forwarded message: Have your friend contact bill dot hilliard at intel dot com. He's on the ICC team and can either answer any questions or works next to someone who can. Scott -- gentoo-dev@lists.gentoo.org mailing list
Re: [gentoo-dev] ICC Profile
I'm not suggesting that it be sold. Gentoo is non-profit anyway, the livecd could be available for download only. The binaries don't have to be licensed if you're not selling them, however the compiler does. This is where the non-commercial free license comes in (with a fetch restriction requiring the user to register for it). This is completely free and I don't see it much more of a pain than it is to go sign up for the IBM developers work to extract their PPC version of java. -- gentoo-dev@lists.gentoo.org mailing list
Re: [gentoo-dev] ICC Profile
Robert Bridge wrote: On Fri, 18 Jul 2008 11:34:11 +0900 Luca Barbato [EMAIL PROTECTED] wrote: Adam Stylinski wrote: The intel C Compiler (icc) icc, xlc, llvm, sunstudio could be interesting fields of discovery. Which are the pitfalls of using icc? lu If I recall correctly, needing some packages compiled with ICC flags set one way, and needing others compiled with the same flag set the oter way. Can it compile a kernel yet? Rob. No. It doesn't implement all of GCCs extensions and all GNUisms that the kernel relies on. -- gentoo-dev@lists.gentoo.org mailing list
Re: [gentoo-dev] ICC Profile
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Apologies if you received this mail already, I'm having problems with my smtp. On Thursday July 17 Adam Stylinski wrote: The intel C Compiler (icc) has an ebuild for gentoo and the wiki has a script to integrate it with portage. This script works will in terms of building binaries, however when mixed with gcc environments there are massive linking issues. I propose that an ICC profile is made which contains specific versions and default flags for people who want to build a mixed icc-gcc environment. ICC is much faster than GCC and although not free, offers a free non-commercial license. I would be very interested in this project and more than willing to help to the best of my abilities. I've already been trying to maintain a mixed environment with some luck, while there have been a lot of problems using dynamically linked libraries (ld from intel and ld from gcc don't always get along), my system is substatially faster. The kernel obviously will still be built under gcc as well as bash (unless intel helps submit patches to make the code work with their compiler). There are many tools icc ! has to offer for vectorization. If these were streamlined into Gentoo with a fetch restriction for ICC, a bootsrapping boot disk could be made and result in a very fast distribution. An icc profile would be welcome. I've been the maintainer of icc (and other Intel tools) for the last year or so more by default than real interest. I would welcome any input from the Gentoo community. Re-adding slots and an icc profile was on my mind, but never found the time to invest in it, and got at the tail of my priority list. So don't hesitate to contact me (email, irc, bugs) and others. There was some attempts a few years ago for rolling up a full Gentoo with icc, but it hit several problems if I recall. Now both icc and gcc have improved since then. Also, if you haven't already, check also some of the old bugs [1,2] we have, and a recurring one [3]. I would like to recall one important issue with the Intel license concerning the free for non-commercial use [4]. It means you can't use it for free if you're paid to use it. Yes, beer is not free for academic scientists too. [1] http://bugs.gentoo.org/show_bug.cgi?id=26757 [2] http://bugs.gentoo.org/show_bug.cgi?id=53808 [3] http://bugs.gentoo.org/show_bug.cgi?id=201596 [4] http://www.intel.com/cd/software/products/asmo-na/eng/219692.htm#0 - - -- Sébastien -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.9 (GNU/Linux) iEYEARECAAYFAkiApboACgkQ1ycZbhPLE2BMHgCggfzfS4SakVyw42r+JnnxYNpL E9gAoJvRrocinIDlInF6kbeSGF2kvX9t =M4XK -END PGP SIGNATURE-
Re: [gentoo-dev] ICC Profile
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Thursday July 17, Adam Stylinski wrote: Pro's: 1.) Bloody fast machine code. Intel obfuscates their architecture but they give back to the community as much as possible to make their hardware marketable toward the open source sysadmin, developer, etc etc. Their drivers are open and they develop for the kernel constantly. This cooperation leads me to believe that they would assist a team of developers in making 100% icc compatible code. Gentoo is not supported from Intel, and they had not plans doing so. As of October 2007, I asked their Software channel whether Gentoo users have similar support as RedHat or SUSE users and the answer was: No, we have no current plans to support Gentoo. Also, Gentoo is NOT a derivative of a Linux we do support. My understanding is that it is independently derived from Kernel.org. Thus it is less likely to work than a distro which is a derivative of a supported distribution. Meanwhile, Debian/Ubuntu got support, so things might change if Gentoo re-becomes/remains popular. Any Intel dev reading this list, please contact us. And as Luca mentioned, having sunstudio, xlc (is this one free?) or llvm would not make Intel a privileged case for Gentoo. 2.) Bloody fast compilation time. In my experience the compiler works much faster even with heavy optimization. I don't experience this that much, but I really don't use it much either. Would be nice to have benchmarks here. 4.) will project gentoo toward the power user more, helps the gentoo image, and overall will make linux a more professional operating system (and a quite competitive alternative to something like a SPARC+Solaris configuration). This would also make cluster farms and science application more respectful toward the gentoo community. The academic and research world already uses ICC to compile their apps for the sake of speed. The interprocedural optimizations for both the fortran and c/c++ compilers make it a must. I would be careful about this, and this needs benchmarks, especially with gcc 4.3. By default icc flags are fairly agressive. For example, for many scientific applications, you don't want a simple -O2 where you loose floating point precision. Add -mp or -mp1 to your icc flags, add some decent gcc flags, and improvement over gcc is much smaller. 5.) It's free, albeit a commercial product. As gentoo is entirely non-profit, there is no restriction when it comes to licensing. The binaries won't be sold for the intel-compiled livecd, and the compiler itself with a fetch restriction allows the user to legally register for their free non-commercial license. Again, as long as you're not being compensated for doing it (for Gentoo I'm not). In summary, I'm completely in favor of trying projects like this, but first, this needs a few benchmarks before going further. - -- Sébastien -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.9 (GNU/Linux) iEYEARECAAYFAkiApcoACgkQ1ycZbhPLE2C0TQCfYLD2+mazHKjosK7wiHFU6FGb d3EAnR1FWZ92O2B9xBJerCj4dj2GRUZ4 =YhT2 -END PGP SIGNATURE- ���^�X��� ��(��j)b�b�
Re: [gentoo-dev] ICC Profile
On Fri, 18 Jul 2008 15:16:18 +0100 Sébastien Fabbro [EMAIL PROTECTED] wrote: There was some attempts a few years ago for rolling up a full Gentoo with icc, but it hit several problems if I recall. Now both icc and gcc have improved since then. Including needing package specific CFLAGS because some packages in @system needed mutually exclusive flag settings. I'll see if I can dig up the link for an old blog on the topic later. -- gentoo-dev@lists.gentoo.org mailing list
Re: [gentoo-dev] ICC Profile
Also, in the academic environment the grad student/university can pay for the license that the student slipstreams into their gentoo installation, making it 100% legal depending on how many seats he or she buys. -- gentoo-dev@lists.gentoo.org mailing list
[gentoo-dev] ICC Profile
The intel C Compiler (icc) has an ebuild for gentoo and the wiki has a script to integrate it with portage. This script works will in terms of building binaries, however when mixed with gcc environments there are massive linking issues. I propose that an ICC profile is made which contains specific versions and default flags for people who want to build a mixed icc-gcc environment. ICC is much faster than GCC and although not free, offers a free non-commercial license. I would be very interested in this project and more than willing to help to the best of my abilities. I've already been trying to maintain a mixed environment with some luck, while there have been a lot of problems using dynamically linked libraries (ld from intel and ld from gcc don't always get along), my system is substatially faster. The kernel obviously will still be built under gcc as well as bash (unless intel helps submit patches to make the code work with their compiler). There are many tools icc ! has to offer for vectorization. If these were streamlined into Gentoo with a fetch restriction for ICC, a bootsrapping boot disk could be made and result in a very fast distribution. -- gentoo-dev@lists.gentoo.org mailing list
Re: [gentoo-dev] ICC Profile
Adam Stylinski wrote: The intel C Compiler (icc) icc, xlc, llvm, sunstudio could be interesting fields of discovery. Which are the pitfalls of using icc? lu -- Luca Barbato Gentoo Council Member Gentoo/linux Gentoo/PPC http://dev.gentoo.org/~lu_zero -- gentoo-dev@lists.gentoo.org mailing list
Re: [gentoo-dev] ICC Profile
On Fri, 18 Jul 2008 11:34:11 +0900 Luca Barbato [EMAIL PROTECTED] wrote: Adam Stylinski wrote: The intel C Compiler (icc) icc, xlc, llvm, sunstudio could be interesting fields of discovery. Which are the pitfalls of using icc? lu If I recall correctly, needing some packages compiled with ICC flags set one way, and needing others compiled with the same flag set the oter way. Can it compile a kernel yet? Rob. -- gentoo-dev@lists.gentoo.org mailing list
Re: [gentoo-dev] ICC Profile
There are very few pitfalls, none of which I see as real killers. These include: 1.) Closed source compiler: Yes this stands against what we believe, and yes by closing their source they're protecting the trade secrets of their architecture. It also could be more difficult to debug, although that's highly unlikely, they have the idb (intel debugger) which works very much like gdb. 2.) Linking issues: So far it's pretty versatile, but it doesn't always cooperate with gcc compiled apps. It may be a good strategy to make the troublesome apps which won't compile with ICC compile with ICC. Pro's: 1.) Bloody fast machine code. Intel obfuscates their architecture but they give back to the community as much as possible to make their hardware marketable toward the open source sysadmin, developer, etc etc. Their drivers are open and they develop for the kernel constantly. This cooperation leads me to believe that they would assist a team of developers in making 100% icc compatible code. 2.) Bloody fast compilation time. In my experience the compiler works much faster even with heavy optimization. 3.) Takes full advantage of SSE enabled hardware. SIMD instructions are quite useful, code is extremely vectorized. 4.) will project gentoo toward the power user more, helps the gentoo image, and overall will make linux a more professional operating system (and a quite competitive alternative to something like a SPARC+Solaris configuration). This would also make cluster farms and science application more respectful toward the gentoo community. The academic and research world already uses ICC to compile their apps for the sake of speed. The interprocedural optimizations for both the fortran and c/c++ compilers make it a must. 5.) It's free, albeit a commercial product. As gentoo is entirely non-profit, there is no restriction when it comes to licensing. The binaries won't be sold for the intel-compiled livecd, and the compiler itself with a fetch restriction allows the user to legally register for their free non-commercial license. -- gentoo-dev@lists.gentoo.org mailing list
Re: [gentoo-dev] ICC Profile
There is some record of a version of the kernel being compiled with some patches involved. It's probably possible, I'd imagine. Though, this is not necessary for the first builds. -- gentoo-dev@lists.gentoo.org mailing list
Re: [gentoo-dev] ICC Profile
Oh yes, and we can also take advantage of the free support that intel offers for all users: http://www.intel.com/support/performancetools/sb/CS-017156.htm Not to mention they have forums filled with intel developers/experts which we can get involved. I'm sure intel would benefit from this and be more than excited to collaborate: http://softwarecommunity.intel.com/isn/Community/en-US/forums/default.aspx -- gentoo-dev@lists.gentoo.org mailing list
Re: [gentoo-dev] ICC Profile
On 14:23 Thu 17 Jul , Adam Stylinski wrote:
The intel C Compiler (icc) has an ebuild for gentoo and the wiki has a
script to integrate it with portage. This script works will in terms
of building binaries, however when mixed with gcc environments there
are massive linking issues. I propose that an ICC profile is made
which contains specific versions and default flags for people who want
to build a mixed icc-gcc environment. ICC is much faster than GCC and
although not free, offers a free non-commercial license. I would be
very interested in this project and more than willing to help to the
best of my abilities. I've already been trying to maintain a mixed
environment with some luck, while there have been a lot of problems
using dynamically linked libraries (ld from intel and ld from gcc
don't always get along), my system is substatially faster. The kernel
obviously will still be built under gcc as well as bash (unless intel
helps submit patches to make the code work with their compiler).
There are many tools icc has to offer for vectorization. If these
were streamlined into Gentoo with a fetch restriction for ICC, a
bootsrapping boot disk could be made and result in a very fast
distribution. -- gentoo-dev@lists.gentoo.org mailing list
Adam,
I've already got a mixed system that works reasonably well. I use
/etc/portage/env/ to control which packages build with icc and
icc-specific CFLAGS by having /etc/portage/env/$CATEGORY/$PN be symlinks
to /etc/portage/env/env.icc with these contents:
echo * Exporting: Intel compilers in $EBUILD_PHASE
CC=icc
CXX=icc
FC=ifort
F77=ifort
FFLAGS=-openmp -parallel -ipo -xO -O2 -no-prec-div
FCFLAGS=${FFLAGS}
export CC CXX FC F77 FFLAGS FCFLAGS
Is what you're proposing here an improvement over something like the
above setup? If so, could you explain how?
--
Thanks,
Donnie
Donnie Berkholz
Developer, Gentoo Linux
Blog: http://dberkholz.wordpress.com
pgpRKtYUHvEJk.pgp
Description: PGP signature
Re: [gentoo-dev] ICC Profile
On 23:24 Thu 17 Jul , Adam Stylinski wrote: There are very few pitfalls, none of which I see as real killers. These include: 1.) Closed source compiler: Yes this stands against what we believe, and yes by closing their source they're protecting the trade secrets of their architecture. It also could be more difficult to debug, although that's highly unlikely, they have the idb (intel debugger) which works very much like gdb. Doesn't really matter, although we could never move to it as the default without violating our social contract. 2.) Linking issues: So far it's pretty versatile, but it doesn't always cooperate with gcc compiled apps. It may be a good strategy to make the troublesome apps which won't compile with ICC compile with ICC. Pretty sure you didn't mean ICC twice here, but sure. Pro's: 1.) Bloody fast machine code. Intel obfuscates their architecture but they give back to the community as much as possible to make their hardware marketable toward the open source sysadmin, developer, etc etc. Their drivers are open and they develop for the kernel constantly. This cooperation leads me to believe that they would assist a team of developers in making 100% icc compatible code. OK. This involves upstream projects more than us, though. 2.) Bloody fast compilation time. In my experience the compiler works much faster even with heavy optimization. 3.) Takes full advantage of SSE enabled hardware. SIMD instructions are quite useful, code is extremely vectorized. Sure, sure, speed is nice. 4.) will project gentoo toward the power user more, helps the gentoo image, and overall will make linux a more professional operating system (and a quite competitive alternative to something like a SPARC+Solaris configuration). I don't buy any parts of this argument, although the rest of your email is pretty good. This would also make cluster farms and science application more respectful toward the gentoo community. The academic and research world already uses ICC to compile their apps for the sake of speed. The interprocedural optimizations for both the fortran and c/c++ compilers make it a must. Yes, ICC is nice. And people can already use it easily within Gentoo for performance-critical apps. 5.) It's free, albeit a commercial product. As gentoo is entirely non-profit, there is no restriction when it comes to licensing. The binaries won't be sold for the intel-compiled livecd, and the compiler itself with a fetch restriction allows the user to legally register for their free non-commercial license. OK, so we can't sell icc-compiled software in our Gentoo store, so the releases would always remain built with gcc. -- Thanks, Donnie Donnie Berkholz Developer, Gentoo Linux Blog: http://dberkholz.wordpress.com pgpyuC3bLpdh5.pgp Description: PGP signature
