Re: [ccp4bb] Refmac executables - win vs linux in RHEL VM
On Sat, Apr 07, 2012 at 08:42:47AM -0700, Bernhard Rupp (Hofkristallrat a.D.) wrote: Something the developers might be interested in: The Refmac_5.6.0117 32-bit windows binaries run native on a win64 3-4x slower than those from the linux distribution run Thanks for benchmarking. If you'd like to test refmac 5.7 that will be included in CCP4 6.3 I can make pre-release binaries later this week and put it on ftp. (I suppose it won't be slower than Linux version, but we'll see.) One thing that slows down *both* Linux and Windows version is the GFORTRAN_UNBUFFERED_ALL env. variable. It was set as a temporary workaround in ver. 6.2, but has been already removed from the coming release. If you unset it for refmac from 6.2 you can get a few lines in the output (in terminal) in wrong order, but the program should run notably faster. Most peculiaralthough I think but I do not know whether the linux binaries are 64 bit If it's from CCP4 it's 32-bit. You can check it by typing file /path/to/binary. You'll get in the output either ELF 32-bit LSB executable or ELF 64-bit LSB executable Marcin
Re: [ccp4bb] Refmac executables - win vs linux in RHEL VM
Hi Bernhard, Maybe the paranoia-checkers in windows slow everything down although I did not see any resources overwhelmed... I wonder whether the windoze refmac binaries can be used through wine in a GNU/Linux environment. If yes, then you could possibly differentiate between the operating-system-dependent and compiler-specific hypotheses. Nicholas -- Nicholas M. Glykos, Department of Molecular Biology and Genetics, Democritus University of Thrace, University Campus, Dragana, 68100 Alexandroupolis, Greece, Tel/Fax (office) +302551030620, Ext.77620, Tel (lab) +302551030615, http://utopia.duth.gr/~glykos/
Re: [ccp4bb] Refmac executables - win vs linux in RHEL VM
Hi Nat, one of my colleagues found (on Linux) that the exp() function provided by g77 was 20-fold slower than the equivalent in the Intel math library. I do not know whether this has recently been changed, but the license for icc-produced executables used to be rather restrictive. If I remember correctly, you were not allowed to distribute the binaries, full stop. This together with the fact that until recently (icc v.11.0.074) the icc-produced executables would not run on specific AMD-based hardware, had made me return to the safety of gcc. My twocents, Nicholas -- Nicholas M. Glykos, Department of Molecular Biology and Genetics, Democritus University of Thrace, University Campus, Dragana, 68100 Alexandroupolis, Greece, Tel/Fax (office) +302551030620, Ext.77620, Tel (lab) +302551030615, http://utopia.duth.gr/~glykos/
Re: [ccp4bb] Refmac executables - win vs linux in RHEL VM
I do not know whether this has recently been changed, but the license for icc-produced executables used to be rather restrictive. If I remember correctly, you were not allowed to distribute the binaries, full stop. Nicholas, this restriction applies (and has always applied) only to Intel's 'evaluation' licence: i.e. you get to try the Intel compilers free for 1 month, but you're not allowed to redistribute any executables you create with them. I don't know if this means that the software actually stops working after a month, I guess it does -they're not as trusting as they used to be! Intel's EULA for all their Software Development Products (including all their compilers) states: Subject to all of the terms and conditions of this Agreement and any specific restrictions which may appear in the Redistributables text files, Intel grants to you a non-exclusive, non-assignable, fully-paid copyright license to distribute (except if you received the Materials under an Evaluation License as specified below) the Redistributables, including any modifications pursuant to Section 2.B, or any portions thereof, as part of the product or application you developed using the Materials.. I had our lawyers check this ~10 years ago when the compiler was at version ~7 (it's now at 11), since we are commercial and wanted to distribute our own sources executables, and the conditions on redistribution of user-created executables have not changed in essence since then (obviously redistribution of the compiler executables themselves has never been allowed). What has changed is that the licence conditions have become somewhat more restrictive in the sense that academic institutional users are no longer eligible for free licences! - though they do get a discount off the fully paid-up commercial licence. A personal non-commercial licence (which does not cover use by academics) is still free. In all cases (except evaluation) executables can be freely distributed, along with any of Intel's DLLs that are required to run it. Please note that I have no financial interest in Intel ;). Cheers -- Ian
Re: [ccp4bb] Refmac executables - win vs linux in RHEL VM
Hi I suspect that this is more to do with the amount of memory required, size of arrays etc; refinement will (in general) be more demanding in terms of these than an integration program like Mosflm. The last time I compared the Mosflm performance (which was a few years ago), running the same batch job on OSX 10.4 (Tiger), and on Windows XP and Linux Feisty Fawn (so you can tell how long ago this was) - both the latter running under virtual machines on the same 32-bit Intel Mac that the OSX job ran on) there was essentially no difference in performance (though I have a vague memory of Ubuntu being a little faster, maybe ~3%). Some caveats - * I used a gfortran build for OSX and Linux, g77 build for Windows * I didn't spend too much time on this * I wasn't running a GUI - all three as foregrounded jobs, nothing else running on the machine (I tried to make sure only the OS and essential services were running). So this wasn't a batch job in the traditional sense... * gfortran builds these days are considerably faster (and compare well to ifort builds) On 7 Apr 2012, at 17:50, Roger Rowlett wrote: I don't know the state of current software, because I haven't tried recently, but when I set up my student crystallography workstations a few years back I noticed many packages (e.g. EPMR, Phaser) that had potentially long run times (where it is really noticeable) would run on the identical hardware about 2-3 times faster in Linux than in Windows XP. Memory swapping wasn't the issue. I was astounded there could be that much overhead in Windows. A Linux VM on a windows machine being faster than native Win7 is pretty weird, though. Cheers, On 4/7/2012 11:42 AM, Bernhard Rupp (Hofkristallrat a.D.) wrote: Something the developers might be interested in: The Refmac_5.6.0117 32-bit windows binaries run native on a win64 3-4x slower than those from the linux distribution run **in a RHEL6.2-64 VMware virtual machine hosted the same windows7/64 system.** VM/RHEL: Refmac_5.6.0117: End of Refmac_5.6.0117 Times: User:1015.3s System: 135.0s Elapsed:19:17 Win native Refmac_5.6.0117: End of Refmac_5.6.0117 Times: User: 0.0s System:0.0s Elapsed:67:49 Most peculiaralthough I think but I do not know whether the linux binaries are 64 bit I don't think that address space is the issue here if they are. Maybe the paranoia-checkers in windows slow everything down although I did not see any resources overwhelmed... Best regards, BR - Bernhard Rupp 001 (925) 209-7429 +43 (676) 571-0536 b...@ruppweb.org hofkristall...@gmail.com http://www.ruppweb.org/ - No animals were hurt or killed during the production of this email. - Harry -- Dr Harry Powell, MRC Laboratory of Molecular Biology, MRC Centre, Hills Road, Cambridge, CB2 0QH
Re: [ccp4bb] Refmac executables - win vs linux in RHEL VM
Hi Ian, Nicholas, this restriction applies (and has always applied) only to Intel's 'evaluation' licence That's right. With a cost of $9,997.00 for a 3-years/2-seats academic license, I couldn't have been talking for anything else ... :-))) All the best, Nicholas -- Nicholas M. Glykos, Department of Molecular Biology and Genetics, Democritus University of Thrace, University Campus, Dragana, 68100 Alexandroupolis, Greece, Tel/Fax (office) +302551030620, Ext.77620, Tel (lab) +302551030615, http://utopia.duth.gr/~glykos/
Re: [ccp4bb] Refmac executables - win vs linux in RHEL VM
On Sun, Apr 08, 2012 at 03:59:22PM +0300, Nikolaos Glykos wrote: Nicholas, this restriction applies (and has always applied) only to Intel's 'evaluation' licence That's right. With a cost of $9,997.00 for a 3-years/2-seats academic license, I couldn't have been talking for anything else ... :-))) Is that a joke? Or did I miss something? We pay about $900 USD/year for our single seat, academic license that includes both the Linux and OS X versions of the Intel Compilers. And if you're an active scientific software developer, we'll let you use them for free: http://www.sbgrid.org/wiki/developers/support -ben -- | Ben Eisenbraun | SBGrid Consortium | http://sbgrid.org | | Harvard Medical School | http://hms.harvard.edu |
Re: [ccp4bb] Refmac executables - win vs linux in RHEL VM
That's right. With a cost of $9,997.00 for a 3-years/2-seats academic license, I couldn't have been talking for anything else ... :-))) Hi Nicholas That sounds like way more than it should be, in fact it sounds like you've been quoted the cost of the commercial licence and then some! From Intel's website the academic licence for icc (Linux/2 seats) is $570 incl 1 year's support. Renewal of support for subsequent years will be less than this, probably around $250/year. I have ifort + icc (Linux/single user) we paid about $1200 for the 1st year, and $500 for subsequent year's support. Cheers -- Ian
Re: [ccp4bb] Refmac executables - win vs linux in RHEL VM
Hi Ian, That sounds like way more than it should be, in fact it sounds like you've been quoted the cost of the commercial licence and then some! From Intel's website the academic licence for icc (Linux/2 seats) is $570 incl 1 year's support. Renewal of support for subsequent years will be less than this, probably around $250/year. I have ifort + icc (Linux/single user) we paid about $1200 for the 1st year, and $500 for subsequent year's support. The $9,997.00 price I quoted are for the XE parallel studio versions (C,C++,Fortran,...) as given at http://softwarestore.ispfulfillment.com/store/Product.aspx?skupart=I23S74 (which is where the page at http://software.intel.com/en-us/intel-sdp-home/ directs to if you select the C++ compiler for linux). For the XE version of C++ the prices for 3-year/2-seat academic is $6,499.00 (http://softwarestore.ispfulfillment.com/store/Product.aspx?skupart=I23S76) and for Fortran alone is $7,800.00 (http://softwarestore.ispfulfillment.com/store/Product.aspx?skupart=I23S91) I do not doubt that the prices you quote are also correct for a different product line (and I do not have anything against Intel :-) Nicholas -- Nicholas M. Glykos, Department of Molecular Biology and Genetics, Democritus University of Thrace, University Campus, Dragana, 68100 Alexandroupolis, Greece, Tel/Fax (office) +302551030620, Ext.77620, Tel (lab) +302551030615, http://utopia.duth.gr/~glykos/
[ccp4bb] Refmac executables - win vs linux in RHEL VM
Something the developers might be interested in: The Refmac_5.6.0117 32-bit windows binaries run native on a win64 3-4x slower than those from the linux distribution run **in a RHEL6.2-64 VMware virtual machine hosted the same windows7/64 system.** VM/RHEL: Refmac_5.6.0117: End of Refmac_5.6.0117 Times: User:1015.3s System: 135.0s Elapsed:19:17 Win native Refmac_5.6.0117: End of Refmac_5.6.0117 Times: User: 0.0s System:0.0s Elapsed:67:49 Most peculiaralthough I think but I do not know whether the linux binaries are 64 bit I don't think that address space is the issue here if they are. Maybe the paranoia-checkers in windows slow everything down although I did not see any resources overwhelmed... Best regards, BR - Bernhard Rupp 001 (925) 209-7429 +43 (676) 571-0536 b...@ruppweb.org hofkristall...@gmail.com http://www.ruppweb.org/ - No animals were hurt or killed during the production of this email. -
Re: [ccp4bb] Refmac executables - win vs linux in RHEL VM
I don't know the state of current software, because I haven't tried recently, but when I set up my student crystallography workstations a few years back I noticed many packages (e.g. EPMR, Phaser) that had potentially long run times (where it is really noticeable) would run on the identical hardware about 2-3 times faster in Linux than in Windows XP. Memory swapping wasn't the issue. I was astounded there could be that much overhead in Windows. A Linux VM on a windows machine being faster than native Win7 is pretty weird, though. Cheers, On 4/7/2012 11:42 AM, Bernhard Rupp (Hofkristallrat a.D.) wrote: Something the developers might be interested in: The Refmac_5.6.0117 32-bit windows binaries run native on a win64 3-4x slower than those from the linux distribution run **in a RHEL6.2-64 VMware virtual machine hosted the same windows7/64 system.** VM/RHEL: Refmac_5.6.0117: End of Refmac_5.6.0117 Times: User:1015.3s System: 135.0s Elapsed:19:17 Win native Refmac_5.6.0117: End of Refmac_5.6.0117 Times: User: 0.0s System:0.0s Elapsed:67:49 Most peculiaralthough I think but I do not know whether the linux binaries are 64 bit I don't think that address space is the issue here if they are. Maybe the paranoia-checkers in windows slow everything down although I did not see any resources overwhelmed... Best regards, BR - Bernhard Rupp 001 (925) 209-7429 +43 (676) 571-0536 b...@ruppweb.org hofkristall...@gmail.com http://www.ruppweb.org/ - No animals were hurt or killed during the production of this email. -
Re: [ccp4bb] Refmac executables - win vs linux in RHEL VM
On Sat, Apr 7, 2012 at 9:50 AM, Roger Rowlett rrowl...@colgate.edu wrote: I don't know the state of current software, because I haven't tried recently, but when I set up my student crystallography workstations a few years back I noticed many packages (e.g. EPMR, Phaser) that had potentially long run times (where it is really noticeable) would run on the identical hardware about 2-3 times faster in Linux than in Windows XP. Memory swapping wasn't the issue. I was astounded there could be that much overhead in Windows. A Linux VM on a windows machine being faster than native Win7 is pretty weird, though. Different compiler implementations will often have a huge effect on runtimes. I recently spent some time trying to get a large amount of C++ code (converted from F77) to compile under Visual C++ 9.0, and I had to disable optimization of at least ten different functions to prevent cl.exe from crashing. This was not especially complex code (and g++ never complains) - just nested 'for' loops over three dimensions. I did not attempt to compare runtimes since I was running Windows in a virtual machine on a Mac, but I would be surprised if the resulting Windows binaries were not slower on identical hardware. And even if the compiler isn't broken, the math libraries may be; one of my colleagues found (on Linux) that the exp() function provided by g77 was 20-fold slower than the equivalent in the Intel math library. So I suspect it is related to the compilers (and optimization flags) used by CCP4 for these platforms. Another good reason to avoid Windows! -Nat
