Hi, Maybe you can leverage some of the techniques outlined in:
Robert W. Robey, Jonathan M. Robey, and Rob Aulwes. 2011. In search of numerical consistency in parallel programming. Parallel Comput. 37, 4-5 (April 2011), 217-229. DOI=10.1016/j.parco.2011.02.009 http://dx.doi.org/10.1016/j.parco.2011.02.009 Hope that helps, Samuel K. Gutierrez Los Alamos National Laboratory On Sep 20, 2011, at 6:25 AM, Reuti wrote: > Am 20.09.2011 um 13:52 schrieb Tim Prince: > >> On 9/20/2011 7:25 AM, Reuti wrote: >>> Hi, >>> >>> Am 20.09.2011 um 00:41 schrieb Blosch, Edwin L: >>> >>>> I am observing differences in floating-point results from an application >>>> program that appear to be related to whether I link with OpenMPI 1.4.3 or >>>> MVAPICH 1.2.0. Both packages were built with the same installation of >>>> Intel 11.1, as well as the application program; identical flags passed to >>>> the compiler in each case. >>>> >>>> I’ve tracked down some differences in a compute-only routine where I’ve >>>> printed out the inputs to the routine (to 18 digits) ; the inputs are >>>> identical. The output numbers are different in the 16th place (perhaps a >>>> few in the 15th place). These differences only show up for optimized >>>> code, not for –O0. >>>> >>>> My assumption is that some optimized math intrinsic is being replaced >>>> dynamically, but I do not know how to confirm this. Anyone have guidance >>>> to offer? Or similar experience? >>> >>> yes, I face it often but always at a magnitude where it's not of any >>> concern (and not related to any MPI). Due to the limited precision in >>> computers, a simple reordering of operation (although being equivalent in a >>> mathematical sense) can lead to different results. Removing the anomalies >>> with -O0 could proof that. >>> >>> The other point I heard especially for the x86 instruction set is, that the >>> internal FPU has still 80 bits, while the presentation in memory is only 64 >>> bit. Hence when all can be done in the registers, the result can be >>> different compared to the case when some interim results need to be stored >>> to RAM. For the Portland compiler there is a switch -Kieee -pc64 to force >>> it to stay always in 64 bit, and a similar one for Intel is -mp (now >>> -fltconsistency) and -mp1. >>> >> Diagnostics below indicate that ifort 11.1 64-bit is in use. The options >> aren't the same as Reuti's "now" version (a 32-bit compiler which hasn't >> been supported for 3 years or more?). > > In the 11.1 documentation they are also still listed: > > http://software.intel.com/sites/products/documentation/hpc/compilerpro/en-us/fortran/lin/compiler_f/index.htm > > I read it in the way, that -mp is deprecated syntax (therefore listed under > "Alternate Options"), but -fltconsistency is still a valid and supported > option. > > -- Reuti > > >> With ifort 10.1 and more recent, you would set at least >> -assume protect_parens -prec-div -prec-sqrt >> if you are interested in numerical consistency. If you don't want >> auto-vectorization of sum reductions, you would use instead >> -fp-model source -ftz >> (ftz sets underflow mode back to abrupt, while "source" sets gradual). >> It may be possible to expose 80-bit x87 by setting the ancient -mp option, >> but such a course can't be recommended without additional cautions. >> >> Quoted comment from OP seem to show a somewhat different question: Does >> OpenMPI implement any operations in a different way from MVAPICH? I would >> think it probable that the answer could be affirmative for operations such >> as allreduce, but this leads well outside my expertise with respect to >> specific MPI implementations. It isn't out of the question to suspect that >> such differences might be aggravated when using excessively aggressive ifort >> options such as -fast. >> >> >>>> libifport.so.5 => >>>> /opt/intel/Compiler/11.1/072/lib/intel64/libifport.so.5 >>>> (0x00002b6e7e081000) >>>> libifcoremt.so.5 => >>>> /opt/intel/Compiler/11.1/072/lib/intel64/libifcoremt.so.5 >>>> (0x00002b6e7e1ba000) >>>> libimf.so => /opt/intel/Compiler/11.1/072/lib/intel64/libimf.so >>>> (0x00002b6e7e45f000) >>>> libsvml.so => /opt/intel/Compiler/11.1/072/lib/intel64/libsvml.so >>>> (0x00002b6e7e7f4000) >>>> libintlc.so.5 => >>>> /opt/intel/Compiler/11.1/072/lib/intel64/libintlc.so.5 (0x00002b6e7ea0a000) >>>> >> >> -- >> Tim Prince >> _______________________________________________ >> users mailing list >> us...@open-mpi.org >> http://www.open-mpi.org/mailman/listinfo.cgi/users >> > > > _______________________________________________ > users mailing list > us...@open-mpi.org > http://www.open-mpi.org/mailman/listinfo.cgi/users
