Re: [Hdf-forum] building hdf5-1.8.10-patch1: H5_CFLAGS variable problems for Cray compiler
Hi Neil, Since the configure process is executed in the front end which is a Linux system, configure naturally thinks it is working in a Linux system, therefore it goes to the Linux configure settings. I think the right solution is to tell configure to build for a cross-compiling Cray system. This will take some effort and won't be available in the pending release v1.8.11. We are planning to provide it in the next release. -Albert Cheng The HDF Group On 5/2/13 3:49 PM, Neil wrote: Hi Albert, We are using a Cray XK6 running Cray's Compute Linux Environment (CLE) with the Cray CCE compiler. The front end nodes run SLES Linux and the compute nodes run CLE. So when building HDF5 (and any other application/library) it's a cross compilation. I can get the library to build successfully, both serial and parallel versions by hacking the config/linux-gnulibcl to remove the inclusion of the intel-flags and intel-fflags files (as stated previously). It would be handy if the configure script could correctly identify the Cray CCE compiler as some users may not be aware that this is the problem. We use pretty much the same method as described in the release_docs/INSTALL_parallel. As the doc says, the key is to set the RUNPARALLEL and RUNSERIAL environment variables. With the Cray CCE compiler it doesn't generate Fortran module files by default, so you have to make sure that -em is set in FFLAGS. For CFLAGS and CXXFLAGS we also have to set -h tolerant -h system_alloc. I'll try to post our configure options for both serial and parallel builds tomorrow when I next have access to the system in case this is of help. Cheers, Neil. -- View this message in context: http://hdf-forum.184993.n3.nabble.com/building-hdf5-1-8-10-patch1-H5-CFLAGS-variable-problems-for-Cray-compiler-tp4026036p4026154.html Sent from the hdf-forum mailing list archive at Nabble.com. ___ Hdf-forum is for HDF software users discussion. Hdf-forum@hdfgroup.org http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org ___ Hdf-forum is for HDF software users discussion. Hdf-forum@hdfgroup.org http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
Re: [Hdf-forum] building hdf5-1.8.10-patch1: H5_CFLAGS variable problems for Cray compiler
Albert, On 5/7/2013 12:50 PM, Albert Cheng wrote: Since the configure process is executed in the front end which is a Linux system, configure naturally thinks it is working in a Linux system, therefore it goes to the Linux configure settings. I think the right solution is to tell configure to build for a cross-compiling Cray system. This will take some effort and won't be available in the pending release v1.8.11. We are planning to provide it in the next release. The Cray wrappers that you provide with configure (cc and ftn) should do all the cross compiling you need. The bug that Neil is talking about does not have to do with cross compiling. There is just an issue when Cray compilers are used in the Cray build wrappers. This issue, we hope, would be fixed in the next release. We recommend now that you don't use the cray compilers. Usually Cray supercomputers provide other compilers like PGI and GNU that should work fine. Thanks, Mohamad ___ Hdf-forum is for HDF software users discussion. Hdf-forum@hdfgroup.org http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
Re: [Hdf-forum] building hdf5-1.8.10-patch1: H5_CFLAGS variable problems for Cray compiler
Here's how we build HDF5 1.8.x on our Cray XK6 for CLE:export CC=ccexport CFLAGS=-O2 -h tolerant -h system_allocexport CXX=CCexport CXXFLAGS=-O2 -h tolerant -h system_allocexport F77=ftnexport F90=ftnexport F9X=ftnexport FFLAGS=-O2 -emexport RUNSERIAL=aprun -n 1 -N 1 -qexport RUNPARALLEL=aprun -n 2 -N 1 -qmake distclean./configure --prefix=/path/to/serial-install/dir --with-zlib --disable-shared --enable-fortran --enable cxx --enable-linux-lfsmake install./configure --prefix=/path/to/parallel-install/dir --with-zlib --disable-shared --enable-fortran --disable cxx --enable-linux-lfs --enable-parallelmake install -- View this message in context: http://hdf-forum.184993.n3.nabble.com/building-hdf5-1-8-10-patch1-H5-CFLAGS-variable-problems-for-Cray-compiler-tp4026036p4026156.html Sent from the hdf-forum mailing list archive at Nabble.com.___ Hdf-forum is for HDF software users discussion. Hdf-forum@hdfgroup.org http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
Re: [Hdf-forum] building hdf5-1.8.10-patch1: H5_CFLAGS variable problems for Cray compiler
Hi Neil, What kind of Cray and OS version do you use? Does it have a frontend for compiling and a backend for compute? If so, what kind of machine and OS version is the frontend? There are problems with the configure process for the Cray platform that we will not be able to fix it in time for this pending release. We plan to fix it in the next release. Meanwhile, can you follow the instruction in release_docs/INSTALL_parallel, under section 2.4. Hopper (Cray XE6) (for v1.8 and later) to see if you can build HDF5 in your Cray? By the way, are you building the serial HDF5 library or the Parallel HDF5 which requires MPI support? -Albert Chen The HDF Group On 4/30/13 4:57 AM, Neil wrote: Hi again, As promised, here is the version output information for the Cray compiler (CCE). Note: this version information is obtained using the command: cc -V. |cc -v| and |cc --version| are not supported. |cc -V /opt/cray/xt-asyncpe/5.17/bin/cc: INFO: Compiling with CRAYPE_COMPILE_TARGET=native. Cray C : Version 8.1.7 Tue Apr 30, 2013 09:48:17| I have also tried out HDF5 1.8.11-pre2, but it also incorrectly identifies the Cray compiler as the Intel compiler. Hope this helps, Neil. View this message in context: Re: building hdf5-1.8.10-patch1: H5_CFLAGS variable problems for Cray compiler http://hdf-forum.184993.n3.nabble.com/building-hdf5-1-8-10-patch1-H5-CFLAGS-variable-problems-for-Cray-compiler-tp4026036p4026141.html Sent from the hdf-forum mailing list archive http://hdf-forum.184993.n3.nabble.com/ at Nabble.com. ___ Hdf-forum is for HDF software users discussion. Hdf-forum@hdfgroup.org http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org ___ Hdf-forum is for HDF software users discussion. Hdf-forum@hdfgroup.org http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
Re: [Hdf-forum] building hdf5-1.8.10-patch1: H5_CFLAGS variable problems for Cray compiler
Hi Albert, We are using a Cray XK6 running Cray's Compute Linux Environment (CLE) with the Cray CCE compiler. The front end nodes run SLES Linux and the compute nodes run CLE. So when building HDF5 (and any other application/library) it's a cross compilation. I can get the library to build successfully, both serial and parallel versions by hacking the config/linux-gnulibcl to remove the inclusion of the intel-flags and intel-fflags files (as stated previously). It would be handy if the configure script could correctly identify the Cray CCE compiler as some users may not be aware that this is the problem. We use pretty much the same method as described in the release_docs/INSTALL_parallel. As the doc says, the key is to set the RUNPARALLEL and RUNSERIAL environment variables. With the Cray CCE compiler it doesn't generate Fortran module files by default, so you have to make sure that -em is set in FFLAGS. For CFLAGS and CXXFLAGS we also have to set -h tolerant -h system_alloc. I'll try to post our configure options for both serial and parallel builds tomorrow when I next have access to the system in case this is of help. Cheers, Neil. -- View this message in context: http://hdf-forum.184993.n3.nabble.com/building-hdf5-1-8-10-patch1-H5-CFLAGS-variable-problems-for-Cray-compiler-tp4026036p4026154.html Sent from the hdf-forum mailing list archive at Nabble.com. ___ Hdf-forum is for HDF software users discussion. Hdf-forum@hdfgroup.org http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
Re: [Hdf-forum] building hdf5-1.8.10-patch1: H5_CFLAGS variable problems for Cray compiler
Hi again, As promised, here is the version output information for the Cray compiler (CCE). Note: this version information is obtained using the command: cc -V. cc -v and cc --version are not supported. cc -V /opt/cray/xt-asyncpe/5.17/bin/cc: INFO: Compiling with CRAYPE_COMPILE_TARGET=native. Cray C : Version 8.1.7 Tue Apr 30, 2013 09:48:17 I have also tried out HDF5 1.8.11-pre2, but it also incorrectly identifies the Cray compiler as the Intel compiler. Hope this helps, Neil. -- View this message in context: http://hdf-forum.184993.n3.nabble.com/building-hdf5-1-8-10-patch1-H5-CFLAGS-variable-problems-for-Cray-compiler-tp4026036p4026141.html Sent from the hdf-forum mailing list archive at Nabble.com.___ Hdf-forum is for HDF software users discussion. Hdf-forum@hdfgroup.org http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
Re: [Hdf-forum] building hdf5-1.8.10-patch1: H5_CFLAGS variable problems for Cray compiler
Hi Mohamad and Albert, Thanks for the replies. I'll try out the 1.8.11 snapshot later this week and let you know how I get on as well as looking at the version information produced by the Cray compilers. As it happens the cc, CC and ftn commands on the Cray (both under Catamount and CLE) are all wrapper scripts to the underlying compilers that were created by Cray. The scripts just examine the user's environment to build up the compiler flags, include options and libraries to link against. So if you have the PrgEnv-pgi module loaded the wrapper scripts will call the PGI compilers for example. In the case of the Cray compilers the wrapper scripts will call craycc etc. under the covers. Hope this helps, Neil. -- View this message in context: http://hdf-forum.184993.n3.nabble.com/building-hdf5-1-8-10-patch1-H5-CFLAGS-variable-problems-for-Cray-compiler-tp4026036p4026140.html Sent from the hdf-forum mailing list archive at Nabble.com. ___ Hdf-forum is for HDF software users discussion. Hdf-forum@hdfgroup.org http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
Re: [Hdf-forum] building hdf5-1.8.10-patch1: H5_CFLAGS variable problems for Cray compiler
Hi Neil, I worked in a Cray about 3 to 4 years ago. Back then the cc and ftn were actually cross compilers provided by PGI. Maybe they are changed and supported by Intel. One of our staff has more recent experience with Cray XE6, Cray Linux Environment (CLE) PrgEnv-pgi/4.0.46 hopper.nersc.gov pgcc 12.5-0 64-bit target on x86-64 Linux -tp shanghai pgf90 12.5-0 64-bit target on x86-64 Linux -tp shanghai pgCC 12.5-0 64-bit target on x86-64 Linux -tp shanghai We are in a pre-release testing for v1.8.11 and a pre-release source tarball is available at: http://www.hdfgroup.uiuc.edu/ftp/pub/outgoing/hdf5/hdf5-1.8.11/hdf5-1.8.11-pre1.tar.gz After your have downloaded the tar file, untar it into hdf5-1.8.11-pre1/, cd into it and tried to follow the instruction in release_docs/INSTALL_parallel, under 2.4. Red Storm (Cray XT3) (for v1.8 and later) or 2.5. Hopper (Cray XE6) (for v1.8 and later). The section 2.4 is more detailed but it is old information for Red Storm. The section 2.5 is current. Please see which one is more relevant to your Cray. (And let me know too.) If it still fails for you in any way, please send me all these information: 1. Machine information (Cray so and so) 2. All commands used and their output. 3. compiler versions (e.g. output of cc --version) Thanks. -Albert Cheng The HDF Group On 4/24/13 9:57 AM, Neil wrote: Hi, I too have had the same problem with both HDF5 1.8.5 and 1.8.8 with the Cray compiler. The -std=c99 flag gets added because the HDF5 configure script incorrectly identifies the Cray compiler has the Intel compiler and consequently it automatically sets a load of compiler flags that are intended for the Intel compilers. To workarround this edit config/linux-gnulibcl and comment out the two lines that source files named intel-flags and intel-fflags. It would be handy if future versions of HDF5 could detect the Cray compiler correctly. Hope this helps, Neil. -- View this message in context: http://hdf-forum.184993.n3.nabble.com/building-hdf5-1-8-10-patch1-H5-CFLAGS-variable-problems-for-Cray-compiler-tp4026036p4026136.html Sent from the hdf-forum mailing list archive at Nabble.com. ___ Hdf-forum is for HDF software users discussion. Hdf-forum@hdfgroup.org http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org ___ Hdf-forum is for HDF software users discussion. Hdf-forum@hdfgroup.org http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
Re: [Hdf-forum] building hdf5-1.8.10-patch1: H5_CFLAGS variable problems for Cray compiler
I managed to replicate the issue. We test only with the default PGI compilers, which work quite fine. However when I switch to the Cray compilers, I see the same issue you guys are seeing. We'll add it to our bug tracking system. Thanks for reporting! Mohamad On 4/26/2013 9:47 AM, Albert Cheng wrote: Hi Neil, I worked in a Cray about 3 to 4 years ago. Back then the cc and ftn were actually cross compilers provided by PGI. Maybe they are changed and supported by Intel. One of our staff has more recent experience with Cray XE6, Cray Linux Environment (CLE) PrgEnv-pgi/4.0.46 hopper.nersc.gov pgcc 12.5-0 64-bit target on x86-64 Linux -tp shanghai pgf90 12.5-0 64-bit target on x86-64 Linux -tp shanghai pgCC 12.5-0 64-bit target on x86-64 Linux -tp shanghai We are in a pre-release testing for v1.8.11 and a pre-release source tarball is available at: http://www.hdfgroup.uiuc.edu/ftp/pub/outgoing/hdf5/hdf5-1.8.11/hdf5-1.8.11-pre1.tar.gz After your have downloaded the tar file, untar it into hdf5-1.8.11-pre1/, cd into it and tried to follow the instruction in release_docs/INSTALL_parallel, under 2.4. Red Storm (Cray XT3) (for v1.8 and later) or 2.5. Hopper (Cray XE6) (for v1.8 and later). The section 2.4 is more detailed but it is old information for Red Storm. The section 2.5 is current. Please see which one is more relevant to your Cray. (And let me know too.) If it still fails for you in any way, please send me all these information: 1. Machine information (Cray so and so) 2. All commands used and their output. 3. compiler versions (e.g. output of cc --version) Thanks. -Albert Cheng The HDF Group On 4/24/13 9:57 AM, Neil wrote: Hi, I too have had the same problem with both HDF5 1.8.5 and 1.8.8 with the Cray compiler. The -std=c99 flag gets added because the HDF5 configure script incorrectly identifies the Cray compiler has the Intel compiler and consequently it automatically sets a load of compiler flags that are intended for the Intel compilers. To workarround this edit config/linux-gnulibcl and comment out the two lines that source files named intel-flags and intel-fflags. It would be handy if future versions of HDF5 could detect the Cray compiler correctly. Hope this helps, Neil. -- View this message in context: http://hdf-forum.184993.n3.nabble.com/building-hdf5-1-8-10-patch1-H5-CFLAGS-variable-problems-for-Cray-compiler-tp4026036p4026136.html Sent from the hdf-forum mailing list archive at Nabble.com. ___ Hdf-forum is for HDF software users discussion. Hdf-forum@hdfgroup.org http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org ___ Hdf-forum is for HDF software users discussion. Hdf-forum@hdfgroup.org http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org ___ Hdf-forum is for HDF software users discussion. Hdf-forum@hdfgroup.org http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
Re: [Hdf-forum] building hdf5-1.8.10-patch1: H5_CFLAGS variable problems for Cray compiler
Oh and please follow Albert's instructions about giving us more information about your system and build failure. More information = Better fix :-) Thanks, Mohamad On 4/26/2013 9:56 AM, Mohamad Chaarawi wrote: I managed to replicate the issue. We test only with the default PGI compilers, which work quite fine. However when I switch to the Cray compilers, I see the same issue you guys are seeing. We'll add it to our bug tracking system. Thanks for reporting! Mohamad On 4/26/2013 9:47 AM, Albert Cheng wrote: Hi Neil, I worked in a Cray about 3 to 4 years ago. Back then the cc and ftn were actually cross compilers provided by PGI. Maybe they are changed and supported by Intel. One of our staff has more recent experience with Cray XE6, Cray Linux Environment (CLE) PrgEnv-pgi/4.0.46 hopper.nersc.gov pgcc 12.5-0 64-bit target on x86-64 Linux -tp shanghai pgf90 12.5-0 64-bit target on x86-64 Linux -tp shanghai pgCC 12.5-0 64-bit target on x86-64 Linux -tp shanghai We are in a pre-release testing for v1.8.11 and a pre-release source tarball is available at: http://www.hdfgroup.uiuc.edu/ftp/pub/outgoing/hdf5/hdf5-1.8.11/hdf5-1.8.11-pre1.tar.gz After your have downloaded the tar file, untar it into hdf5-1.8.11-pre1/, cd into it and tried to follow the instruction in release_docs/INSTALL_parallel, under 2.4. Red Storm (Cray XT3) (for v1.8 and later) or 2.5. Hopper (Cray XE6) (for v1.8 and later). The section 2.4 is more detailed but it is old information for Red Storm. The section 2.5 is current. Please see which one is more relevant to your Cray. (And let me know too.) If it still fails for you in any way, please send me all these information: 1. Machine information (Cray so and so) 2. All commands used and their output. 3. compiler versions (e.g. output of cc --version) Thanks. -Albert Cheng The HDF Group On 4/24/13 9:57 AM, Neil wrote: Hi, I too have had the same problem with both HDF5 1.8.5 and 1.8.8 with the Cray compiler. The -std=c99 flag gets added because the HDF5 configure script incorrectly identifies the Cray compiler has the Intel compiler and consequently it automatically sets a load of compiler flags that are intended for the Intel compilers. To workarround this edit config/linux-gnulibcl and comment out the two lines that source files named intel-flags and intel-fflags. It would be handy if future versions of HDF5 could detect the Cray compiler correctly. Hope this helps, Neil. -- View this message in context: http://hdf-forum.184993.n3.nabble.com/building-hdf5-1-8-10-patch1-H5-CFLAGS-variable-problems-for-Cray-compiler-tp4026036p4026136.html Sent from the hdf-forum mailing list archive at Nabble.com. ___ Hdf-forum is for HDF software users discussion. Hdf-forum@hdfgroup.org http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org ___ Hdf-forum is for HDF software users discussion. Hdf-forum@hdfgroup.org http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org ___ Hdf-forum is for HDF software users discussion. Hdf-forum@hdfgroup.org http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org ___ Hdf-forum is for HDF software users discussion. Hdf-forum@hdfgroup.org http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
