Hi Albert, The distribution is SuSE, the contents of the SuSE-release file are
SUSE Linux Enterprise Server 11 (x86_64) VERSION = 11 PATCHLEVEL = 2 Thanks for looking into this. Nitya On 07/09/13 20:41, Albert Cheng wrote: > Hi Nitya, > > Looks like you are in a Linux 3.0 system. > Can you find out what distribution it is? E.g., Redhat, Fedora, Suse, …. > There is no one way to find this information but you often can find a file > like > /etc/<XYZ>-release. If you can locate such a file, please send its content. > > -Albert Cheng > > On Jul 4, 2013, at 4:31 AM, Nitya <[email protected]> wrote: > >> Hi Albert, Dana, >> >> Thanks for your replies. I give the output of uname below >> >> Linux 3.0.74-0.6.10-default #1 SMP x86_64 x86_64 x86_64 GNU/Linux >> >> I was earlier using v1.8.9 and then tried using v1.8.11. I have given >> the libhdf5.settings file for both versions below. I downloaded the >> library files of v1.8.11 from the HDF5 website, so haven't compiled >> anything. v1.8.9 has been compiled by the system administrators. >> >> ------------------- >> SUMMARY OF THE HDF5 CONFIGURATION >> ================================= >> >> General Information: >> ------------------- >> HDF5 Version: 1.8.9 >> Configured on: Thu Nov 29 11:11:42 CET 2012 >> Configured by: wwn@qa01 >> Configure mode: production >> Host system: x86_64-unknown-linux-gnu >> Uname information: Linux qa01 3.0.42-0.7-default #1 SMP >> Tue Oct 9 11:58:45 UTC 2012 (a8dc443) x86_64 x86_64 x86_64 GNU/Linux >> Byte sex: little-endian >> Libraries: >> Installation point: >> /afs/rzg/common/soft/hdf5/1.8.9/amd64_sles11/intel/12.1/serial >> >> Compiling Options: >> ------------------ >> Compilation Mode: production >> C Compiler: >> /afs/@cell/common/soft/intel/ics/2011.0.013/12.1/bin/intel64/bin/icc ( >> Intel(R) C Intel(R) 64 Compiler Version 12.1 Build 20120928) >> CFLAGS: -fPIC >> H5_CFLAGS: -std=c99 -O3 >> AM_CFLAGS: >> CPPFLAGS: >> H5_CPPFLAGS: -D_POSIX_C_SOURCE=199506L -DNDEBUG >> -UH5_DEBUG_API >> AM_CPPFLAGS: -D_LARGEFILE_SOURCE >> -D_LARGEFILE64_SOURCE -D_BSD_SOURCE >> Shared C Library: yes >> Static C Library: yes >> Statically Linked Executables: no >> LDFLAGS: >> H5_LDFLAGS: >> AM_LDFLAGS: >> Extra libraries: -lz -lrt -lm >> Archiver: ar >> Ranlib: ranlib >> Debugged Packages: >> API Tracing: no >> >> Languages: >> ---------- >> Fortran: yes >> Fortran Compiler: >> /afs/@cell/common/soft/intel/ics/2011.0.013/12.1/bin/intel64/bin/ifort ( >> Intel(R) Fortran Intel(R) 64 Compiler Version 12.1 Build 20120928) >> Fortran 2003 Compiler: yes >> Fortran Flags: -fPIC >> H5 Fortran Flags: -O3 >> AM Fortran Flags: >> Shared Fortran Library: yes >> Static Fortran Library: yes >> C++: yes >> C++ Compiler: >> /afs/@cell/common/soft/intel/ics/2011.0.013/12.1/bin/intel64/bin/icpc ( >> Intel(R) C++ Intel(R) 64 Compiler Version 12.1 Build 20120928) >> C++ Flags: >> H5 C++ Flags: >> AM C++ Flags: >> Shared C++ Library: yes >> Static C++ Library: yes >> >> Features: >> --------- >> Parallel HDF5: no >> High Level library: yes >> Threadsafety: no >> Default API Mapping: v18 >> With Deprecated Public Symbols: yes >> I/O filters (external): deflate(zlib) >> I/O filters (internal): shuffle,fletcher32,nbit,scaleoffset >> MPE: no >> Direct VFD: no >> dmalloc: no >> Clear file buffers before write: yes >> Using memory checker: no >> Function Stack Tracing: no >> GPFS: no >> Strict File Format Checks: no >> Optimization Instrumentation: no >> Large File Support (LFS): yes >> ----------------------------------------------- >> >> >> v1.8.11 >> *************************************** >> SUMMARY OF THE HDF5 CONFIGURATION >> ================================= >> >> General Information: >> ------------------- >> HDF5 Version: 1.8.11 >> Configured on: Wed May 8 16:20:56 CDT 2013 >> Configured by: hdftest@koala >> Configure mode: production >> Host system: x86_64-unknown-linux-gnu >> Uname information: Linux koala 2.6.18-348.1.1.el5 #1 SMP >> Tue Jan 22 16:19:19 EST 2013 x86_64 x86_64 x86_64 GNU/Linux >> Byte sex: little-endian >> Libraries: static, shared >> Installation point: >> /mnt/scr1/pre-release/hdf5/v1811/thg-builds/koala >> >> Compiling Options: >> ------------------ >> Compilation Mode: production >> C Compiler: /usr/bin/gcc ( gcc (GCC) 4.1.2 20080704 ) >> CFLAGS: >> H5_CFLAGS: -std=c99 -pedantic -Wall -Wextra >> -Wundef -Wshadow -Wpointer-arith -Wbad-function-cast -Wcast-qual >> -Wcast-align -Wwrite-strings -Wconversion -Wag >> gregate-return -Wstrict-prototypes -Wmissing-prototypes >> -Wmissing-declarations -Wredundant-decls -Wnested-externs -Winline >> -Wno-long-long -Wfloat-equal -Wmissing-format-attribut >> e -Wmissing-noreturn -Wpacked -Wdisabled-optimization -Wformat=2 >> -Wunreachable-code -Wendif-labels -Wdeclaration-after-statement >> -Wold-style-definition -Winvalid-pch -Wvariadic- >> macros -Wnonnull -Winit-self -Wmissing-include-dirs -Wswitch-default >> -Wswitch-enum -Wunused-macros -Wunsafe-loop-optimizations -Wc++-compat >> -Wvolatile-register-var -O3 -fomit-fr >> ame-pointer -finline-functions >> AM_CFLAGS: >> CPPFLAGS: >> H5_CPPFLAGS: -D_POSIX_C_SOURCE=199506L -DNDEBUG >> -UH5_DEBUG_API >> AM_CPPFLAGS: >> -I/mnt/hdf/packages/szip/shared/encoder/Linux2.6-x86_64-gcc/include >> -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_BSD_SOURCE >> Shared C Library: yes >> Static C Library: yes >> Statically Linked Executables: yes >> LDFLAGS: >> H5_LDFLAGS: >> AM_LDFLAGS: >> -L/mnt/hdf/packages/szip/shared/encoder/Linux2.6-x86_64-gcc/lib >> Extra libraries: -lsz -lz -lrt -ldl -lm >> Archiver: ar >> Ranlib: ranlib >> Debugged Packages: >> API Tracing: no >> Languages: >> ---------- >> Fortran: yes >> Fortran Compiler: /usr/bin/gfortran ( GNU Fortran (GCC) >> 4.1.2 20080704 ) >> Fortran 2003 Compiler: no >> Fortran Flags: >> H5 Fortran Flags: >> AM Fortran Flags: >> Shared Fortran Library: yes >> Static Fortran Library: yes >> >> C++: yes >> C++ Compiler: /usr/bin/g++ ( g++ (GCC) 4.1.2 20080704 ) >> C++ Flags: >> H5 C++ Flags: >> AM C++ Flags: >> Shared C++ Library: yes >> Static C++ Library: yes >> >> Features: >> --------- >> Parallel HDF5: no >> High Level library: yes >> Threadsafety: no >> Default API Mapping: v18 >> With Deprecated Public Symbols: yes >> I/O filters (external): deflate(zlib),szip(encoder) >> I/O filters (internal): shuffle,fletcher32,nbit,scaleoffset >> MPE: no >> Direct VFD: no >> dmalloc: no >> Clear file buffers before write: yes >> Using memory checker: no >> Function Stack Tracing: no >> GPFS: no >> Strict File Format Checks: no >> Optimization Instrumentation: no >> Large File Support (LFS): yes >> **************************************** >> >> I had another query. What value does H5P_DEFAULT contain and where is >> the value for this variable set. I was not able to find it from the >> v1.8.9 source, please let me know. I ask, since the code I am working >> with, sets the file access to H5P_DEFAULT if MPI is not being used, else >> its set to H5Pset_fapl_mpio. >> >> Please let me know if you need more information. >> >> Thanks >> >> Nitya >> >> ----------------------------- >> >> Message: 4 >> Date: Tue, 2 Jul 2013 14:33:07 -0500 >> From: Albert Cheng <[email protected]> >> To: HDF Users Discussion List <[email protected]> >> Subject: Re: [Hdf-forum] HDF5 - reading files larger than 2GB >> Message-ID: <[email protected]> >> Content-Type: text/plain; charset="iso-8859-1" >> >> Hi Nitya, >> >> Can you also give more information about the machine in which you ran your >> application >> that got the failures? E.g., >> >> 1. OS version ("uname -a" usually shows it.) >> 1.1. For Mac, try "sw_vers". >> >> 2. Did you build the HDF5 library from source? >> 2.1. if yes, please send a copy of the output of configure. >> 2.2. If the output of configure is not available, please send a copy of >> libhdf5.settings. >> You can usually find "libhdf5.settings" in the same directory where >> you find the >> HDF5 library, libhdf5.a or libhdf5.so. >> >> This will help us to try to repeat your problems in the appropriate platform. >> Thanks. >> >> -Albert Change >> THG Staff >> >> On Jul 2, 2013, at 11:20 AM, Dana Robinson <[email protected]> wrote: >> >>> Hi Nitya, >>> >>> I'll take a look into this. It's a holiday weekend here, so I probably >>> won't get to this until next week. >>> >>> Dana >>> >>> >>> On Mon, Jul 1, 2013 at 1:37 PM, Nitya Hariharan <[email protected]> >>> wrote: >>> Hi, >>> >>> I am having some trouble reading in a HDF5 file that is ~5GB. I was >>> initially having some problems even writing out a file this large and >>> looked at some postings in the HDF5 forum related to this. >>> >>> http://hdf-forum.184993.n3.nabble.com/Trouble-writing-2GB-dataset-from-single-task-with-HDF5-1-8-10-td4025821.html >>> >>> I tried using the serial version of the HDF5 library, v1.8.9 and was >>> able to write out a file of size ~5GB. However, when I try to read it >>> back in, I get the following error. >>> >>> ---------------- >>> #000: /home/hdftest/snapshots-bin-hdf5_1_8_11/current/src/H5Dio.c line >>> 182 in H5Dread(): can't read data >>> major: Dataset >>> minor: Read failed >>> #001: /home/hdftest/snapshots-bin-hdf5_1_8_11/current/src/H5Dio.c line >>> 550 in H5D__read(): can't read data >>> major: Dataset >>> minor: Read failed >>> #002: /home/hdftest/snapshots-bin-hdf5_1_8_11/current/src/H5Dcontig.c >>> line 543 in H5D__contig_read(): contiguous read failed >>> major: Dataset >>> minor: Read failed >>> #003: /home/hdftest/snapshots-bin-hdf5_1_8_11/current/src/H5Dselect.c >>> line 278 in H5D__select_read(): read error >>> major: Dataspace >>> minor: Read failed >>> #004: /home/hdftest/snapshots-bin-hdf5_1_8_11/current/src/H5Dselect.c >>> line 213 in H5D__select_io(): read error >>> major: Dataspace >>> minor: Read failed >>> #005: /home/hdftest/snapshots-bin-hdf5_1_8_11/current/src/H5Dcontig.c >>> line 873 in H5D__contig_readvv(): can't perform vectorized sieve buffer read >>> major: Dataset >>> minor: Can't operate on object >>> #006: /home/hdftest/snapshots-bin-hdf5_1_8_11/current/src/H5V.c line >>> 1457 in H5V_opvv(): can't perform operation >>> major: Internal error (too specific to document in detail) >>> minor: Can't operate on object >>> #007: /home/hdftest/snapshots-bin-hdf5_1_8_11/current/src/H5Dcontig.c >>> line 674 in H5D__contig_readvv_sieve_cb(): block read failed >>> major: Dataset >>> minor: Read failed >>> #008: /home/hdftest/snapshots-bin-hdf5_1_8_11/current/src/H5Fio.c line >>> 113 in H5F_block_read(): read through metadata accumulator failed >>> major: Low-level I/O >>> minor: Read failed >>> #009: /home/hdftest/snapshots-bin-hdf5_1_8_11/current/src/H5Faccum.c >>> line 258 in H5F_accum_read(): driver read request failed >>> major: Low-level I/O >>> minor: Read failed >>> #010: /home/hdftest/snapshots-bin-hdf5_1_8_11/current/src/H5FDint.c >>> line 142 in H5FD_read(): driver read request failed >>> major: Virtual File Layer >>> minor: Read failed >>> #011: /home/hdftest/snapshots-bin-hdf5_1_8_11/current/src/H5FDsec2.c >>> line 725 in H5FD_sec2_read(): file read failed: time = Mon Jul 1 >>> 20:21:57 2013 >>> , filename = '/tmp/file.hdf5', file descriptor = 5, errno = 14, error >>> message = 'Bad address', buf = 0x2ae9619ea010, total read size = >>> 4677466176, bytes this sub-read = 4677466176, bytes actually read = >>> 18446744073709551615, offset = 744468544 >>> ----------------- >>> >>> I looked at the forum again and saw this posting, which mentioned that >>> there was a bug fix done for POSIX issues. >>> >>> http://mail.lists.hdfgroup.org/pipermail/hdf-forum_lists.hdfgroup.org/2012-December/006348.html >>> >>> I was using v1.8.9, but thought of using the latest 1.8.11 to rule out >>> any issues with the HDF5 version I was using. However, I still get the >>> above error. >>> >>> Please could someone provide some feedback on why this is happening. If >>> I am able to write out such a large file, I should be able to read it in >>> as well? Ofcourse, small sized files work perfectly fine in my application. >>> >>> Since the library I am using, that in turn calls the HDF5 routines, >>> needs the interface of v1.6 of the HDF5 library, I use the flag >>> H5_USE_16_API while compiling. Would this in anyway cause the problem. >>> >>> Thanks in advance. >>> >>> -- >>> Regards >>> >>> Nitya >>> >>> >>> _______________________________________________ >>> Hdf-forum is for HDF software users discussion. >>> [email protected] >>> http://mail.lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org >>> >>> _______________________________________________ >>> Hdf-forum is for HDF software users discussion. >>> [email protected] >>> http://mail.lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org >> >> >> >> >> >> >> _______________________________________________ >> Hdf-forum is for HDF software users discussion. >> [email protected] >> http://mail.lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org > > _______________________________________________ > Hdf-forum is for HDF software users discussion. > [email protected] > http://mail.lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org -- Regards Nitya _______________________________________________ Hdf-forum is for HDF software users discussion. [email protected] http://mail.lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
