Re: [Hdf-forum] Collective IO and filters

2017-11-08 Thread Dana Robinson
The public H5allocate/resize/free_memory() API calls use the library's memory allocator to manage memory, if that is what you are looking for. https://support.hdfgroup.org/HDF5/doc/RM/RM_H5.html Dana Robinson Software Developer The HDF Group From: Hdf-forum on behalf of Jordan Hend

Re: [Hdf-forum] Collective IO and filters

2017-11-08 Thread Dana Robinson
t as of the current implementation, but in terms of the interface contract) to use H5free_memory() on the buffer passed into an H5Z plugin, replacing it with a new (post-compression) buffer allocated via H5allocate()? On Wed, Nov 8, 2017 at 1:23 PM, Dana Robinson wrote:

Re: [Hdf-forum] Collective IO and filters

2017-11-08 Thread Dana Robinson
, 2017 at 13:46 To: Dana Robinson , "m.k.edwa...@gmail.com" , HDF List Subject: Re: [Hdf-forum] Collective IO and filters Dana, would it then make sense for all outside filters to use these routines? Due to Parallel Compression's internal nature, it uses buffers allocated via H5M

Re: [Hdf-forum] Collective IO and filters

2017-11-08 Thread Dana Robinson
filters on Windows. It's interesting that those functions will now be needed for parallel compression. Dana From: Hdf-forum on behalf of Dana Robinson Reply-To: HDF List Date: Wednesday, November 8, 2017 at 13:52 To: Jordan Henderson , "m.k.edwa...@gmail.com" , HDF List Subject:

Re: [Hdf-forum] Collective IO and filters

2017-11-09 Thread Dana Robinson
In develop, H5MM_malloc() and H5MM_calloc() will throw an assert if size is zero. That should not be there and the function docs even say that we return NULL on size zero. The bad line is at lines 271 and 360 in H5MM.c if you want to try yanking that out and rebuilding. Dana On 11/9/17, 09:06

Re: [Hdf-forum] [netcdfgroup] Make the Cmake Windows build staticplease !

2013-06-10 Thread Dana Robinson
Hi Roel, Note that it says "...to link statically to the CRT _in a DLL_ unless...". > Yes, linking the CRT statically into a DLL that is meant for redistribution > is a bad idea. HOWEVER, that doesn't mean that static linking per se is > bad. Linking a whole program statically (including the CRT

Re: [Hdf-forum] [netcdfgroup] Make the Cmake Windows build staticplease !

2013-06-10 Thread Dana Robinson
Hi Roel, On Mon, Jun 10, 2013 at 9:30 AM, Roel Vanhout wrote: > ** > Hi, > > > This is not true. Even when each static library is built with the same > > compiler at the same time and in the same Visual Studio solution. The > reason > > for this is that each static library is built as a separ

Re: [Hdf-forum] [netcdfgroup] Make the Cmake Windows build staticplease !

2013-06-10 Thread Dana Robinson
Hi all, I stand corrected. I just created a test project and inspected the assembler for the output and it turns out that the CRT does share state when statically linked together. For example, this is what I get when I create a static library that calls free via a library function: foo_free:

Re: [Hdf-forum] HDF5 - reading files larger than 2GB

2013-07-02 Thread Dana Robinson
Hi Nitya, I'll take a look into this. It's a holiday weekend here, so I probably won't get to this until next week. Dana On Mon, Jul 1, 2013 at 1:37 PM, Nitya Hariharan wrote: > Hi, > > I am having some trouble reading in a HDF5 file that is ~5GB. I was > initially having some problems even w

Re: [Hdf-forum] Repeated H5Dwrite calls increase memory usage

2013-07-22 Thread Dana Robinson
Hi Martijn, Which compiler are you using? On Sat, Jul 20, 2013 at 1:22 AM, Martijn Jasperse wrote: > Hi all, > I'm writing data-logging software storing in an HDF5 file, to be run for > months at a time. I'm intending to use a chunked dataset, and append a row > every time a measurement is mad

Re: [Hdf-forum] Repeated H5Dwrite calls increase memory usage

2013-07-22 Thread Dana Robinson
Hi Martijn, I'm not so sure that your problem is in HDF5. I ran your program on both 32-bit Linux and 64-bit Windows 7 w/ VS 2010 (I don't have a 32-bit Win7 VM) and I'm not seeing a memory leak (via Valgrind on Linux and _CrtDumpMemoryLeaks() on Windows). In all cases, I used HDF5 1.8.11. As f

Re: [Hdf-forum] HDF5 Compatibility with NFS and Fault Tolerance

2013-08-27 Thread Dana Robinson
the larger concern I have is with file consistency with a network interruption or failure. [Dana Robinson] At this time, the HDF5 library does not support concurrent access when a writer is involved. In the future HDF5 1.10.0 release (release date: TBA), we plan to include a feature that will all

Re: [Hdf-forum] HDF5 1.8.12 release candidate (pre1) is available for testing

2013-10-17 Thread Dana Robinson
I second Sean's opinion in this matter. We should truly address the issue and not paper over it by ignoring flags designed to suss it out. Dana > -Original Message- > From: Hdf-forum [mailto:hdf-forum-boun...@lists.hdfgroup.org] On Behalf > Of Sean McBride > Sent: Thursday, October 17,

Re: [Hdf-forum] H5SL_insert_common(): can't insert duplicate key

2014-01-06 Thread Dana Robinson
Hi MIke, Did you build the thread-safe version of the library? (i.e., configure with --enable-threadsafe) Dana From: Hdf-forum [mailto:hdf-forum-boun...@lists.hdfgroup.org] On Behalf Of Cook, Michael J (GE Healthcare) Sent: Friday, January 03, 2014 5:03 PM To: hdf-forum@lists.hdfgroup.org Sub

Re: [Hdf-forum] Extended-length windows paths & H5Fcreate

2014-01-06 Thread Dana Robinson
Hi Teo, How long is your path? Dana From: Hdf-forum [mailto:hdf-forum-boun...@lists.hdfgroup.org] On Behalf Of Teo Shaw Sent: Monday, January 06, 2014 10:14 AM To: hdf-forum@lists.hdfgroup.org Subject: [Hdf-forum] Extended-length windows paths & H5Fcreate Hi, is there any support for the "\\?\

Re: [Hdf-forum] Extended-length windows paths & H5Fcreate

2014-01-06 Thread Dana Robinson
n Mon, Jan 6, 2014 at 3:57 PM, Dana Robinson mailto:derob...@hdfgroup.org>> wrote: Hi Teo, How long is your path? Dana From: Hdf-forum [mailto:hdf-forum-boun...@lists.hdfgroup.org<mailto:hdf-forum-boun...@lists.hdfgroup.org>] On Behalf Of Teo Shaw Sent: Monday, January 06, 2014

Re: [Hdf-forum] Extended-length windows paths & H5Fcreate

2014-01-28 Thread Dana Robinson
UNC paths (\\server\share ) do, in fact, work with HDF5. I gave it a quick test this morning and server\\share works fine. Dana From: Hdf-forum [mailto:hdf-forum-boun...@lists.hdfgroup.org] On Behalf Of Dana Robinson Sent: Monday, January 06, 2014 12:42 PM To: HDF Users Discussion List

Re: [Hdf-forum] thread safety binaries?

2014-02-03 Thread Dana Robinson
issues. 1) Are winx86/x64 binaries available that have been built with the H5_HAVE_THREADSAFE compile flags? [Dana Robinson] We do not supply thread-safe binaries. You'll have to build them yourself, which is not that difficult to do with CMake + Visual Studio. 2) Any gotchas/tip

Re: [Hdf-forum] Hdf5 view crashes without error message when opening a (corrupt) file

2014-02-05 Thread Dana Robinson
Hi Christoph, Did the software crash while writing the file? It looks like the file is missing an object header. Since we use a metadata cache, this can happen if a program crashes while the cache contains unflushed metadata. HDF5 is not (currently) journaled, so we don't order or group metada

Re: [Hdf-forum] RFC: Freeing Memory Allocated by the HDF Library

2014-03-25 Thread Dana Robinson
w to deal with such situations. Werner On 24.03.2014 19:34, Dana Robinson wrote: Hi all, This is a brief RFC that outlines a new HDF5 API function, H5free_memory(), which can be used to free memory returned from the HDF5 library (e.g., via functions like H5Tget_member_name()). This wi

Re: [Hdf-forum] RFC: Thread-Local Storage Cleanup in Thread-Safe HDF5 for Windows

2014-03-25 Thread Dana Robinson
anup mechanism via a small peace of C++ code where a global object's constructor/destructor does the same as what DllMain would do in a dynamic library. Werner On 24.03.2014 19:41, Dana Robinson wrote: Hi all, This is another (very) brief RFC that outlines how we will address s

Re: [Hdf-forum] [Re] RFC: Freeing Memory Allocated by the HDF Library

2014-03-25 Thread Dana Robinson
/FreePascal) it's next to > impossible to properly free memory allocated inside the library. The > document should probably mention this also. [Dana Robinson] I do mention this in section 4.3. Dana ___ Hdf-forum is for HDF software

Re: [Hdf-forum] [Re] RFC: Freeing Memory Allocated by the HDF Library

2014-03-25 Thread Dana Robinson
The point is to visually distinguish the function from free(). After all, if the user uses H5free_memory() to free a buffer they allocated, then they have the original problem, only reversed :) The longer function name indicates that it is special and has a particular use. Dana > -Origina

Re: [Hdf-forum] RFC: Thread-Local Storage Cleanup in Thread-Safe HDF5 for Windows

2014-03-25 Thread Dana Robinson
even without the dllmain. I also had the same thought but, frankly, I then thought that it would be easier to hack the thing myself once you deliver it... Cheers -- ds 2014-03-25 15:06 GMT+01:00 Dana Robinson mailto:derob...@hdfgroup.org>>: This is a C library issue, so there are no construct

Re: [Hdf-forum] Access violation after deleting link and recreating dataset

2014-04-21 Thread Dana Robinson
multi-threading issues in my client app. A separate thread(s) was accessing the file through a different fileid. I've reworked my thread synchronization and am preventing multiple instances from getting created and all is looking good. [Dana Robinson] Are you using the thread-safe versi

Re: [Hdf-forum] Direct access to compressed block of data?

2014-04-29 Thread Dana Robinson
Are you looking to do something like this? http://www.hdfgroup.org/HDF5/doc/Advanced/DirectChunkWrite/UsingDirectChunkWrite.pdf Dana From: Hdf-forum [mailto:hdf-forum-boun...@lists.hdfgroup.org] On Behalf Of Jebb Stewart - NOAA Affiliate Sent: Tuesday, April 29, 2014 12:45 PM To: daniel.k...@ss

Re: [Hdf-forum] Direct access to compressed block of data?

2014-04-29 Thread Dana Robinson
Stewart NOAA's Earth System Research Laboratory (ESRL) Global Systems Division 325 Broadway, R/GSD6, Boulder, CO 80305 303-497-6724 On Tue, Apr 29, 2014 at 11:14 AM, Dana Robinson mailto:derob...@hdfgroup.org>> wrote: Are you looking to do something like this? http://www.hdfgroup.org/HDF5/d

Re: [Hdf-forum] CMake error when enabling thread-safe DLLs with C++ in HDF5 1.8.13

2014-05-15 Thread Dana Robinson
Hi Frederik, The thread-safe library w/ the C++ wrappers is not a supported configuration. This is enforced when built with the autotools, where --enable-unsupported must be used if you want this combination. We've been slowly bringing CMake into line with the more official autotools build opti

Re: [Hdf-forum] CMake error when enabling thread-safe DLLs with C++ in HDF5 1.8.13

2014-05-15 Thread Dana Robinson
Hi Frederik, > > The biggest potential problem is that the thread-safe locking > > mechanism that we use does not extend into the C++ wrappers. > > Addressing this is a fairly high-priority task for us. Pending funding, > we'd like to get it into the library as soon as possible. > > This sounds a

Re: [Hdf-forum] Asynchronous I/O

2014-05-27 Thread Dana Robinson
Hi, The HDF5 library does not support asynchronous I/O at this time. We are looking into including async I/O support in a future release, however. If you'd like to hurry this work along with financial support, there are people here you should talk to :) Dana From: Hdf-forum [mailto:hdf-forum-

Re: [Hdf-forum] HDF5 1.8.12 with OS X 10.10 Yosemite

2014-10-22 Thread Dana Robinson
It builds and passes tests using the autotools on Yosemite. I seriously doubt that CMake will make much of a difference. My Mac does not have Fortran, so I did not test that. Dana > -Original Message- > From: Hdf-forum [mailto:hdf-forum-boun...@lists.hdfgroup.org] On Behalf Of > Michael

Re: [Hdf-forum] Best structure for an Hdf5 file to store historical data and how to access quickly from python

2014-10-27 Thread Dana Robinson
This is not necessarily true. Many people store this type of data in PyTables, which is the backing store for Pandas and uses HDF5 internally. I'm travelling today, but I'll provide a real response to this in a day or two. Dana > -Original Message- > From: Hdf-forum [mailto:hdf-forum-bo

Re: [Hdf-forum] Thread-safe hdf5.dll leaks memory

2014-11-28 Thread Dana Robinson
Hi Andrey, There have been issues in the past with thread-local storage on Windows, but I thought I had fixed those when I moved TLS setup and teardown to a DllMain function in the library. I'll create a JIRA issue for this and take another look during this release cycle. One thing I did not d

Re: [Hdf-forum] Thread-safe hdf5.dll leaks memory

2014-11-28 Thread Dana Robinson
For future reference, the JIRA ticket is HDFFV-9033. > -Original Message- > From: Hdf-forum [mailto:hdf-forum-boun...@lists.hdfgroup.org] On Behalf Of > Dana Robinson > Sent: Friday, November 28, 2014 8:17 PM > To: HDF Users Discussion List > Subject: Re: [Hdf-forum] Th

Re: [Hdf-forum] Thread-safe hdf5.dll leaks memory

2014-12-01 Thread Dana Robinson
> Subject: Re: [Hdf-forum] Thread-safe hdf5.dll leaks memory > > 29.11.2014 4:16, Dana Robinson пишет: > > I'll create a JIRA issue for this and take another look during this > release cycle. One thing I did not do was test repeated load/unload from a > managed environmen

[Hdf-forum] Does anyone use the --enable-default-vfd option in configure?

2015-01-26 Thread Dana Robinson
someone has a compelling use case. Thanks, Dana Robinson Software Engineer The HDF Group ___ Hdf-forum is for HDF software users discussion. Hdf-forum@lists.hdfgroup.org http://mail.lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org Twitter: https://twitter.com/hdf5

Re: [Hdf-forum] Does anyone use the --enable-default-vfd option in configure?

2015-01-26 Thread Dana Robinson
Sorry, that should be --with-default-vfd= Dana > -Original Message- > From: Hdf-forum [mailto:hdf-forum-boun...@lists.hdfgroup.org] On Behalf Of > Dana Robinson > Sent: Monday, January 26, 2015 6:33 AM > To: HDF Users Discussion List (hdf-forum@lists.hdfgroup.org) > S

Re: [Hdf-forum] Multithreading and CPP

2015-02-04 Thread Dana Robinson
Hi Roland, The reason we don't support the higher-level "wrapper" libraries like C++, Fortran, and the high-level (HL) library is that the global lock is at the C API call level, not the wrapper level. Some of these wrapper API calls make multiple C API calls and a context switch during the wra

Re: [Hdf-forum] Multithreading and CPP

2015-02-11 Thread Dana Robinson
Hi Roland, Yes. Werner's clarification is correct. Dana Robinson Software Engineer The HDF Group > -Original Message- > From: Hdf-forum [mailto:hdf-forum-boun...@lists.hdfgroup.org] On Behalf Of > Roland Schwarz > Sent: Thursday, February 12, 2015 2:01 A

Re: [Hdf-forum] SWMR issues

2015-06-24 Thread Dana Robinson
The source code issues are fixed. Dana Robinson Software Developer The HDF Group > -Original Message- > From: Hdf-forum [mailto:hdf-forum-boun...@lists.hdfgroup.org] On Behalf Of > Baker, William L Jr CIV USARMY MEDCOM AISR (US) > Sent: Wednesday, June 24, 2015 12:00 PM >

Re: [Hdf-forum] SWMR and Win32

2015-06-26 Thread Dana Robinson
Hi, SWMR has not been tested on Windows. The existing SWMR test program is a shell script that has not been converted to a scheme usable on Windows. We've also not tested it on Cygwin or MinGW at this time. I'd have to look more carefully at the code for the atomic test, but I would not be sur

Re: [Hdf-forum] Best Version for OSX 10.9+

2015-07-08 Thread Dana Robinson
Hi, Can you provide sample code that isolates the problem? I'd like to file a bug report if there is an issue. Dana Robinson Software Engineer The HDF Group From: Hdf-forum mailto:hdf-forum-boun...@lists.hdfgroup.org>> on behalf of Bo Zhou mailto:bo.schwarzst...@gmail.com>&

Re: [Hdf-forum] Compiling HDF5 with the multithread, static version of the run-time library

2015-07-20 Thread Dana Robinson
e CRT, we don't support it. We disabled it due to potential problems with memory being allocated in one statically linked CRT and freed in another (e.g.: in plugins, user virtual file drivers, etc). Dana Robinson Software Engineer The HDF Group > -Original Message- > From: Hdf-f

Re: [Hdf-forum] Compiling HDF5 with the multithread, static version of the run-time library

2015-07-20 Thread Dana Robinson
icated functionality such as plugins and VFD. Can this still cause problems? If yes, then this is a huge problem for me, because I need HDF5 to be compatible with my other libraries. Please suggest a solution. All the best, Samer On 7/20/2015 5:44 PM, Dana Robinson wrote: Hi Samer, A fully st

Re: [Hdf-forum] C++ versus parallel

2015-10-16 Thread Dana Robinson
nto the higher-level wrappers, but there's no immediate plans to do this. A sponsor with funding for such a feature would obviously change the situation :) Cheers, Dana Robinson Software Engineer The HDF Group -Original Message- From: Hdf-forum [mailto:hdf-forum-boun...@lists.hdfgroup.o

Re: [Hdf-forum] C++ versus parallel

2015-10-16 Thread Dana Robinson
Whoops! Sorry! I read that as thread-safe, not parallel. It's early here and I haven't made much progress on my coffee. The lack of parallel and C++ being official supported is, I believe, due to a lack of testing and not due to a particular bug or problem. Dana Robinson Software En

Re: [Hdf-forum] C++ versus parallel

2015-10-16 Thread Dana Robinson
th parallel hdf : I would like to test if this may fix this problem. Franck Le 2015-10-16 12:47, Dana Robinson a écrit : > Whoops! Sorry! I read that as thread-safe, not parallel. It's early > here and I haven't made much progress on my coffee. > > The lack of parallel a

Re: [Hdf-forum] Building Static Thread Safe Libs

2015-11-04 Thread Dana Robinson
hreads are used. With no dll, there are no hooks. Cheers, Dana Robinson Software Engineer The HDF Group From: Hdf-forum [mailto:hdf-forum-boun...@lists.hdfgroup.org] On Behalf Of David Sent: Wednesday, November 4, 2015 3:49 PM To: HDF Users Discussion List Subject: [Hdf-forum] Building Static Threa

Re: [Hdf-forum] Building Static Thread Safe Libs

2015-11-04 Thread Dana Robinson
uild procedure to do it. On Wed, Nov 4, 2015 at 12:54 PM, Dana Robinson mailto:derob...@hdfgroup.org>> wrote: Hi David, Since you say "dll" I'm going to assume you are on Windows. Thread-safety is not supported in static builds of HDF5 on Windows. The reason for this is that,

Re: [Hdf-forum] Multiply Opened Files

2015-11-16 Thread Dana Robinson
ing a Win32 equivalent. Dana Robinson Software Engineer The HDF Group From: Hdf-forum [mailto:hdf-forum-boun...@lists.hdfgroup.org] On Behalf Of David Sent: Monday, November 16, 2015 12:46 PM To: HDF Users Discussion List Subject: [Hdf-forum] Multiply Opened Files I found this in the reference m

Re: [Hdf-forum] Bug report & file locking for Windows

2015-12-10 Thread Dana Robinson
DF5 1.10.0. Thanks for your report! I'll post back on the forum with any changes we'll be making but I probably won't get to it until the new year due to the holidays and 1.10.0 alpha release coming up. Cheers, Dana Robinson Software Developer The HDF Group From: Hdf-for

Re: [Hdf-forum] Working with lots of HDF5 files

2016-01-26 Thread Dana Robinson
1.8 compatibility, though, so keep that in mind. https://www.hdfgroup.org/HDF5/doc/RM/RM_H5P.html#Property-SetLibverBounds Dana Robinson Software Engineer The HDF Group From: Hdf-forum [mailto:hdf-forum-boun...@lists.hdfgroup.org] On Behalf Of SOLTYS Radoslaw Sent: Tuesday, January

Re: [Hdf-forum] H5Pset_fapl_mpio() and H5Pset_dxpl_mpio() not defined

2016-03-14 Thread Dana Robinson
I tried this a while back and I can get recent versions of HDF5 1.8 to build using the latest MS MPI, but mpiexec seems to only be available on Windows Server 2012 with the HPC pack installed. I haven't gotten around to setting up a VM for that so I don't know if it works there or

Re: [Hdf-forum] HDF5 SWMR Document design question

2016-03-15 Thread Dana Robinson
ce to a structure that has not been flushed from the cache yet, a reader will encounter problems when it tries to read the not-on-disk-only-in-memory structure from the disk. One of the key changes that SWMR makes under the hood is to order metadata flushes to avoid this situation. Dana Rob

Re: [Hdf-forum] SWMR requirement to be POSIX compliant

2016-03-19 Thread Dana Robinson
systems (NFS, SMB/Windows file shares, etc.), however, do not guarantee write ordering and are not supported under SWMR. Dana Robinson Software Engineer The HDF Group From: Hdf-forum [mailto:hdf-forum-boun...@lists.hdfgroup.org] On Behalf Of Ryland, Andrew Sent: Wednesday, March 16, 2016 11:47 AM To

Re: [Hdf-forum] Release of HDF5-1.10.0-pre1

2016-03-19 Thread Dana Robinson
s for building since the testing makes heavy use of shell scripts, fork, etc. which are not available on Windows. I'll see if I can #ifdef around the problem. Dana Robinson Software Engineer The HDF Group -Original Message- From: Hdf-forum [mailto:hdf-forum-boun...@lists.hdfgroup.org] On

Re: [Hdf-forum] Release of HDF5-1.10.0-pre2

2016-03-25 Thread Dana Robinson
Hi Rob, - I'm not sure what to make of the timezone struct. Can you send me the src/H5pubconf.h that is generated by configure and your configure output? I'd like to see what MinGW thinks you have. - Those problematic files in tests were designed for POSIX and assume the presence of fork(), pi

Re: [Hdf-forum] Release of HDF5-1.10.0-pre2

2016-03-25 Thread Dana Robinson
forum-boun...@lists.hdfgroup.org] On Behalf Of sisyph...@optusnet.com.au Sent: Friday, March 25, 2016 7:45 PM To: HDF Users Discussion List Subject: Re: [Hdf-forum] Release of HDF5-1.10.0-pre2 -Original Message----- From: Dana Robinson Sent: Saturday, March 26, 2016 12:34 AM To: HDF Users Discu

Re: [Hdf-forum] Release of HDF5-1.10.0-pre2

2016-03-27 Thread Dana Robinson
riginal Message- From: Hdf-forum [mailto:hdf-forum-boun...@lists.hdfgroup.org] On Behalf Of sisyph...@optusnet.com.au Sent: Saturday, March 26, 2016 2:41 AM To: HDF Users Discussion List Subject: Re: [Hdf-forum] Release of HDF5-1.10.0-pre2 -----Original Message- From: Dana Robinson Sent: Sat

Re: [Hdf-forum] Release of HDF5-1.10.0-pre2

2016-03-27 Thread Dana Robinson
hange into 1.10.0, though, so it'll have to wait another revision. Cheers, Dana -Original Message- From: Hdf-forum [mailto:hdf-forum-boun...@lists.hdfgroup.org] On Behalf Of Dana Robinson Sent: Sunday, March 27, 2016 4:36 AM To: HDF Users Discussion List Subject: Re: [Hdf-forum] Relea

Re: [Hdf-forum] HDF4 jpeg library installation

2016-04-10 Thread Dana Robinson
I see that you have "Documents" in your jpeg library path. Are you building on Windows via Cygwin or MinGW where the ~ expands into a path that contains spaces? Moving the jpeg library to a location where the path has no spaces might be something to try. Dana Robinson Software Engine

Re: [Hdf-forum] HDF5 release 1.10.0 giving "use_append_chunk" failures

2016-04-24 Thread Dana Robinson
Hi Carl, What file system are you testing on? Is it a network file system like NFS, AFS, or SMB? That test was added in HDF5 1.10.0 and tests single-writer/multiple-readers (SWMR) functionality. Since that is a new feature for 1.10.0, the test is not a part of the HDF5 1.8 release. Dana

Re: [Hdf-forum] Unresolved Symbols?

2016-04-26 Thread Dana Robinson
You defined H5_BUILD_AS_DYNAMIC, not BUILT. Dana Robinson Software Engineer The HDF Group -Original Message- From: Hdf-forum [mailto:hdf-forum-boun...@lists.hdfgroup.org] On Behalf Of jeff womble Sent: Tuesday, April 26, 2016 10:08 PM To: HDF Users Discussion List Subject: Re: [Hdf

Re: [Hdf-forum] many warnings during "make" : installation of hdf5 1.8.16

2016-05-05 Thread Dana Robinson
Hi Arindam, Those warnings are generally harmless and you see them because we have aggressive gcc warnings turned on by default. We are working to reduce them but it'll probably be a while before they are gone. Dana Robinson Software Engineer The HDF Group From: Hdf-forum [mailto:hdf-

Re: [Hdf-forum] HDF5-1.10.0 and flock()

2016-05-16 Thread Dana Robinson
-time mechanism for disabling file locking implemented for HDF5 1.10.1. Dana Robinson Software Engineer The HDF Group -Original Message- From: Hdf-forum [mailto:hdf-forum-boun...@lists.hdfgroup.org] On Behalf Of Elena Pourmal Sent: Sunday, May 15, 2016 9:01 PM To: HDF Users Discussion List Su

Re: [Hdf-forum] HDF5-1.10.0 and flock()

2016-05-17 Thread Dana Robinson
t file systems, or if the sysadmin changes their mind on a daily basis to enable or disable file locking? Werner On 16.05.2016 18:03, Dana Robinson wrote: > If a suitable way to lock files cannot be determined at configure time, a > no-op function is substituted. This is c

Re: [Hdf-forum] Cannot find source file: /path/to/hdf5/svn/src/H5Edefin.h

2016-05-27 Thread Dana Robinson
configuration. This file is already created in our release branches and tarballs, so you normally don't need Perl to build from there. Dana Robinson Software Engineer The HDF Group From: Hdf-forum [mailto:hdf-forum-boun...@lists.hdfgroup.org] On Behalf Of Nico Schlömer Sent: Friday, May 27, 2

Re: [Hdf-forum] h5fcreate 1.10 unable to lock

2016-06-02 Thread Dana Robinson
ASAP so hopefully you won't have to wait long for official functionality that addresses your problem. Dana Robinson Software Engineer The HDF Group -Original Message- From: Hdf-forum [mailto:hdf-forum-boun...@lists.hdfgroup.org] On Behalf Of Greg Werner Sent: Thurs

[Hdf-forum] HDF5 1.10.0-patch1 source patch to disable file locking

2016-06-20 Thread Dana Robinson
controlling the file locking behavior. This patch is not necessary for Win32 (non-Cygwin) Windows since a file locking scheme was never implemented on that platform. The patch can be applied by running 'patch -p0 < file-lock-removal.diff' in the HDF5 source code directory. Dana Robi

Re: [Hdf-forum] Turn off file locking

2016-06-28 Thread Dana Robinson
Hi Brendan, There is a source patch for HDF5 1.10 that disables file locking located here: <https://www.hdfgroup.org/HDF5/release/obtainsrc5110.html#patch>https://www.hdfgroup.org/HDF5/release/obtainsrc5110.html#patch Does that solve your problem? Dana Robinson Software Engineer The HDF

Re: [Hdf-forum] Cross-platform Unicode filename support?

2016-07-05 Thread Dana Robinson
ld have to be done on our sustained engineering budget, which gets lower priority than paid-for work. And we're working on making a public bug tracker available. Hopefully that will be available soon. Cheers, Dana Robinson Software Engineer The HDF Group From: Hdf-forum [mailto

Re: [Hdf-forum] Error detected in HDF5 (1.10.0) thread 0: /home/hdftest/snapshots-hdf5_1_10_alpha/current/src

2016-07-25 Thread Dana Robinson
problem should be available in HDF5 1.10.1, but in the meantime you can apply a source patch found here: https://www.hdfgroup.org/HDF5/release/obtainsrc5110.html#conf under "Patch to Disable File Locking" at the bottom of the page. Cheers, Dana Robinson Software Engineer The HDF G

Re: [Hdf-forum] Error detected in HDF5 (1.10.0) thread 0: /home/hdftest/snapshots-hdf5_1_10_alpha/current/src

2016-07-27 Thread Dana Robinson
Hi Haijun, If you are still getting a flock error, I'd be suspicious that you aren't linking to the patched HDF5 library. The source patch completely removes flock so you shouldn't be seeing file locking errors. Dana Robinson Software Engineer The HDF Group From: Hdf-forum [m

Re: [Hdf-forum] Error detected in HDF5 (1.10.0) thread 0: /home/hdftest/snapshots-hdf5_1_10_alpha/current/src

2016-07-29 Thread Dana Robinson
sets that up for you. Dana Robinson Software Engineer The HDF Group From: Hdf-forum [mailto:hdf-forum-boun...@lists.hdfgroup.org] On Behalf Of Prentice Bisbal Sent: Friday, July 29, 2016 9:43 AM To: hdf-forum@lists.hdfgroup.org Subject: Re: [Hdf-forum] Error detected in HDF5 (1.10.0) thread 0:

Re: [Hdf-forum] Error detected in HDF5 (1.10.0) thread 0: /home/hdftest/snapshots-hdf5_1_10_alpha/current/src

2016-08-01 Thread Dana Robinson
! Haijun Yu At 2016-07-29 23:40:52, "Dana Robinson" mailto:derob...@hdfgroup.org>> wrote: Hi all, The source patch removes all references to flock() so this error should not be occurring, regardless of the permissions. If you are still seeing errors, I suspect that you are linkin

Re: [Hdf-forum] HDF5 -- multithreading support or not?

2016-08-29 Thread Dana Robinson
F5 library." Dana Robinson Software Engineer The HDF Group From: Hdf-forum [mailto:hdf-forum-boun...@lists.hdfgroup.org] On Behalf Of Isaac Gerg Sent: Friday, August 26, 2016 6:31 PM To: hdf-forum@lists.hdfgroup.org Subject: [Hdf-forum] HDF5 -- multithreading support or not? Hi Folks, I

Re: [Hdf-forum] HDF5 -- multithreading support or not?

2016-08-29 Thread Dana Robinson
rary." To me, "generally" implies that h5close() is not guaranteed to be called in exit() OR that its preferred by hdf library that the user calls H5close() right before calling exit(). Thank you, Isaac On Mon, Aug 29, 2016 at 11:14 AM, Dana Robinson mailto:derob...@hdfgroup

Re: [Hdf-forum] HDF5 -- multithreading support or not?

2016-08-30 Thread Dana Robinson
: [Hdf-forum] HDF5 -- multithreading support or not? Interesting... I am using H5Fclose(). If 2 different threads call this function for the same file (but different file handles), does HDF handle this? On Mon, Aug 29, 2016 at 11:29 AM, Dana Robinson mailto:derob...@hdfgroup.org>> wro

Re: [Hdf-forum] Simply using the library from separate threads (C++ API)

2016-09-22 Thread Dana Robinson
Hi Elvis, Did you build your HDF5 library with thread-safety enabled (--enable-threadsafe w/ configure)? Dana Robinson Software Engineer The HDF Group Get Outlook for Android<https://aka.ms/ghei36> From: Elvis Stansvik Sent: Thursday, September 22, 12:43 Subject: [Hdf-forum] Simply usi

Re: [Hdf-forum] Simply using the library from separate threads (C++ API)

2016-09-22 Thread Dana Robinson
e --enable-unsupported. Thanks for the info! Elvis > > Scot > > >> On Sep 22, 2016, at 12:36 PM, Elvis Stansvik >> wrote: >> >> 2016-09-22 19:23 GMT+02:00 Elvis Stansvik : >>> 2016-09-22 19:17 GMT+02:00 Dana Robinson : >>>> Hi Elvis, >>

Re: [Hdf-forum] Simply using the library from separate threads (C++ API)

2016-09-22 Thread Dana Robinson
rch package with --enable-threadsafe >> --enable-unsupported. >> >> Thanks for the info! >> >> Elvis >> >>> Scot >>> >>> >>>> On Sep 22, 2016, at 12:36 PM, Elvis Stansvik >>>> wrote: >>>> >>>> 2016

Re: [Hdf-forum] thread safe work in windows for a hdf static?

2016-10-04 Thread Dana Robinson
actually quite minimal. Basically, you just have to initialize and clean up thread-local storage when you attach and detach threads, respectively. Cheers, Dana Robinson Software Engineer The HDF Group From: Hdf-forum [mailto:hdf-forum-boun...@lists.hdfgroup.org] On Behalf Of Werner Benger Sent

Re: [Hdf-forum] thread safe work in windows for a hdf static?

2016-10-04 Thread Dana Robinson
work in windows for a hdf static? Thank you all. Just to be clear, is the native shared lib built with threadsafe? On Tue, Oct 4, 2016 at 5:06 AM, Dana Robinson mailto:derob...@hdfgroup.org>> wrote: Hi all, Werner is correct. You could build the thread-safe + static library with ALLOW_

[Hdf-forum] RFC concerning the disabling of file locking in HDF5

2016-10-16 Thread Dana Robinson
airly easy to implement. Please let us know if you have any questions or concerns. Dana Robinson Software Engineer The HDF Group RFC - Disable File Locking in HDF5 1.10 - v3.docx Description: RFC - Disable File Locking in HDF5 1.10 - v3.docx ___ Hdf-

Re: [Hdf-forum] Irregular 2D array write (C++)

2016-10-17 Thread Dana Robinson
Keep in mind that if you go with a variable-length type, the data cannot be compressed so your file sizes will be larger. Dana Robinson Software Engineer The HDF Group From: Hdf-forum [mailto:hdf-forum-boun...@lists.hdfgroup.org] On Behalf Of Steven Walton Sent: Monday, October 17, 2016 12:35

Re: [Hdf-forum] strange error on Fclose()

2016-11-02 Thread Dana Robinson
Hi Dimos, What version of HDF5 are you using? Are you using H5Pset_libver_bounds() to set a particular version of the file format (e.g.: H5F_LIBVER_LATEST)? Can you reduce this to a small C program that demonstrates the problem? Thanks, Dana Robinson Software Engineer The HDF Group From: Hdf

Re: [Hdf-forum] Building hdf5-1.10.0-patch 1 on Blue Gene Q

2016-12-08 Thread Dana Robinson
me know if that doesn't work for you and we can diagnose further. Cheers, Dana Robinson Software Engineer The HDF Group From: Hdf-forum [mailto:hdf-forum-boun...@lists.hdfgroup.org] On Behalf Of Sjaardema, Gregory D Sent: Thursday, December 8, 2016 3:23 PM To: HDF Users Discussion List

Re: [Hdf-forum] [EXTERNAL] Re: Building hdf5-1.10.0-patch 1 on Blue Gene Q

2016-12-12 Thread Dana Robinson
in H5Dcreate2(): not a location ID / -- "A supercomputer is a device for turning compute-bound problems into I/O-bound problems” From: Hdf-forum mailto:hdf-forum-boun...@lists.hdfgroup.org>> on behalf of Dana Robinson mailto:derob...@hdfgroup.org>> Reply-To: HDF User

Re: [Hdf-forum] Setting plugin path programmatically

2017-03-03 Thread Dana Robinson
Hi Andrey, I'll create an issue for this. We have a Windows layer that substitutes Win32 equivalents for some POSIX commands and it should be straightforward to use SetEnvironmentVariable() on Windows. Dana Robinson Software Engineer The HDF Group -Original Message- From: Hdf-

Re: [Hdf-forum] Setting plugin path programmatically

2017-03-03 Thread Dana Robinson
Oops. Sorry, I read this too quickly. I'll have to think more about your problem. I will create an issue for this, though. Dana -Original Message- From: Hdf-forum [mailto:hdf-forum-boun...@lists.hdfgroup.org] On Behalf Of ?? ? Sent: Friday, March 3, 2017 3:43 AM To: HDF User

Re: [Hdf-forum] Setting plugin path programmatically

2017-03-03 Thread Dana Robinson
--Original Message- From: Hdf-forum [mailto:hdf-forum-boun...@lists.hdfgroup.org] On Behalf Of Dana Robinson Sent: 03 March 2017 15:58 To: HDF Users Discussion List Subject: Re: [Hdf-forum] Setting plugin path programmatically Hi Andrey, I'll create an issue for this. We have a Win

Re: [Hdf-forum] HDF5 files on NFS?

2017-04-03 Thread Dana Robinson
Hi all, Sorry I've missed most of this discussion, but HDF5 1.10.1 checks an environment variable that you can use to disable file locking (HDF5_USE_FILE_LOCKING). Setting this to "FALSE" disables all file lock calls. Cheers, Dana Robinson Software Engineer The HDF Group On 4

Re: [Hdf-forum] HDF5 files on NFS?

2017-04-03 Thread Dana Robinson
on, Apr 3, 2017 at 5:14 PM, Dave Allured - NOAA Affiliate mailto:dave.allu...@noaa.gov>> wrote: Dana, Is HDF5_USE_FILE_LOCKING applied at configure time, build time, or run time? --Dave On Mon, Apr 3, 2017 at 4:27 PM, Dana Robinson mailto:derob...@hdfgroup.org>> wrote: Hi all, Sorry

Re: [Hdf-forum] How to free memory after H5Fclose

2017-04-24 Thread Dana Robinson
ion that the OS typically discards those freed pages easily when other processes need the memory so you shouldn't be forced to go to the disk for swap space. Dana Robinson Software Engineer The HDF Group -Original Message- From: Hdf-forum [mailto:hdf-forum-boun...@lists.hdfgroup.org]

Re: [Hdf-forum] compiling for the first time from source code

2017-06-16 Thread Dana Robinson
Do you have MPI installed? Dana Robinson Software Developer The HDF Group From: Hdf-forum on behalf of Muhammad ILYAS Reply-To: HDF List Date: Friday, June 16, 2017 at 11:06 To: HDF List Subject: [Hdf-forum] compiling for the first time from source code Hello, This is my first attempt of

Re: [Hdf-forum] Avoiding corruption of the HDF5 File

2017-09-25 Thread Dana Robinson
Hi all, (I'm jumping in to the middle of this discussion while still on vacation, so please excuse me if I'm missing something.) Those documents describe cache flushing. Flush ordering should not affect the file format – any transient 'file corruption' would be due to a second reader inspectin