A Monday 25 October 2010 23:19:33 Marco Lui escrigué: > Hello Fransesc! > > > No. That's different. It is probably PyTables fault because it > > should check that other third-party library has already closed the > > file (i.e. PyTables should not be the only one accessing the > > file). > > > > However, I must warn you that opening the same file with two > > different libraries is a sure receipt for failure, mainly because > > PyTables's internal caches for some HDF5 hierarchical aspect of > > the file. > > Yes, I fully agree with you. However, the behavior I report occurs > without the user explicitly accessing a single with different > bindings. I discovered it when, in my application which uses > PyTables, I tried to use a module which did its own internal > bookkeeping with h5py. My code never asks the two modules to operate > on the same files, but somehow h5py's exitfunc does something that > operates on files opened by pytables. I am not familiar with the > underlying hdf5 library, but it looks like there is some provision > in the hdf5 library for iterating over all open file handles, and > h5py is using this to close them. The relevant code for h5py's exit > handler is this: > > # === Library init > ============================================================def > _exithack(): """ Internal function; do not call unless you want to > lose all your data. """ # If any identifiers have reference > counts > 1 when the library closes, # it freaks out and dumps a > message to stderr. So we have Python dec_ref # everything when > the interpreter's about to exit. cdef int count cdef int i > cdef hid_t *objs count = H5Fget_obj_count(H5F_OBJ_ALL, > H5F_OBJ_ALL) if count > 0: objs = > <hid_t*>malloc(sizeof(hid_t)*count) try: > H5Fget_obj_ids(H5F_OBJ_ALL, H5F_OBJ_ALL, count, objs) for i from > 0<=i<count: while > H5Iget_type(objs[i]) != H5I_BADID and H5Iget_ref(objs[i]) > 0: > H5Idec_ref(objs[i]) finally: free(objs) > _conv.unregister_converters() > > > Source: http://code.google.com/p/h5py/source/browse/trunk/h5py/h5.pyx > line 332 onwards. > > > I suspect that the actual fault here is of h5py, since they end up > closing files they do not open. However, this bad behavior ends up > biting unsuspecting PyTables users such as myself! Perhaps as you say > it may be possible to check for this condition, and issue the user a > warning about it? > > > I have reported the issue to h5py as well now: > > http://code.google.com/p/h5py/issues/detail?id=136
Thanks for the additional investigation. Yes, as you and Andrew say, this seems a problem with h5py, so I've closed the PyTables ticket. I hope that h5py could fix this more sooner than later. -- Francesc Alted ------------------------------------------------------------------------------ Nokia and AT&T present the 2010 Calling All Innovators-North America contest Create new apps & games for the Nokia N8 for consumers in U.S. and Canada $10 million total in prizes - $4M cash, 500 devices, nearly $6M in marketing Develop with Nokia Qt SDK, Web Runtime, or Java and Publish to Ovi Store http://p.sf.net/sfu/nokia-dev2dev _______________________________________________ Pytables-users mailing list Pytables-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/pytables-users
