Hi there,
I have a large HDF5file (~19 Gig size) with which I apparently having
some data retrieval problems. It's hard to pinpoint exactly what the
problem is, but it seems to be with the value in a column which is
indexed. When I run the script and explicitly search with the record
ID (qcid) 1035139, there are data retrieved and all is well. As soon
as I run this for 300,000 such IDs, the following error occurs,
sometimes at different places. If it was caused by a corrupt index or
something similar, I would have expected the error to be reproducible
specifically with that ID, but this is not the case.
I am currently using the open source version, but we have recently
applied for a site-license for the Pro version: I am mentioning this
in case this error has something to do with the improved indexing that
comes with the 'Pro' version.
Any help / ideas / tips would be greatly appreciated: The particular
part of code causing the problem is here, followed by the error;
qdata = [x['qdat'] for x in table.where('(qcid == %r)' %key)]
Exception tables.exceptions.HDF5ExtError: HDF5ExtError('Problems
reading the array data.',) in
'tables.indexesExtension.IndexArray._g_readSortedSlice' ignored
Traceback (most recent call last):
File "pytable_trial_READdata.py", line 404, in <module>
runner.main()
File "pytable_trial_READdata.py", line 142, in main
clusterassessed = self.getOphs(clusterassessed,table)
File "pytable_trial_READdata.py", line 233, in mapOrphans
qdata = [x['qdat'] for x in table.where('(qcid == %r)' %key)]
File "tableExtension.pyx", line 862, in tables.tableExtension.Row.__next__
File "tableExtension.pyx", line 946, in
tables.tableExtension.Row.__next__indexed
File "lrucacheExtension.pyx", line 332, in
tables.lrucacheExtension.ObjectCache.removeslot_
KeyError: ((('qcid', ('eq',), ('1035139',)),), 'e0', (0L, 140242666L, 1L))
Closing remaining open files: isphfile...HDF5-DIAG: Error detected in
HDF5 library version: 1.6.7-rc2 thread 0. Back trace follows.
#000: \hdf\hdf5-16\vnet\release-testing\std\src\H5D.c line 1224 in
H5Dclose(): not a dataset
major(01): Function arguments
minor(03): Inappropriate type
#001: \hdf\hdf5-16\vnet\release-testing\std\src\H5I.c line 1258 in
H5I_find_id(): invalid ID
major(07): Atom layer
minor(32): Unable to find atom information (already closed?)
HDF5-DIAG: Error detected in HDF5 library version: 1.6.7-rc2 thread 0.
Back trace follows.
#000: \hdf\hdf5-16\vnet\release-testing\std\src\H5D.c line 1224 in
H5Dclose(): not a dataset
major(01): Function arguments
minor(03): Inappropriate type
#001: \hdf\hdf5-16\vnet\release-testing\std\src\H5I.c line 1258 in
H5I_find_id(): invalid ID
major(07): Atom layer
minor(32): Unable to find atom information (already closed?)
thanks a lot
Toby
------------------------------------------------------------------------------
This SF.net email is sponsored by:
SourcForge Community
SourceForge wants to tell your story.
http://p.sf.net/sfu/sf-spreadtheword
_______________________________________________
Pytables-users mailing list
Pytables-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/pytables-users
