Hi Toby,
This seems a bug in the optimized LRU cache of Pro. It would be great
if I can have access to your file so that I can reproduce the problem
by myself. It is a bit largish, but I think I can download it in a few
hours. Can you put it in accessible place to me? In case you can't do
that, it would be helpful if you can come up with a smaller file that
can be shared and a small script that is able to reproduce the problem.
Thanks,
Francesc
A Monday 19 January 2009, Toby Mathieson escrigué:
> Hi there,
>
> I have a large HDF5file (~19 Gig size) with which I apparently having
> some data retrieval problems. It's hard to pinpoint exactly what
> the problem is, but it seems to be with the value in a column which
> is indexed. When I run the script and explicitly search with the
> record ID (qcid) 1035139, there are data retrieved and all is well.
> As soon as I run this for 300,000 such IDs, the following error
> occurs, sometimes at different places. If it was caused by a corrupt
> index or something similar, I would have expected the error to be
> reproducible specifically with that ID, but this is not the case.
>
> I am currently using the open source version, but we have recently
> applied for a site-license for the Pro version: I am mentioning this
> in case this error has something to do with the improved indexing
> that comes with the 'Pro' version.
>
> Any help / ideas / tips would be greatly appreciated: The particular
> part of code causing the problem is here, followed by the error;
>
> qdata = [x['qdat'] for x in table.where('(qcid == %r)' %key)]
>
> Exception tables.exceptions.HDF5ExtError: HDF5ExtError('Problems
> reading the array data.',) in
> 'tables.indexesExtension.IndexArray._g_readSortedSlice' ignored
> Traceback (most recent call last):
> File "pytable_trial_READdata.py", line 404, in <module>
> runner.main()
> File "pytable_trial_READdata.py", line 142, in main
> clusterassessed = self.getOphs(clusterassessed,table)
> File "pytable_trial_READdata.py", line 233, in mapOrphans
>
> qdata = [x['qdat'] for x in table.where('(qcid == %r)' %key)]
> File "tableExtension.pyx", line 862, in
> tables.tableExtension.Row.__next__ File "tableExtension.pyx", line
> 946, in
> tables.tableExtension.Row.__next__indexed
> File "lrucacheExtension.pyx", line 332, in
> tables.lrucacheExtension.ObjectCache.removeslot_
> KeyError: ((('qcid', ('eq',), ('1035139',)),), 'e0', (0L, 140242666L,
> 1L)) Closing remaining open files: isphfile...HDF5-DIAG: Error
> detected in HDF5 library version: 1.6.7-rc2 thread 0. Back trace
> follows. #000: \hdf\hdf5-16\vnet\release-testing\std\src\H5D.c line
> 1224 in H5Dclose(): not a dataset
> major(01): Function arguments
> minor(03): Inappropriate type
> #001: \hdf\hdf5-16\vnet\release-testing\std\src\H5I.c line 1258 in
> H5I_find_id(): invalid ID
> major(07): Atom layer
> minor(32): Unable to find atom information (already closed?)
> HDF5-DIAG: Error detected in HDF5 library version: 1.6.7-rc2 thread
> 0. Back trace follows.
> #000: \hdf\hdf5-16\vnet\release-testing\std\src\H5D.c line 1224 in
> H5Dclose(): not a dataset
> major(01): Function arguments
> minor(03): Inappropriate type
> #001: \hdf\hdf5-16\vnet\release-testing\std\src\H5I.c line 1258 in
> H5I_find_id(): invalid ID
> major(07): Atom layer
> minor(32): Unable to find atom information (already closed?)
>
> thanks a lot
> Toby
>
> ---------------------------------------------------------------------
>--------- This SF.net email is sponsored by:
> SourcForge Community
> SourceForge wants to tell your story.
> http://p.sf.net/sfu/sf-spreadtheword
> _______________________________________________
> Pytables-users mailing list
> Pytables-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/pytables-users
--
Francesc Alted
------------------------------------------------------------------------------
This SF.net email is sponsored by:
SourcForge Community
SourceForge wants to tell your story.
http://p.sf.net/sfu/sf-spreadtheword
_______________________________________________
Pytables-users mailing list
Pytables-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/pytables-users
