I'm using pytables (1.3beta2) with mod_python (3.2.8) to manage a brain
imaging data repository.
Right now I break things a lot, and I managed to corrupt a file such that
pytables cannot read it. Matlab, however, can read it.
I am very interested in best-practices for maintaining integrity of my files.
I was using the undo feature for this, but suspect this was part of the
corruption...
More details:
1) I have undo enabled on this file
2) Matlab 7.1 and 7.2 use HDF5 v1.6.2 and v1.6.4 (so far as I can tell), and
both can read the file
3) PyTables error claims 'HDF5-DIAG: Error detected in HDF5 library version:
1.6.5 thread 3086534336.' (full error message below my sig)
4) The file was created with (non-pytables) matlab/hdf5 and later updated /
modified with pytables - mostly copying groups from other hdf5 files
recursively.
5) There were occassions (though I'm not sure about the one right before the
corruption) where mod_python would bomb out and didn't execute a close on the
file that was opened with mode 'a'.
So, the problem may indeed be some strange bug with HDF5 1.6.5. It also may
be that I was using undo with a not-fully-pytables compliant file. Or, it
may simply be that I bombed out of a session where pytables had a file open
(with undo enabled) and didn't properly close it.
Any ideas? I'm not that concerned about fixing this file, but mostly about
ensuring that it doesn't happen when the system goes into production...
corruption of the central repository would be terrible!
Cheers,
--
Dav Clark
@Stanford / Wandell Lab
917-544-8408
))<>((
HDF5-DIAG: Error detected in HDF5 library version: 1.6.5 thread 3086534336.
Back trace follows.
#000: H5Dio.c line 499 in H5Dread(): can't read data
major(15): Dataset interface
minor(24): Read failed
#001: H5Dio.c line 756 in H5D_read(): can't read data
major(15): Dataset interface
minor(24): Read failed
#002: H5Dio.c line 1636 in H5D_chunk_read(): optimized read failed
major(15): Dataset interface
minor(24): Read failed
#003: H5Dselect.c line 510 in H5D_select_read(): read error
major(14): Dataspace interface
minor(24): Read failed
#004: H5Distore.c line 2025 in H5D_istore_readvv(): unable to read raw data
chunk
major(05): Low-level I/O layer
minor(24): Read failed
#005: H5Distore.c line 1612 in H5D_istore_lock(): data pipeline read failed
major(19): Data filters layer
minor(24): Read failed
#006: H5Z.c line 997 in H5Z_pipeline(): filter returned failure during read
major(19): Data filters layer
minor(24): Read failed
#007: H5Zdeflate.c line 113 in H5Z_filter_deflate(): inflate() failed
major(19): Data filters layer
minor(29): Unable to initialize object
---------------------------------------------------------------------------
tables.exceptions.HDF5ExtError Traceback (most
recent call last)
/orange/scr1/mrFile_testing/roi_test/<ipython console>
/usr/lib/python2.4/site-packages/tables/File.py in openFile(filename, mode,
title, trMap, rootUEP, filters, nodeCacheSize)
223 # Finally, create the File instance, and return it
224 return File(path, mode, title, trMap, rootUEP, filters,
--> 225 METADATA_CACHE_SIZE, nodeCacheSize)
226
227
/usr/lib/python2.4/site-packages/tables/File.py in __init__(self, filename,
mode, title, trMap, rootUEP, filters, metadataCacheSize, nodeCacheSize)
599 if not new and _transGroupPath in self:
600 # It does. Enable the undo.
--> 601 self.enableUndo()
602
603
/usr/lib/python2.4/site-packages/tables/File.py in enableUndo(self, filters)
1438 # Open the action log and go to the end of it
1439 self._actionlog = tgroup.actionlog
-> 1440 for row in self._actionlog:
1441 if row["opcode"] == _opToCode["MARK"]:
1442 name = row["arg2"]
/orange/scr1/mrFile_testing/roi_test/TableExtension.pyx in
TableExtension.Row.__next__()
/orange/scr1/mrFile_testing/roi_test/TableExtension.pyx in
TableExtension.Row.__next__general()
/orange/scr1/mrFile_testing/roi_test/TableExtension.pyx in
TableExtension.Row._read_records()
HDF5ExtError: Problems reading records.
-------------------------------------------------------
This SF.Net email is sponsored by xPML, a groundbreaking scripting language
that extends applications into web and mobile media. Attend the live webcast
and join the prime developer group breaking into this new coding territory!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642
_______________________________________________
Pytables-users mailing list
Pytables-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/pytables-users
