Hello,
I am trying to track down a problem I'm having with pytables segfaulting. The
basic application involves a python thread that asynchronously posts data to a
Queue, and then another thread reads data from the Queue and writes it to an H5
file. The main thread opens a new H5 file, then closes it after a few seconds of
data has been written, then repeats opening and closing like this forever. I
have flags in place to ensure that the writing thread does not write to the h5
file while it's in the process of being closed. However, I am still getting seg
faults when closing the h5 file. Specifically I have traced it to closing the
data Table. I have found that if I add the lines
import gc
gc.collect()
the seg faulting seems to go away, but I imagine this is just hiding the 
problem.

The output for print_versions on this machine is:
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
PyTables version:  2.0.4
HDF5 version:      1.8.0
NumPy version:     1.1.0
Zlib version:      1.2.3.3
LZO version:       2.02 (Oct 17 2005)
BZIP2 version:     1.0.4 (20-Dec-2006)
Python version:    2.5.2 (r252:60911, Jul 23 2008, 23:54:29)
[GCC 4.1.3 20070929 (prerelease) (Ubuntu 4.1.2-16ubuntu2)]
Platform:          linux2-x86_64
Byte-ordering:     little
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=

Just now, instead of a seg fault, I got a slightly useful error message:

HDF5-DIAG: Error detected in HDF5 (1.8.0) thread 0:
  #000: H5Dio.c line 441 in H5Dwrite(): can't write data
    major: Dataset
    minor: Write failed
  #001: H5Dio.c line 680 in H5D_write(): src and dest data spaces have different
sizes
    major: Invalid arguments to routine
    minor: Bad value
Traceback (most recent call last):
  File "testh5.py", line 56, in <module>
    myHandler.stop_recording()
  File "/home/gej/spec/spec/spectroscopy/backend_components/datahandler.py",
line 192, in stop_recording
    self.ts.close()
  File "/home/gej/spec/spec/spectroscopy/data/timeseries.py", line 197, in close
    self.h5.close()
  File "/usr/local/lib/python2.5/site-packages/tables/file.py", line 2051, in 
close
    lambda path: aliveNodes[path])
  File "/usr/local/lib/python2.5/site-packages/tables/file.py", line 2024, in _c
loseNodes
    node._f_close()
  File "/usr/local/lib/python2.5/site-packages/tables/table.py", line 2342, in
_f_close
    self.flush()
  File "/usr/local/lib/python2.5/site-packages/tables/table.py", line 2280, in 
flush
    self.row._flushBufferedRows()
  File "tableExtension.pyx", line 1084, in tableExtension.Row._flushBufferedRows
  File "/usr/local/lib/python2.5/site-packages/tables/table.py", line 1769, in
_saveBufferedRows
    self._append_records(lenrows)
  File "tableExtension.pyx", line 460, in tableExtension.Table._append_records
tables.exceptions.HDF5ExtError: Problems appending the records.


Running through gdb gives this back trace, which I don't imagine is very useful.
#0  0x00002b5ef7423765 in raise () from /lib/libc.so.6
#1  0x00002b5ef74251c0 in abort () from /lib/libc.so.6
#2  0x00002b5ef745c60b in ?? () from /lib/libc.so.6
#3  0x00002b5ef7467826 in free () from /lib/libc.so.6
#4  0x00002b5ef7a9d77e in H5MM_xfree () from /usr/lib/libhdf5.so.5
#5  0x00002b5ef7a111f2 in H5D_istore_flush_entry () from /usr/lib/libhdf5.so.5
#6  0x00002b5ef7a12148 in H5D_istore_flush () from /usr/lib/libhdf5.so.5
#7  0x00002b5ef79fe3c5 in H5D_flush_real () from /usr/lib/libhdf5.so.5
#8  0x00002b5ef79fe5d4 in H5D_flush_cb () from /usr/lib/libhdf5.so.5
#9  0x00002b5ef7a912cc in H5I_search () from /usr/lib/libhdf5.so.5
#10 0x00002b5ef79fe076 in H5D_flush () from /usr/lib/libhdf5.so.5
#11 0x00002b5ef7a253cd in H5F_flush () from /usr/lib/libhdf5.so.5
#12 0x00002b5ef7a2922c in H5Fflush () from /usr/lib/libhdf5.so.5
#13 0x00002b5efe7d31e0 in __pyx_f_13hdf5Extension_4Leaf__g_flush (
    __pyx_v_self=0x1e295f0, __pyx_args=<value optimized out>,
    __pyx_kwds=<value optimized out>) at src/hdf5Extension.c:2598
#14 0x0000000000488de0 in PyEval_EvalFrameEx (f=0x74ac4f0,
    throwflag=<value optimized out>) at Python/ceval.c:3573
#15 0x00000000004887c0 in PyEval_EvalFrameEx (f=0x1f8dc60,
    throwflag=<value optimized out>) at Python/ceval.c:3659
#16 0x00000000004887c0 in PyEval_EvalFrameEx (f=0x1f8da80,
    throwflag=<value optimized out>) at Python/ceval.c:3659
#17 0x0000000000489f9f in PyEval_EvalCodeEx (co=0xeaacd8,
    globals=<value optimized out>, locals=<value optimized out>, args=0x2,
    argcount=1, kws=0x2eb1f80, kwcount=0, defs=0xf297e8, defcount=1,
    closure=0x0) at Python/ceval.c:2836
#18 0x0000000000488366 in PyEval_EvalFrameEx (f=0x2eb1dd0,
    throwflag=<value optimized out>) at Python/ceval.c:3669
#19 0x0000000000489f9f in PyEval_EvalCodeEx (co=0xf63eb8,
    globals=<value optimized out>, locals=<value optimized out>,
    args=0x2eb1cf0, argcount=3, kws=0x2eb1d08, kwcount=0, defs=0xf38368,
    defcount=1, closure=0x0) at Python/ceval.c:2836
#20 0x0000000000488366 in PyEval_EvalFrameEx (f=0x2eb1b30,
    throwflag=<value optimized out>) at Python/ceval.c:3669
#21 0x0000000000489f9f in PyEval_EvalCodeEx (co=0xf68198,
    globals=<value optimized out>, locals=<value optimized out>, args=0x0,
    argcount=1, kws=0x2eb1ae8, kwcount=0, defs=0x0, defcount=0, closure=0x0)
    at Python/ceval.c:2836
#22 0x0000000000488366 in PyEval_EvalFrameEx (f=0x2eb1960,
    throwflag=<value optimized out>) at Python/ceval.c:3669
#23 0x00000000004887c0 in PyEval_EvalFrameEx (f=0x2eb17a0,
    throwflag=<value optimized out>) at Python/ceval.c:3659
#24 0x00000000004887c0 in PyEval_EvalFrameEx (f=0x77bd10,
    throwflag=<value optimized out>) at Python/ceval.c:3659
#25 0x0000000000489f9f in PyEval_EvalCodeEx (co=0x2b5ef69c9288,
    globals=<value optimized out>, locals=<value optimized out>, args=0x0,
    argcount=0, kws=0x0, kwcount=0, defs=0x0, defcount=0, closure=0x0)
    at Python/ceval.c:2836
#26 0x0000000000489fe2 in PyEval_EvalCode (co=0x3130, globals=0x3130,
    locals=0x6) at Python/ceval.c:494
#27 0x00000000004ab4ce in PyRun_FileExFlags (fp=0x759010,
    filename=0x7fffb4178897 "testh5.py", start=<value optimized out>,
    globals=0x77c640, locals=0x77c640, closeit=1, flags=0x7fffb4176650)
    at Python/pythonrun.c:1273
#28 0x00000000004ab760 in PyRun_SimpleFileExFlags (fp=0x759010,
    filename=0x7fffb4178897 "testh5.py", closeit=1, flags=0x7fffb4176650)
    at Python/pythonrun.c:879
#29 0x0000000000414645 in Py_Main (argc=<value optimized out>,
    argv=0x7fffb4176778) at Modules/main.c:523
#30 0x00002b5ef740fb44 in __libc_start_main () from /lib/libc.so.6
#31 0x0000000000413b89 in _start ()


Thanks for any help you can offer,
Glenn


-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Pytables-users mailing list
Pytables-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/pytables-users

Reply via email to