Hello, I am trying to track down a problem I'm having with pytables segfaulting. The basic application involves a python thread that asynchronously posts data to a Queue, and then another thread reads data from the Queue and writes it to an H5 file. The main thread opens a new H5 file, then closes it after a few seconds of data has been written, then repeats opening and closing like this forever. I have flags in place to ensure that the writing thread does not write to the h5 file while it's in the process of being closed. However, I am still getting seg faults when closing the h5 file. Specifically I have traced it to closing the data Table. I have found that if I add the lines import gc gc.collect() the seg faulting seems to go away, but I imagine this is just hiding the problem.
The output for print_versions on this machine is: -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= PyTables version: 2.0.4 HDF5 version: 1.8.0 NumPy version: 1.1.0 Zlib version: 1.2.3.3 LZO version: 2.02 (Oct 17 2005) BZIP2 version: 1.0.4 (20-Dec-2006) Python version: 2.5.2 (r252:60911, Jul 23 2008, 23:54:29) [GCC 4.1.3 20070929 (prerelease) (Ubuntu 4.1.2-16ubuntu2)] Platform: linux2-x86_64 Byte-ordering: little -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= Just now, instead of a seg fault, I got a slightly useful error message: HDF5-DIAG: Error detected in HDF5 (1.8.0) thread 0: #000: H5Dio.c line 441 in H5Dwrite(): can't write data major: Dataset minor: Write failed #001: H5Dio.c line 680 in H5D_write(): src and dest data spaces have different sizes major: Invalid arguments to routine minor: Bad value Traceback (most recent call last): File "testh5.py", line 56, in <module> myHandler.stop_recording() File "/home/gej/spec/spec/spectroscopy/backend_components/datahandler.py", line 192, in stop_recording self.ts.close() File "/home/gej/spec/spec/spectroscopy/data/timeseries.py", line 197, in close self.h5.close() File "/usr/local/lib/python2.5/site-packages/tables/file.py", line 2051, in close lambda path: aliveNodes[path]) File "/usr/local/lib/python2.5/site-packages/tables/file.py", line 2024, in _c loseNodes node._f_close() File "/usr/local/lib/python2.5/site-packages/tables/table.py", line 2342, in _f_close self.flush() File "/usr/local/lib/python2.5/site-packages/tables/table.py", line 2280, in flush self.row._flushBufferedRows() File "tableExtension.pyx", line 1084, in tableExtension.Row._flushBufferedRows File "/usr/local/lib/python2.5/site-packages/tables/table.py", line 1769, in _saveBufferedRows self._append_records(lenrows) File "tableExtension.pyx", line 460, in tableExtension.Table._append_records tables.exceptions.HDF5ExtError: Problems appending the records. Running through gdb gives this back trace, which I don't imagine is very useful. #0 0x00002b5ef7423765 in raise () from /lib/libc.so.6 #1 0x00002b5ef74251c0 in abort () from /lib/libc.so.6 #2 0x00002b5ef745c60b in ?? () from /lib/libc.so.6 #3 0x00002b5ef7467826 in free () from /lib/libc.so.6 #4 0x00002b5ef7a9d77e in H5MM_xfree () from /usr/lib/libhdf5.so.5 #5 0x00002b5ef7a111f2 in H5D_istore_flush_entry () from /usr/lib/libhdf5.so.5 #6 0x00002b5ef7a12148 in H5D_istore_flush () from /usr/lib/libhdf5.so.5 #7 0x00002b5ef79fe3c5 in H5D_flush_real () from /usr/lib/libhdf5.so.5 #8 0x00002b5ef79fe5d4 in H5D_flush_cb () from /usr/lib/libhdf5.so.5 #9 0x00002b5ef7a912cc in H5I_search () from /usr/lib/libhdf5.so.5 #10 0x00002b5ef79fe076 in H5D_flush () from /usr/lib/libhdf5.so.5 #11 0x00002b5ef7a253cd in H5F_flush () from /usr/lib/libhdf5.so.5 #12 0x00002b5ef7a2922c in H5Fflush () from /usr/lib/libhdf5.so.5 #13 0x00002b5efe7d31e0 in __pyx_f_13hdf5Extension_4Leaf__g_flush ( __pyx_v_self=0x1e295f0, __pyx_args=<value optimized out>, __pyx_kwds=<value optimized out>) at src/hdf5Extension.c:2598 #14 0x0000000000488de0 in PyEval_EvalFrameEx (f=0x74ac4f0, throwflag=<value optimized out>) at Python/ceval.c:3573 #15 0x00000000004887c0 in PyEval_EvalFrameEx (f=0x1f8dc60, throwflag=<value optimized out>) at Python/ceval.c:3659 #16 0x00000000004887c0 in PyEval_EvalFrameEx (f=0x1f8da80, throwflag=<value optimized out>) at Python/ceval.c:3659 #17 0x0000000000489f9f in PyEval_EvalCodeEx (co=0xeaacd8, globals=<value optimized out>, locals=<value optimized out>, args=0x2, argcount=1, kws=0x2eb1f80, kwcount=0, defs=0xf297e8, defcount=1, closure=0x0) at Python/ceval.c:2836 #18 0x0000000000488366 in PyEval_EvalFrameEx (f=0x2eb1dd0, throwflag=<value optimized out>) at Python/ceval.c:3669 #19 0x0000000000489f9f in PyEval_EvalCodeEx (co=0xf63eb8, globals=<value optimized out>, locals=<value optimized out>, args=0x2eb1cf0, argcount=3, kws=0x2eb1d08, kwcount=0, defs=0xf38368, defcount=1, closure=0x0) at Python/ceval.c:2836 #20 0x0000000000488366 in PyEval_EvalFrameEx (f=0x2eb1b30, throwflag=<value optimized out>) at Python/ceval.c:3669 #21 0x0000000000489f9f in PyEval_EvalCodeEx (co=0xf68198, globals=<value optimized out>, locals=<value optimized out>, args=0x0, argcount=1, kws=0x2eb1ae8, kwcount=0, defs=0x0, defcount=0, closure=0x0) at Python/ceval.c:2836 #22 0x0000000000488366 in PyEval_EvalFrameEx (f=0x2eb1960, throwflag=<value optimized out>) at Python/ceval.c:3669 #23 0x00000000004887c0 in PyEval_EvalFrameEx (f=0x2eb17a0, throwflag=<value optimized out>) at Python/ceval.c:3659 #24 0x00000000004887c0 in PyEval_EvalFrameEx (f=0x77bd10, throwflag=<value optimized out>) at Python/ceval.c:3659 #25 0x0000000000489f9f in PyEval_EvalCodeEx (co=0x2b5ef69c9288, globals=<value optimized out>, locals=<value optimized out>, args=0x0, argcount=0, kws=0x0, kwcount=0, defs=0x0, defcount=0, closure=0x0) at Python/ceval.c:2836 #26 0x0000000000489fe2 in PyEval_EvalCode (co=0x3130, globals=0x3130, locals=0x6) at Python/ceval.c:494 #27 0x00000000004ab4ce in PyRun_FileExFlags (fp=0x759010, filename=0x7fffb4178897 "testh5.py", start=<value optimized out>, globals=0x77c640, locals=0x77c640, closeit=1, flags=0x7fffb4176650) at Python/pythonrun.c:1273 #28 0x00000000004ab760 in PyRun_SimpleFileExFlags (fp=0x759010, filename=0x7fffb4178897 "testh5.py", closeit=1, flags=0x7fffb4176650) at Python/pythonrun.c:879 #29 0x0000000000414645 in Py_Main (argc=<value optimized out>, argv=0x7fffb4176778) at Modules/main.c:523 #30 0x00002b5ef740fb44 in __libc_start_main () from /lib/libc.so.6 #31 0x0000000000413b89 in _start () Thanks for any help you can offer, Glenn ------------------------------------------------------------------------- This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/ _______________________________________________ Pytables-users mailing list Pytables-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/pytables-users