New submission from algo_project <algo_proj...@protonmail.com>:
I believe the 'tables' module is the cause of this error due to debugging done
with gdb (see pic related - script is tested on two different servers with the
same script to ensure it's not a hardware issue).
I receive "Segmentation Fault (core dumped)" while running a data collection
program. It crashes around 1 to 5 minutes after start time, sometimes less.
The program uses Pandas and writes to h5 files on 14 threads. Each thread is
collecting data for cryptocurrencies on different exchanges and a separate h5
file is created for each coin and exchange (ie no possible race conditions). In
fact when attempting to fix this problem I placed a threadlock on all calls to
write (or read) to any h5 file but the segmentation fault persists.
I am running a different program on a separate server that isn't multithreaded
and that has been running fine for weeks.
A difference between the two programs is the multithreaded one makes use of
<pandas.HDFStore(path)>.append(), whereas the sequential one only uses the
write() method. Can't be sure if it's relevant but it could be worth including.
I have tried searching for this bug but had no success. Upon crashing it
sometimes produces "Bus Error (core dumped)" and it sometimes produces
"Segmentation Error (core dumped)", but each time the debugging error message
from gdb is the same.
For convenience I'll paste the error message here so you can copy instead of
typing from the image:
Program terminated with signal SIGSEGV, Segmentation fault.
#0 0x00007f6e584715ba in H5SL_release_common ()
from
/usr/local/lib/python3.5/dist-packages/tables/.libs/libhdf5-f635fc60.so.10.2.1
Also, this is my first bug report so my apologies if the etiquette isn't
correct. I read over the guidelines before posting but since I have no
experience with memory management I feel I don't have a good enough
understanding to explain the bug in more depth!
Cheers.
----------
files: segmentation_fault_debugging_message.JPG
messages: 3445
nosy: algo_project
priority: bug
status: unread
title: PyTables giving SIGSEGV when using HDF5 with multithreading
topic: crash
_______________________________________________________
PSF Meta Tracker <metatrac...@psf.upfronthosting.co.za>
<http://psf.upfronthosting.co.za/roundup/meta/issue653>
_______________________________________________________
_______________________________________________
Tracker-discuss mailing list
Tracker-discuss@python.org
https://mail.python.org/mailman/listinfo/tracker-discuss
Code of Conduct: https://www.python.org/psf/codeofconduct/
