Hi guys,
I'm having some issues with multiple threads and pytables. I have a
single lock attached to the file and use it when both reading and
writing to the file. When I'm only running one thread there aren't
any problems, though several threads cause some strange and
unpredictable behaviour. It seems to be more stable on linux then
windows for some reason.
Symptoms include:
- program crash. no exceptions. no trace back. app simply disappears
- unexpected exceptions
- write failures
It would be nice if someone could advice me on some best practices for
using threads with pytables. What should I consider as an "atomic"
operation from my side?
Hope you all have a nice day.
Regards,
Christian
Here's a collection of some typical exceptions:
Exception exceptions.KeyError: ('/securities/HKSE/0718.HK/historics',) in <bound
method Group.__del__ of /securities/ASX/AFG.AX (Group) ''
children := ['historics' (Table)]> ignored
Exception exceptions.KeyError: ('/securities/NasdaqNM/ACAT.O/historics',) in <bo
und method Table.__del__ of /securities/NYSE/AMD.N/historics (Table(6558L,), shu
ffle, zlib(5)) ''
description := {
"close": Float64Col(shape=(), dflt=0.0, pos=0),
"high": Float64Col(shape=(), dflt=0.0, pos=1),
"low": Float64Col(shape=(), dflt=0.0, pos=2),
"open": Float64Col(shape=(), dflt=0.0, pos=3),
"time": Int64Col(shape=(), dflt=0, pos=4),
"volume": Int64Col(shape=(), dflt=0, pos=5)}
byteorder := 'little'
chunkshape := (170L,)> ignored
Write failure: dtype('int64') |
Write failure: dtype('int64') |
File "C:\lynxgraphs\lynxgraphs\finance\datatype\timeseries.py", line 173, in _
data_default
return self.read_records()
File "C:\lynxgraphs\lynxgraphs\finance\datatype\timeseries.py", line 153, in r
ead_records
node_exists = self.path in self.h5f
File "C:\Python25\lib\site-packages\pytables-2.0.4.0001-py2.5-win32.egg\tables
\file.py", line 1332, in __contains__
self.getNode(path)
File "C:\Python25\lib\site-packages\pytables-2.0.4.0001-py2.5-win32.egg\tables
\file.py", line 992, in getNode
node = self._getNode(nodePath)
File "C:\Python25\lib\site-packages\pytables-2.0.4.0001-py2.5-win32.egg\tables
\file.py", line 918, in _getNode
parentNode = self._reviveNode(parentPath)
File "C:\Python25\lib\site-packages\pytables-2.0.4.0001-py2.5-win32.egg\tables
\file.py", line 2219, in _reviveNode
node = self._deadNodes.pop(nodePath)
File "C:\Python25\lib\site-packages\pytables-2.0.4.0001-py2.5-win32.egg\tables
\misc\lrucache.py", line 201, in pop
self.__heap.remove(node)
ValueError: list.remove(x): x not in list
Exception exceptions.KeyError: ('/securities/NYSE/ACE.N/historics',) in <bound m
ethod Table.__del__ of /securities/NCM/ADES.OQ/historics (Table(1308L,), shuffle
, zlib(5)) ''
description := {
"close": Float64Col(shape=(), dflt=0.0, pos=0),
"high": Float64Col(shape=(), dflt=0.0, pos=1),
"low": Float64Col(shape=(), dflt=0.0, pos=2),
"open": Float64Col(shape=(), dflt=0.0, pos=3),
"time": Int64Col(shape=(), dflt=0, pos=4),
"volume": Int64Col(shape=(), dflt=0, pos=5)}
byteorder := 'little'
chunkshape := (170L,)> ignored
------------------------------------------------------------------------------
_______________________________________________
Pytables-users mailing list
Pytables-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/pytables-users
