Hello,
I am attempting to use PyTables (v2.3.1) to store timestamped data and
things were going well until I added a column index. While the column
is indexed no data is returned from a table.where call!
This behavior is demonstrated with the following test code:
---begin test.py---
import tables
import random
class Descr(tables.IsDescription):
when = tables.Time64Col(pos = 1)
value = tables.Float32Col(pos = 2)
h5f = tables.openFile('/tmp/tmp.h5', 'w')
tbl = h5f.createTable('/', 'test', Descr)
tbl.cols.when.createIndex(_verbose = True)
t = 1321031471.0 # 11/11/11 11:11:11
tbl.append([(t + i, random.random()) for i in range(1000)])
tbl.flush()
def query(s):
print 'is_index =', tbl.cols.when.is_indexed
print [(row['when'], row['value']) for row in tbl.where(s)]
print tbl.readWhere(wherestr)
wherestr = '(when >= %d) & (when < %d)'%(t, t+5)
query(wherestr)
tbl.cols.when.removeIndex()
query(wherestr)
h5f.close()
---end test.py---
This creates the table for storing time/value pairs, inserts some
synthetic data, and then checks to see if there is data in the table.
When the table is created there is an index added to the 'where'
column. The first query returns no data (which is incorrect). Then
the column index is removed (via table.removeIndex) and the query is
repeated. This time 5 results are returned as expected. The data is
clearly there however the index is somehow breaking the where logic.
Here is the output I get:
---begin output---
is_index = True
[]
[]
is_index = False
[(1321031471.0, 0.6449417471885681), (1321031472.0,
0.7889317274093628), (1321031473.0, 0.609708845615387), (1321031474.0,
0.9120397567749023), (1321031475.0, 0.2386845201253891)]
[(1321031471.0, 0.6449417471885681) (1321031472.0, 0.7889317274093628)
(1321031473.0, 0.609708845615387) (1321031474.0, 0.9120397567749023)
(1321031475.0, 0.2386845201253891)]
---end output---
Creating the index after the data has been inserted produces the same
behavior (no data is returned while the index exists). Any
suggestions would be greatly appreciated.
Alan
------------------------------------------------------------------------------
All the data continuously generated in your IT infrastructure
contains a definitive record of customers, application performance,
security threats, fraudulent activity, and more. Splunk takes this
data and makes sense of it. IT sense. And common sense.
http://p.sf.net/sfu/splunk-novd2d
_______________________________________________
Pytables-users mailing list
Pytables-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/pytables-users
