A Wednesday 27 October 2010 11:31:38 Gerrit Holl escrigué:
> Hi,
>
> I have two tables in my dataset, a large one and a small one. The
> smaller table contains (among other data) pointers to row-numbers in
> the larger one. I would like to restrict the search to those rows.
>
> This approach works:
>
> indices =
> t.root.collocs.mean_collocations.read(start=1,stop=1494,field="FIRST
> ") S = set(indices) # faster searching
> res = (row for row in
> t.root.collocs.collocations.where(condition="(B_LAT < 0)",
> start=1, stop=65508)
> if row.nrow in S)
>
> However, I wonder if I can do this in-kernel somehow.
> mytable[indices].where(...) doesn't work, because mytable[indices] is
> a numpy.ndarray already, not a table that supports in-kernel
> searching. I tried something like mytable.where("(B_LAT < 0) & (NO in
> S)") (where I need to add NO explicitly to the table?), but this
> results in an exception "TypeError: variable ``S`` has data type
> ``object``, not allowed in conditions". If I use "...NO in
> indices..." instead (should be much slower) I get "TypeError:
> argument of type 'VariableNode' is not iterable"
>
> Can this be done in-kernel?
>
> I could rearrange my data in a quite different way, and rather than
> have those 'pointers', use an embedded table. However, quite often I
> am only interested in the larger table, and searching in this one
> would become much slower. Yet another approach that would allow
> in-kernel searches would be to repeat the entire row from the larger
> table in the smaller table, but that would be inefficient
> storage-wise.
Mmh, I don't think that using a in-kernel query for this scenario is
going to be possible. I'd add a new column in the large table saying
whether the row should be selected or not. This requires a little more
space, but if compression is used, perhaps you can bear the overhead.
Cheers,
--
Francesc Alted
------------------------------------------------------------------------------
Nokia and AT&T present the 2010 Calling All Innovators-North America contest
Create new apps & games for the Nokia N8 for consumers in U.S. and Canada
$10 million total in prizes - $4M cash, 500 devices, nearly $6M in marketing
Develop with Nokia Qt SDK, Web Runtime, or Java and Publish to Ovi Store
http://p.sf.net/sfu/nokia-dev2dev
_______________________________________________
Pytables-users mailing list
Pytables-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/pytables-users
