Hey Tom,
On Sun, Jan 30, 2011 at 8:01 PM, Tom Fawcett <tom.fawc...@gmail.com> wrote:
> Thanks for your response. I suspect you are right that sets are not
> regular enough. Are all column types of fixed size? Is this a necessary
> condition for pytables? I though I’d seen an example of variable-length
> strings with pytables, but I maybe I was mistaken.
>
Ahh Yes all columns are of a fixed size. What you probably saw with strings
is that you can store a length = 1, 2, .... N-1 string in a length N string
column. The string value simply determines where (effectively) the
null-terminated bytes goes. In terms of storage, even a length zero string
in a length N column will still take up length N space in memory.
This is pretty much what I was proposing for a set column.
> I was thinking of solving the problem by creating a new table the
> set-valued column and inserting values (along with row IDs) into it. I
> could then do a join with the main table to retrieve the rows I want.
> Although pytables can’t do table joins naturally, I could use the join
> method described in “Hints for SQL Users”. Though I’m not using indices,
> this may be fast enough.
>
Yup this is pretty much the way I have solved similar problems. Joins can
be pretty fast in pytables if you use the where() method to get the indices
that you need.
Enjoy!
Be Well
Anthony
> Thanks again,
> -Tom
>
> On Jan 30, 2011, at 12:35 AM, Anthony Scopatz wrote:
>
> Hey Tom,
>
> I am not sure what you are asking is possible. I don't think sets are
> regular enough* container type. Perhaps if you had a sets of consistent (or
> maximal) length and primitive data type, this kind of thing would be
> possible. (Eg every set in your column only contained integers and has less
> than or equal to 10 elements.) However, this is functionally equivalent to
> an array**.
>
> If you need the data in set form why not use something like the following:
>
> for row in table:
> s = set(row['col_name'])
>
> Additionally, you could store a set as an attribute on a Node (
> http://www.pytables.org/docs/manual/ch04.html#AttributeSetClassDescr) or
> perhaps you could try storing set definitions or pickled sets in a series of
> filenodes (http://www.pytables.org/docs/manual/ch06.html). It should be
> noted that either of these methods will likely be slower than the array/set
> idea.
>
> Be Well
> Anthony
>
> *ie you can't memory map into a set the same way that you can an array.
> Elements of a set can be anywhere in memory. Elements of an array
> are guaranteed to be available sequentially.
>
> **You could probably hack an array column that reads out into a set
> automatically, rather than a numpy array. However, it would still be stored
> under the covers as an array, and thus subject to all the array-constraints.
>
> On Sun, Jan 30, 2011 at 7:20 AM, Tom Fawcett <tom.fawc...@gmail.com>wrote:
>
>> Hi. I recently discovered pytables, and I’m impressed with its speed and
>> flexibility. But there’s one feature I’d like and I can’t figure out how
>> whether pytables can do it.
>>
>> I’d like to have essentially a set-valued column in a table. I’d like a
>> column data type that is very similar to a Python set. I’m not sure how to
>> achieve this naturally with pytables – it looks like none of the Cols
>> subclasses can be variable length. Any ideas?
>>
>> Thanks,
>> -Tom
>>
>> ------------------------------------------------------------------------------
>> Special Offer-- Download ArcSight Logger for FREE (a $49 USD value)!
>> Finally, a world-class log management solution at an even better
>> price-free!
>> Download using promo code Free_Logger_4_Dev2Dev. Offer expires
>> February 28th, so secure your free ArcSight Logger TODAY!
>> http://p.sf.net/sfu/arcsight-sfd2d
>> _______________________________________________
>> Pytables-users mailing list
>> Pytables-users@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/pytables-users
>>
>
>
> ------------------------------------------------------------------------------
> Special Offer-- Download ArcSight Logger for FREE (a $49 USD value)!
> Finally, a world-class log management solution at an even better
> price-free!
> Download using promo code Free_Logger_4_Dev2Dev. Offer expires
> February 28th, so secure your free ArcSight Logger TODAY!
>
> http://p.sf.net/sfu/arcsight-sfd2d_______________________________________________
> Pytables-users mailing list
> Pytables-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/pytables-users
>
>
>
>
> ------------------------------------------------------------------------------
> Special Offer-- Download ArcSight Logger for FREE (a $49 USD value)!
> Finally, a world-class log management solution at an even better
> price-free!
> Download using promo code Free_Logger_4_Dev2Dev. Offer expires
> February 28th, so secure your free ArcSight Logger TODAY!
> http://p.sf.net/sfu/arcsight-sfd2d
> _______________________________________________
> Pytables-users mailing list
> Pytables-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/pytables-users
>
>
------------------------------------------------------------------------------
Special Offer-- Download ArcSight Logger for FREE (a $49 USD value)!
Finally, a world-class log management solution at an even better price-free!
Download using promo code Free_Logger_4_Dev2Dev. Offer expires
February 28th, so secure your free ArcSight Logger TODAY!
http://p.sf.net/sfu/arcsight-sfd2d
_______________________________________________
Pytables-users mailing list
Pytables-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/pytables-users
