thanks
created
https://github.com/PyTables/PyTables/issues/198
I can be reached on my cell (917)971-6387
________________________________
From: Anthony Scopatz <scop...@gmail.com>
To: Jeff Reback <j...@reback.net>; Discussion list for PyTables
<pytables-users@lists.sourceforge.net>
Sent: Monday, December 3, 2012 11:15 AM
Subject: Re: [Pytables-users] variable length strings in tables?
On Sun, Dec 2, 2012 at 2:49 PM, Jeff Reback <jreb...@yahoo.com> wrote:
Hi,
>
>Pandas uses pytables as a storage backend and has worked out quite well
>fyi ... http://pandas.pydata.org/pandas-docs/dev/io.html#hdf5-pytables
>
>I have a particular use case where I build a table, then later append to it.
>Fixed types are no problem. However, I often index these tables by StringCols,
>which I pre-allocated
>to the largest size I think that i'll need. So, wanted to think about
>supporting
>variable-length string columns in the table.
>
>any thoughts on these strategies:
>1) any way to directly support a variable-length string in a particular
>column? (e.g. VLStringCol doesn't exist but a stand-alone VLStringAtom does)
This is possible as the underlying HDF5 library will support it. However, no
one has had the time to write it. Please open an issue (or possibly a pull
request related to this.)
2) As an alternative, I could store along with the table a VLArray the same #
of rows as the table and keep string data here
> -- of course have to keep the synchronization up to date (and this doesn't
>help with an 'indexing' column, just with 'data' columns)
This is what I do in PyTables and HDF5 itself. It works out quite well for me.
This has the advantage that the VLString data get compressed separately from
the numeric data (if using compression). Yes, it is one more thing to manage,
but the file sizes I are much significantly smaller.
Be Well
Anthony
>thanks,
>
>Jeff
>------------------------------------------------------------------------------
>Keep yourself connected to Go Parallel:
>DESIGN Expert tips on starting your parallel project right.
>http://goparallel.sourceforge.net/
>_______________________________________________
>Pytables-users mailing list
>Pytables-users@lists.sourceforge.net
>https://lists.sourceforge.net/lists/listinfo/pytables-users
>
>
------------------------------------------------------------------------------
Keep yourself connected to Go Parallel:
BUILD Helping you discover the best ways to construct your parallel projects.
http://goparallel.sourceforge.net
_______________________________________________
Pytables-users mailing list
Pytables-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/pytables-users
