I feel like this is the same (only?) message I always send to people
on the list...
Have you considered simply storing each item as a separate atomic
array? Then you don't need to mess with any of this VLArray stuff, or
tables at all. I know it's called PyTables, but you can actually get
a lot done simply with named arrays (still get compression, some
cacheing and so forth).
Sure, it may be slower than a table, but you can get it done. Here's
an example of what I mean:
for i in range(10000):
f.createArray('/', 'array%d' % i,
np.random.random(np.random.random_integers(10)))
That takes a fair few seconds, but you can always make it faster
later... Two separate "tables" is currently the only way to deal with
variable length arrays in a Tabley way. Or, you can pad all arrays to
the same length.
Cheers,
Dav
On May 21, 2009, at 4:08 PM, Robert Ferrell wrote:
Okay, I'm completely confused. I've gone through the docs and there
are many examples, but I can't find anything as simple as I want.
Maybe I'm not using the right tool?
In it's simplest form, I have a large number of (name, np array)
tuples. (I happen to store them in a dict, and the np arrays are
actually TimeSeries instances, but I don't think that's important for
my very basic lack of understanding.)
I'd like to write a table with 2 columns, the first is a string (for
the name) the second is an np array (for the data). How do I do
that? I was expecting something like:
class SimpTab(tb.IsDescription):
name = tb.StringCol(itemsize=10)
dta = *something that says this is a numpy array*
I've found the createArray method, but I don't think that does what I
want. I thought there'd be some NpArrayCol() type thing.
I don't need to do any fancy selection, I just need to dump the data
to disk and then read it back in. The whole data set is only about
1GB, so it's no problem reading the whole thing into memory. Am I
using the wrong tool?
Sorry to be so dense.
thanks,
-robert
------------------------------------------------------------------------------
Register Now for Creativity and Technology (CaT), June 3rd, NYC. CaT
is a gathering of tech-side developers & brand creativity
professionals. Meet
the minds behind Google Creative Lab, Visual Complexity, Processing, &
iPhoneDevCamp asthey present alongside digital heavyweights like
Barbarian
Group, R/GA, & Big Spaceship. http://www.creativitycat.com
_______________________________________________
Pytables-users mailing list
Pytables-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/pytables-users
------------------------------------------------------------------------------
Register Now for Creativity and Technology (CaT), June 3rd, NYC. CaT
is a gathering of tech-side developers & brand creativity professionals. Meet
the minds behind Google Creative Lab, Visual Complexity, Processing, &
iPhoneDevCamp asthey present alongside digital heavyweights like Barbarian
Group, R/GA, & Big Spaceship. http://www.creativitycat.com
_______________________________________________
Pytables-users mailing list
Pytables-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/pytables-users
