Hi Angel
On Thu, Mar 15, 2012 at 3:29 PM, Angel Medrano <asmedr...@gmail.com> wrote:
> Hi,
> I come from RDMS background and am experimenting with Pytables + HDF5 to
> store a massive amount of records we keep. Im trying to understand how to
> make the most out of pytables but am having some trouble understanding some
> concepts.
>
> Currently my data structure looks like this
>
> Group_1
> ----nested_array_1
> ----nested_array_2
> Group_2
> ----nested_array_1
> ----nested_array_2
>
> That's it in a nutshell.
>
> When i want to "query it" id do something
> like hdf5file.getNode("/Group_1/nested_array_1").
>
Querying is actually done using where() methods and functions. The
getNode() method simply
gives you a handle (object) which refers to this node. Think of HDF5 files
as a self-contained
file system. Thus you can query datasets (files) but only get metadata out
of the nodes (dirs,
filehandlers)
>
> So my questions are... is this the best way of doing this?
> Am I on the right track?
>
Seems ok to me.
>
> If I use a Table instead... how do you store a large array?
>
The same way you store a small array ;)
> Would indexing (table.col.id.createIndex()) speed things up?
>
Usually. It depends on what your data looks like and your queries.
> Just a side note, so far, I've gotten faster results using Postgres +
> psycopg2.
>
I imagine that this is dependent on the data that you are storing and how
much of it
you have. HDF5 & PyTables really shine with large amounts of numeric data
that is
regularly structured.
If you have a comparison script lying around, I would be interested!
Be Well
Anthony
>
> Anyway, thanks in advance!
>
> -Angel Medrano
>
>
>
>
>
> --
> *Angel Medrano*
> angelmedrano.com
> @asmedrano
> 214-392-2597
>
>
>
> ------------------------------------------------------------------------------
> This SF email is sponsosred by:
> Try Windows Azure free for 90 days Click Here
> http://p.sf.net/sfu/sfd2d-msazure
> _______________________________________________
> Pytables-users mailing list
> Pytables-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/pytables-users
>
>
------------------------------------------------------------------------------
This SF email is sponsosred by:
Try Windows Azure free for 90 days Click Here
http://p.sf.net/sfu/sfd2d-msazure
_______________________________________________
Pytables-users mailing list
Pytables-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/pytables-users
