Hi Francesc, Hearing that row-major order is hardcoded behind the scene is rather a bad news for me. You are right, there is no way to store ordering metadata in HDF5, but I do not see any need for any. I have been using HDF5 with fortran/matlab data for a very long time. One just needs to know the ordering of the data, and/or store it in XDMF light xml description. Does this sound reasonable for pytables?
Dominik On Mon, Dec 13, 2010 at 10:22 AM, Francesc Alted <fal...@pytables.org>wrote: > A Monday 13 December 2010 09:34:25 Dominik Szczerba escrigué: > > Hi Francesc, > > > > Thanks for confirming my guess. > > I would vote for a different solution than removing this warning in > > the 'expert' mode. > > How about explicitly declaring ordering somewhere in pytables? > > Or is this warning the only place where the ordering matters? > > No, the ordering also affects the way iterators walk data on datasets. > On a Fortran order dataset, the iterators should return slices > orthogonal to the trailing dimension, not the leading one (default). > > Hmm, supporting Fortran order in PyTables is appealing, but I see a > major drawback: there is not a standard way (that I'm aware of) for > saving this metainfo in HDF5. So, unless the file has been created with > PyTables, you will not be able to guess the data order. > > Having said this, PyTables has a feature that can 'fake' in some way a > 'Fortran ordered' array. This is by using an EArray (just a HDF5 > dataset with one of its dimensions that is set to be enlargeable; this > dimension is called the 'main' dimension). EArrays are iterated through > the enlargeable (main) dimension instead of the leading one. This is > not exactly a Fortran-ordered dataset, but can be useful in some > circumstances (specially for 2-dimensional arrays). > > OTOH, if what you want is to get rid of the warning, another solution is > to play with the IO_BUFFER_SIZE and BUFFER_TIMES parameters: > > http://www.pytables.org/docs/manual/apc.html#id364831 > > until you reach a good balance for your datasets. > > Now that I think, it would be a good idea to add a hint about suggesting > tailoring IO_BUFFER_SIZE and BUFFER_TIMES params in the warning rather > than adding a brand-new EXPERT_MODE one. Updated ticket: > > http://pytables.org/trac/ticket/327#comment:1 > > -- > Francesc Alted > > > ------------------------------------------------------------------------------ > Oracle to DB2 Conversion Guide: Learn learn about native support for > PL/SQL, > new data types, scalar functions, improved concurrency, built-in packages, > OCI, SQL*Plus, data movement tools, best practices and more. > http://p.sf.net/sfu/oracle-sfdev2dev > _______________________________________________ > Pytables-users mailing list > Pytables-users@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/pytables-users > >
------------------------------------------------------------------------------ Oracle to DB2 Conversion Guide: Learn learn about native support for PL/SQL, new data types, scalar functions, improved concurrency, built-in packages, OCI, SQL*Plus, data movement tools, best practices and more. http://p.sf.net/sfu/oracle-sfdev2dev
_______________________________________________ Pytables-users mailing list Pytables-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/pytables-users
