On Fri, Feb 08, 2008 at 09:43:45AM +0100, Ivan Vilata i Balaguer wrote: > I don't quite get this arrangement. Do you mean one ``EArray`` of > characters with the pickled representation of each list one after the > other, and another ``EArray`` storing the indices where each pickled > list starts in the first one? Now this is an interesting approach, and
Yes, this is exactly what I have done. > you may be approaching how ``VLArray`` objects work internally... That was a lucky co-incidence and which explains the closeness in speeds. > Well, this may either be a problem of inherent slowness in ``VLArray``, > or a problem with the particular way you rebuild your lists. It'd be It may be a problem in the way I am rebuilding the lists as I do not do anything smart with that respect. I will look at the split function you mentioned and will reply to the other email with my new timings. > interesting to measure both times separately. Also, this is the only > language-independent arrangement, which may iterest you portability- > wise. It is true that this is the most portable with respect to accessing the database through C for example. I have not known the cPickle is not platform independant but it makes sense, so therefore even python on a different platform would not read the database. However, for my use it is not really a problem, and the database can be dumped into an external representation (which was originally used as input) and recreated on a different machine. So it is not so much of a problem for me, but it could be problematic. > The demand for variable length fields has been a recurring topic in the > list, but the implementation doesn't seem trivial at all and we would > most probably need some external funding. So does HDF5 support these structures. I am new to HDF5 and pytables as you can guess from my various questions, so I do not know how far has pytables went with respect to wrapping the HDF5 database. Is there anything that Pytables does that cannot be reacreated with maybe some more c-code. Or is there some functionality that is present when programming in C that has not yet made it into pytables namely the variable width tables. Thanks, -- Hatem Nassrat ------------------------------------------------------------------------- This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ _______________________________________________ Pytables-users mailing list Pytables-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/pytables-users
