Hi Stephen, On Tuesday 12 May 2009 21:36:26 Stephen Simmons wrote: > The recent postings "about pytables interface to sqlite" started me > thinking about where PyTables was headed. Or perhaps, as PyTables is > Francesc's baby, where he wants it to head.
Well, it is not only my baby: I'd like it to be the baby of other PyTables users and open to anybody that want to contribute back to the project too. > I don't have any specific > point to make here, just wondering things such as: > > - Are there opportunities to create "value added" packages giving > higher level functionality? For example I've written disk-based sort, > merge and split routines for tables too big to fit in memory, iterators > that return chunks of tables as numpy recarrays, iterators with hooks > for progress bars, etc. I don't recalls seeing any community discussions > about contributing back to the PyTables ecosystem. Wow, that's a lot of very interesting stuff. As I said before, I'm really open to integrate this into PyTables. And regarding community contributions, there have been many, like the complex support (Tom Hedley), the CArray object (Antonio Valentino) or the netCDF3 API support (Jeffrey Whitaker), just to name some of the most important. It is true that lately there have been little contributions, but this does not mean that I'm less open to do that than before :) > - Can the audience for PyTables be increased by a cookbook with recipes > combining PyTables with numpy? Speaking for myself, it took many months > for me to realise my "for row in tbl" code could be parallelised with > the idiom "arr = tbl.read[100:1000]". In other words, to see PyTables as > a super-flexible persistence layer for numpy. Especially as a newbie, I > would have a benefitted from more examples to get me thinking. If I recall correctly, we had a Cookbook page in the PyTables site, but I've decided to remove it as it was semi-empty. However, there is a page for people that wants to contribute documentation: http://www.pytables.org/moin/UserDocuments You are welcome to publish your info there, or if people likes it better, I can bring the Cookbook into life again (in the hope that somebody will contribute something). > - And then there's the more heretical question of whether to use h5py > instead of PyTables. I don't know enough about h5py to compare the two. > Right now, PyTables appears to be in a period of maturity and stability > (which is good) while h5py doesn't yet support LZO compression, which I > need for reading historical data. So I stick with PyTables, and > occasionally check the h5py web site to see if there has been a new > release. The beauty of the free software and Internet is that it allows you to easily try new packages. h5py is a nice project, and I'm really happy that PyTables is not alone offering a Python interface to HDF5. Both projects have different approaches and goals though, so you should choose the best for your needs (or better yet, use both at the time if you feel this is good for you). > Anyway, that's enough from me. I don't have any concrete answers, but > would be pleased if others posted their thoughts or reactions. Your thoughts have been very interesting for me, and touched important points. Hopefully they would serve to trigger some reflections/discussions on the PyTables community. Thanks for your feedback! Regards, -- Francesc Alted ------------------------------------------------------------------------------ The NEW KODAK i700 Series Scanners deliver under ANY circumstances! Your production scanning environment may not be a perfect world - but thanks to Kodak, there's a perfect scanner to get the job done! With the NEW KODAK i700 Series Scanner you'll get full speed at 300 dpi even with all image processing features enabled. http://p.sf.net/sfu/kodak-com _______________________________________________ Pytables-users mailing list Pytables-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/pytables-users
