Hi List, I'm happy to announce the 0.3.1 version of carray, and in-memory compressed data container.
carray comes with a ctable object, that from this version on, can emulate most of the features of the Table object in PyTables. However, as a ctable object is 1) resident in-memory and 2) column wise ordered, it is able to get much more faster speeds than a regular Table object. The drawback of a ctable object is that it is not persistent (but you can always use PyTables to serialize ctable objects). My plan is to give full support of carray objects in PyTables (be ready to see a new flavor named `carray` for PyTables 3), because it can be a good complement to it (e.g. think of compressed, extremely fast I/O buffers implemented with carray objects). carray is still in beta, but already comes with a pretty complete test suite, and there are no *known* bugs at the moment. So you may want to try it out and see if it can serve your purposes of dealing with your huge datasets faster. Now, the official announcement, ======================== Announcing carray 0.3.1 ======================== What's new ========== This version implements multidimensional carrays and support for all NumPy dtypes, except 'object'. Some fixes have been included too. For more detailed info, see the release notes in: https://github.com/FrancescAlted/carray/wiki/Release-0.3.1 What it is ========== carray is a container for numerical data that can be compressed in-memory. The compresion process is carried out internally by Blosc, a high-performance compressor that is optimized for binary data. Having data compressed in-memory can reduce the stress of the memory subsystem. The net result is that carray operations may be faster than using a traditional ndarray object from NumPy. carray also supports fully 64-bit addressing (both in UNIX and Windows). Resources ========= Visit the main carray site repository at: http://github.com/FrancescAlted/carray You can download a source package from: http://carray.pytables.org/download Manual: http://carray.pytables.org/docs/manual Home of Blosc compressor: http://blosc.pytables.org User's mail list: car...@googlegroups.com http://groups.google.com/group/carray Share your experience ===================== Let us know of any bugs, suggestions, gripes, kudos, etc. you may have. ---- Enjoy! -- Francesc Alted ------------------------------------------------------------------------------ Protect Your Site and Customers from Malware Attacks Learn about various malware tactics and how to avoid them. Understand malware threats, the impact they can have on your business, and how you can protect your company and customers by using code signing. http://p.sf.net/sfu/oracle-sfdevnl _______________________________________________ Pytables-users mailing list Pytables-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/pytables-users
