Hi List, Just a brief note to say that Blosc passed the most exigent tests suite in its integrated benchmark suite. Just today the complete 'extremesuite' bunch of tests has passed, after 3 days of intensive testing, on a Windows 64-bit box with a grand total of 95 TB compressed and 95 TB decompressed, without a single failure. These test suite uses synthetic data, but varies its entropy in a very wide range, and using many different type sizes and buffer lengths.
Also, other people have reported no problems with the 'hardsuite' in other platforms like Linux, FreeBSD or Mac OSX. Hence, I'd say that Blosc is perfectly suitable to be used in production scenarios. However, I won't declare it stable yet, until more test have been done with PyTables 2.2rc2 (to be out very soon now), where I'll include the latest revision of Blosc. Yesterday I was doing some benchmarks and the benefits of Blosc are more than apparent. With maximum compression level, it can reach compression ratios comparable to LZO (the fastest compressor that I was aware of up to now), while being typically 60% faster than it during compression, and 25% faster during decompression. For the lowest compression level, it can be up to 2.5x faster during compression and 2x during decompression. For PyTables Pro users, preliminary results says that the indexing speed is between 1.5x and 2.5x faster too. Definitely, after a a bit more of a year of working with Blosc, I'm very satisfied of these results: they are way better than I expected! Many thanks to all who collaborated with this large-scale testing effort! -- Francesc Alted ------------------------------------------------------------------------------ ThinkGeek and WIRED's GeekDad team up for the Ultimate GeekDad Father's Day Giveaway. ONE MASSIVE PRIZE to the lucky parental unit. See the prize list and enter to win: http://p.sf.net/sfu/thinkgeek-promo _______________________________________________ Pytables-users mailing list Pytables-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/pytables-users
