On Sun, May 9, 2010 at 8:33 PM, Jim Fulton <j...@zope.com> wrote: > Our recent discussion of compression made me curious so I did some > analysis of pickle sizes in one of our large databases. This is for a > content management system. The database is packed weekly. It doesn't > include media, which are in blobs. > > There were ~19 million transaction in the database and around 130 > million data records. About 60% of the size was taken up by BTrees. > Compressing pickles using zlib with default compression reduced the > pickle sizes by ~58%. The average uncompressed record size was 1163 > bytes. The average compressed size was ~493 bytes. > > This is probably enough of a savings to make compression interesting. > > Jim > > -- > Jim Fulton > _______________________________________________ > For more information about ZODB, see the ZODB Wiki: > http://www.zope.org/Wikis/ZODB/ > > ZODB-Dev mailing list - ZODB-Dev@zope.org > https://mail.zope.org/mailman/listinfo/zodb-dev >
That's really interesting! Did you notice any issues performance wise, or didn't you check that yet? I wonder what DirectoryStorage on btrfs with transparent compression can do btw. -- Roel Bruggink http://www.fourdigits.nl/mensen/roel-bruggink Four Digits BV http://www.fourdigits.nl Willemsplein 44, 6811 KD, Arnhem tel: +31(0)26 4422700 fax: +31(0)84 2206117 KVK 091621370000 BTW 8161.22.234.B01
_______________________________________________ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
