On Wed, Apr 4, 2012 at 1:18 PM, Ashod Nakashian <ashodnakash...@yahoo.com> wrote:
> That's an easy question. The answer is that at *best* they'll do as good as > in-place compression. However, in practice > they'll do much worse. The reason is that the OS level compression works on > not only the single file level, but actually > at the block level. This is to make modifications reasonably fast (read > compressed data, uncompress, modify, write > recompressed data). If the complete file is compressed then even changing a > single byte (neglecting that no storage > works on the byte-level anyway) will yield performance that will at least > linearly degrade by the filesize. FWIW, that is exactly my concern with your custom file format. I do not see how you can achieve the benefits you expect without needing to repack files and I do not see how that can perform reasonably. That said, you also seem aware that the solution has to perform well so at worst it is just a question as to whether you want to spend the cycles to prove it can work and achieve all the goals. I am skeptical but look forward to being wrong. The lazy part of me thinks storing files up to 32KB in SQLite and storing the rest as just single compressed files would give 99% of our users what they want and would be less likely to have issues. -- Thanks Mark Phippard http://markphip.blogspot.com/
