On Sun, Jul 28, 2013 at 7:27 PM, Radio młodych bandytów < [email protected]> wrote:
> Thanks for the data. I took a quick glance now, later will do some more. > It appears that the requirement to halve input size is just too much for > lz4 to notably compress even English text...I suspect that few workloads > will benefit from LZ4 compression then. There's a need for something > stronger... > I think, the compression will be beneficial for log files, source code files and some types of uncompressed files. I don't think that changing the algorithm will significantly change how things are, because we have certain limitations, like we compress only one block at a time and we don't reference previous blocks at this moment, so the algorithms don't perform at their full potential. But I'll try other algorithms if I'll have enough time. > Anyway, my results with LZ4 r97 are slightly different, it managed to > halve 1 block of book1. Which version do you use? > I think I use r97 as well. Are you sure that your compression conditions are the same as mine? I only try to compress one 64KB block at a time and I don't use the information from previous blocks or anything else. > Overall, I have a feeling that a stronger LZ77 would be a better fit. > Something like https://code.google.com/p/data-shrinker/ (Warning: the > code is demo quality), Shrinker is the strongest-of-fast. > OK, this looks promising. When I was choosing an algorithm for the compression feature, one of the candidates was DEFLATE, which is based on LZ77. So, I'll definitely consider this data-shrinker as an alternative to LZ4. Thank you. Daniel
