Many data-compression schemes are conceptually composed of two steps:
LZ77 (finding repeated sequences) and entropy encoding. But I've never
found a compression library that treats those steps as separate
components. So I made one: github.com/andybalholm/pack. It defines
interfaces for the two stages, and includes a few example
implementations (Encoders for snappy, flate, and brotli; and
MatchFinders based on snappy and flate.BestSpeed).
Of course the abstraction of using these interfaces has a performance
penalty, but my flate.BestSpeed implementation is still faster than the
standard library flate package.
The biggest disadvantage of dividing things up like this is that it
prevents taking advantage of unique features of a compression format,
like brotli's static dictionary, or feedback from the entropy encoder to
the LZ77 stage like brotli and zopfli use at higher compression levels.
But by breaking the problem up into smaller pieces, it makes it much
easier to experiment with compression algorithms.
Andy
--
You received this message because you are subscribed to the Google Groups
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/golang-nuts/052bed1d-79f8-0910-7517-53e8cb04eced%40balholm.com.
