I was hoping that someone in the community would provide some alternatives since there are quite a few implementations.
On Tue, Dec 3, 2019 at 8:20 AM Amogh Tiwari <[email protected]> wrote: > Hi Luke, > > I agree with your thoughts and observations. But, airlift:aircompressor is > the only implementation of LZO in pure java. That straight away solves #5. > The other implementations that I found either have licensing issues (since > LZO natively uses GNU GPL licence) or are implemented using .c, .h and jni > (which again make them dependent on the OS). Please refer these: > twitter/hadoop-lzo > <https://github.com/twitter/hadoop-lzo/tree/master/src/main/java/com/hadoop/compression/lzo> > and > shevek/lzo-java > <https://github.com/shevek/lzo-java/tree/master/lzo-core/src/main/java/org/anarres/lzo> > . > These were the main reasons why we based this on airlift:aircompressor. > > Thanks and Regards, > Amogh > > > > On Tue, Dec 3, 2019 at 2:59 AM Luke Cwik <[email protected]> wrote: > >> I took a look. My biggest concern is finding a good LZO implementation. >> Looking for one that preferably has: >> 1) Apache license >> 2) Has zero transitive dependencies >> 3) Is small >> 4) Is performant >> 5) Is native java or supports execution on the three main OSs (Windows, >> Linux, Mac) >> >> In your PR you suggested using io.airlift:aircompressor:0.16 which >> doesn't meet item #2 and its transitive dependency fails #3. >> >> On Mon, Dec 2, 2019 at 12:16 PM Amogh Tiwari <[email protected]> wrote: >> >>> Hi, >>> I have filed a PR for an extension that will enable Apache Beam to work >>> with LZO/LZOP compression. Please refer >>> <https://issues.apache.org/jira/browse/BEAM-8564>. >>> I would love it if someone can take this up and review it. >>> Please feel free to share your thoughts/suggestions. >>> Regards, >>> Amogh >>> >>
