> > Interestingly arrow::Compression::type::LZ4 worked with streams in Arrow > 0.16.0, but triggers this NotImprelmented exception in Arrow 3.0.0. Maybe > the meaning has changed between versions.
Sorry for the late reply. yes, I think there have been a few rounds to disambiguate this frmo 0.16.0 through 3.0.0 On Tue, Jun 22, 2021 at 12:45 PM Rares Vernica <rvern...@gmail.com> wrote: > Thanks for the pointer. > > Interestingly arrow::Compression::type::LZ4 worked with streams in Arrow > 0.16.0, but triggers this NotImprelmented exception in Arrow 3.0.0. Maybe > the meaning has changed between versions. > > On Tue, Jun 22, 2021 at 12:33 PM Micah Kornfield <emkornfi...@gmail.com> > wrote: > > > It sounds like you are trying to use the buffer level compression > described > > by the specification? If so only LZ4_FRAME is currently allowed [1] > > > > [1] https://github.com/apache/arrow/blob/master/format/Message.fbs#L45 > > > > > > On Tue, Jun 22, 2021 at 12:28 PM Rares Vernica <rvern...@gmail.com> > wrote: > > > > > Hello, > > > > > > Using Arrow 3.0.0 I tried to compress a stream with LZ4 and got this > > error > > > message: > > > > > > NotImplemented: Streaming compression unsupported with LZ4 raw format. > > Try > > > using LZ4 frame format instead. > > > > > > Is it because LZ4 raw was not enabled when the .so was compiled or > > actually > > > not implemented? > > > > > > Is LZ4 frame the recommended format for compressing stream/IPC buffers? > > > > > > Thank you! > > > Rares > > > > > >
