hi David, You don't want to pass the compression option to TableSaver.new -- compression is something that's configured in the Parquet writer. This would need to be an option on save_as_parquet, but it doesn't look like it is exposed right now
https://github.com/apache/arrow/blob/master/ruby/red-parquet/lib/parquet/arrow-table-savable.rb#L21 It's available in GLib though so this could be added to the Ruby library https://github.com/apache/arrow/blob/master/c_glib/parquet-glib/arrow-file-writer.h - Wes On Thu, Apr 23, 2020 at 3:13 PM David Lahn <[email protected]> wrote: > > Hi, > > > > Does anyone have any examples of how to output a Parquet file with Snappy > compression using the Ruby gem? > > > > We have tested trying to set compression to “snappy” on the TableSaver, but > we get the following: > > > > [compressed-output-stream][new]: NotImplemented: Streaming compression > unsupported with Snappy (Arrow::Error::NotImplemented) > > > > Example: > > > > Arrow::TableSaver.new(table, 'test.parquet', {compression: 'snappy'}).save > > > > Or are we completely turned around on how to accomplish this? > > > > Dave > > > David Lahn > DevOps Lead > Development > > ForwardPMX > Privacy Policy > > e: [email protected] > d: +44 (0)203 476 3725 (main office number) > m: +1 519 573 1624 > > > This e-mail is confidential to ForwardPMX intended for use by the recipient. > If you received this in error or are not the intended recipient, you are > hereby notified that any review, retransmission, copying or other use of, or > taking of any action in reliance upon this information is strictly prohibited. >
