If you use a CompressedOutputStream, then you get a compressed (e.g. gzip) file.
If you want to use the Arrow IPC buffer compression, you need to specify in IpcWriteOptions.
Regards Antoine. Le 20/05/2021 à 18:45, Rares Vernica a écrit :
Hello, Just a clarifying question, when a CompressedOutputStream is used with RecordBatchStreamWriter, are the composing Arrow arrays compressed independently or the entire output file is compressed at once? For example if we use GZIP, is the resulting file a valid GZIP file that we can uncompress from the command line? Thanks! Rares
