Wes McKinney created ARROW-300: ---------------------------------- Summary: [Format] Add buffer compression option to IPC file format Key: ARROW-300 URL: https://issues.apache.org/jira/browse/ARROW-300 Project: Apache Arrow Issue Type: New Feature Components: Format Reporter: Wes McKinney
It may be useful if data is to be sent over the wire to compress the data buffers themselves as their being written in the file layout. I would propose that we keep this extremely simple with a global buffer compression setting in the file Footer. Probably only two compressors worth supporting out of the box would be zlib (higher compression ratios) and lz4 (better performance). What does everyone think? -- This message was sent by Atlassian JIRA (v6.3.4#6332)