Carl Boettiger created ARROW-16144:
--------------------------------------
Summary: Write compressed data streams (particularly over S3)
Key: ARROW-16144
URL: https://issues.apache.org/jira/browse/ARROW-16144
Project: Apache Arrow
Issue Type: Improvement
Components: R
Affects Versions: 7.0.0
Reporter: Carl Boettiger
The python bindings have `CompressedOutputStream`, but I don't see how we can
do this on the R side (e.g. with `write_csv_arrow()`). It would be wonderful
if we could both read and write compressed streams, particularly for CSV and
particularly for remote filesystems, where this can provide considerable
performance improvements.
(For comparison, readr will write a compressed stream automatically based on
the extension for the given filename, e.g. `readr::write_csv(data,
"file.csv.gz")` or `write_csv("data.file.xz")` )
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
