nealrichardson commented on a change in pull request #9685:
URL: https://github.com/apache/arrow/pull/9685#discussion_r593318541
##########
File path: r/tests/testthat/test-dataset.R
##########
@@ -295,6 +295,29 @@ test_that("CSV dataset", {
)
})
+test_that("compressed CSV dataset", {
+ skip_if_not_available("gzip")
+ dst_dir <- make_temp_dir()
+ dst_file <- file.path(dst_dir, "data.csv.gz")
+ write.csv(df1, gzfile(dst_file), row.names = FALSE, quote = FALSE)
+ format <- FileFormat$create("csv", compression = CompressionType$GZIP)
Review comment:
With the above suggestion, this becomes:
```suggestion
format <- FileFormat$create("csv", compression = "gzip")
```
##########
File path: r/R/dataset-format.R
##########
@@ -101,8 +101,10 @@ IpcFileFormat <- R6Class("IpcFileFormat", inherit =
FileFormat)
#' @rdname FileFormat
#' @export
CsvFileFormat <- R6Class("CsvFileFormat", inherit = FileFormat)
-CsvFileFormat$create <- function(..., opts =
csv_file_format_parse_options(...)) {
- dataset___CsvFileFormat__Make(opts)
+CsvFileFormat$create <- function(...,
+ opts = csv_file_format_parse_options(...),
+ compression = CompressionType$UNCOMPRESSED) {
+ dataset___CsvFileFormat__Make(opts, compression)
Review comment:
We have a helper used in a few places that makes this a little more
human-friendly:
```suggestion
compression = "uncompressed") {
dataset___CsvFileFormat__Make(opts, compression_from_name(compression))
```
Should also add a note in the FileFormat$create docs above (around L48) that
compression is an option now.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
