thisisnic commented on code in PR #51662:
URL: https://github.com/apache/arrow/pull/51662#discussion_r4242421569
##########
r/R/csv.R:
##########
@@ -722,6 +722,10 @@ readr_to_csv_read_options <- function(skip = 0, col_names
= TRUE) {
#' and LF (`0x0a`) characters?
#' @param ignore_empty_lines Logical: should empty lines be ignored (default)
or
#' generate a row of missing values (if `FALSE`)?
+#' @param pad_short_rows Logical: should rows with fewer columns than expected
be
+#' padded with missing values (default `FALSE`) or raise an error?
+#' @param ignore_extra_columns Logical: should columns beyond the number
expected be
+#' ignored (default `FALSE`) or raise an error?
Review Comment:
Please can you regenerate the docs so that the corresponding `.Rd` files are
regenerated? You can either run `devtools::document()` from the `arrow/r`
directory, or run `make doc` from that same place.
##########
r/tests/testthat/test-csv.R:
##########
@@ -691,6 +691,54 @@ test_that("CSV reading/parsing/convert options can be
passed in as lists", {
expect_equal(tab1, tab2)
})
+test_that("pad_short_rows and ignore_extra_columns parse options", {
+ # Rows with fewer columns than expected are padded with nulls (GH-50925), and
+ # extra columns are dropped (GH-50967).
+ short <- I("a,b,c\n1,2,9\n3")
+ expect_error(read_csv_arrow(short))
+ expect_identical(
+ read_csv_arrow(short, parse_options = csv_parse_options(pad_short_rows =
TRUE)),
+ tibble::tibble(a = c(1L, 3L), b = c(2L, NA), c = c(9L, NA))
+ )
+
+ extra <- I("a,b\n1,2,3\n4,5")
+ expect_error(read_csv_arrow(extra))
Review Comment:
When we use `expect_error()` it can be helpful to use the `regexp` parameter
to specify a regular expression to make sure the error is the expected one.
That said though, I think it's totally fine here to drop to contrast between
the error without the extra option and successful running with the option
specified, and just include the successful running of the code.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
