scttfrdmn opened a new issue, #53138:
URL: https://github.com/apache/arrow/issues/53138
### Describe the bug
The R bindings do not expose `page_checksum_verification`, so an R user has
no way to turn on page-CRC verification when reading Parquet. Arrow's own
deliberately-corrupt test files are read without error.
C++ has the option and Python exposes it; only R does not.
| | |
|---|---|
| C++ | `cpp/src/parquet/properties.h:148-150` —
`page_checksum_verification()` / `set_page_checksum_verification(bool
check_crc)`, defaulting to `false` at line 166 |
| Python | `python/pyarrow/parquet/core.py:269,325,346` — a documented
`page_checksum_verification=False` parameter |
| **R** | `r/R/parquet.R:656-682` — `ParquetReaderProperties` declares and
implements **only** `thrift_string_size_limit` and
`thrift_container_size_limit` |
### Reproduction
Using the files `parquet-testing` ships expressly so readers can be shown to
reject a bad CRC — `datapage_v1-corrupt-checksum.parquet` ("uncompressed INT32
columns in v1 data pages with a mismatching CRC") and
`rle-dict-uncompressed-corrupt-checksum.parquet`:
```r
library(arrow)
packageVersion("arrow") # 25.0.0, libarrow 25.0.0
nrow(read_parquet("datapage_v1-corrupt-checksum.parquet")) # 5120
-- no error
nrow(read_parquet("rle-dict-uncompressed-corrupt-checksum.parquet")) # 1000
-- no error
ls(ParquetReaderProperties$create())
#> [1] "class_title" "clone" "initialize" "pointer" "print" "set_pointer"
#> [7] "set_thrift_container_size_limit" "set_thrift_string_size_limit"
#> [9] "thrift_container_size_limit" "thrift_string_size_limit"
# no page_checksum_verification
```
Measured on linux/aarch64, R 4.5.3, arrow 25.0.0 / libarrow 25.0.0, with
`parquet-testing` at `e74785d85a4e` (the `cpp/submodules/parquet-testing`
commit at `apache-arrow-25.0.0`).
### Why it matters
Reading without CRC verification is a reasonable *default* — it matches C++
and Python. The gap is that R offers no way to opt in, so an R pipeline cannot
detect page-level corruption even when it wants to. Silent corruption is the
failure mode page CRCs exist to prevent, and the files above exist to prove a
reader catches it.
It also means Arrow's own conformance corpus cannot be used from R for its
intended purpose: those two files are *supposed* to fail, and from R they are
indistinguishable from the matching-CRC files beside them
(`datapage_v1-uncompressed-checksum.parquet`,
`plain-dict-uncompressed-checksum.parquet`).
### Component(s)
R, Parquet
### Sketch of a fix
Not offering a PR, but the shape looks small — two accessors alongside the
existing thrift ones in `r/R/parquet.R`:
```r
page_checksum_verification = function() {
parquet___arrow___ReaderProperties__get_page_checksum_verification(self)
},
set_page_checksum_verification = function(check_crc) {
parquet___arrow___ReaderProperties__set_page_checksum_verification(self,
check_crc)
},
```
plus the two `cpp11` shims in `r/src/parquet.cpp` forwarding to the existing
C++ getter/setter, and a line each in the `@description` block and `NAMESPACE`.
Happy to be told if there is a reason it was left out deliberately — #36997
touched the same class for a different reason and did not add it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
