NedJWestern opened a new issue, #37798:
URL: https://github.com/apache/arrow/issues/37798
### Describe the bug, including details regarding any error messages,
version, and platform.
The R session will hang upon failure. This issue appears to be intermittent,
it fails around 1 in 10 times with the current settings.
Windows Server 2019
RStudio 2023.03.1
R 4.1.3
Arrow 13.0.0
dplyr 1.1.2
Here is the minimum working example, adjust `vec_length` to refine the
failure rate:
```
# create synthetic dataframe of integers (larger dfs are more likely to fail)
vec_length <- 10000000
test_data <- data.frame(test_values = sample(0:10, vec_length, replace =
TRUE))
# path to temporarily store the test_dataset
test_dataset_path <- file.path(getwd(), "test_data")
# write to parquet arrow dataset (single file)
arrow::write_dataset(
dataset = test_data,
path = test_dataset_path,
format = "parquet"
)
# attempt to summarise the arrow dataset
for (i in 1:100) {
print(paste("Attempt", i, "to read and summarise test arrow dataset..."))
largest_value <-
arrow::open_dataset(test_dataset_path) |>
dplyr::summarise(max_value = max(test_values)) |>
dplyr::collect()
print("Success.")
}
# remove test data
unlink(test_dataset_path)
```
### Component(s)
R
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
