etiennebacher opened a new issue, #40231: URL: https://github.com/apache/arrow/issues/40231
### Describe the bug, including details regarding any error messages, version, and platform. `arrow` produces invalid `arrow_table` when it is called in a `multisession` plan created by [`future`](https://www.futureverse.org/) (or its related packages). In the examples below, `future_lapply()` fails when the plan is `multisession` but not when the plan is `sequential`. I added a simple `lapply()` call to show that it works normally and that the problem really comes from `future_lapply()`. * `plan(sequential)` ``` r library(arrow, warn.conflicts = FALSE) library(future.apply) #> Loading required package: future plan(sequential) lapply(1, \(x) as_arrow_table(iris)) #> [[1]] #> Table #> 150 rows x 5 columns #> $Sepal.Length <double> #> $Sepal.Width <double> #> $Petal.Length <double> #> $Petal.Width <double> #> $Species <dictionary<values=string, indices=int8>> #> #> See $metadata for additional Schema metadata future_lapply(1, \(x) as_arrow_table(iris)) #> [[1]] #> Table #> 150 rows x 5 columns #> $Sepal.Length <double> #> $Sepal.Width <double> #> $Petal.Length <double> #> $Petal.Width <double> #> $Species <dictionary<values=string, indices=int8>> #> #> See $metadata for additional Schema metadata ``` * `plan(multisession)` ``` r library(arrow, warn.conflicts = FALSE) library(future.apply) #> Loading required package: future plan(multisession) lapply(1, \(x) as_arrow_table(iris)) #> [[1]] #> Table #> 150 rows x 5 columns #> $Sepal.Length <double> #> $Sepal.Width <double> #> $Petal.Length <double> #> $Petal.Width <double> #> $Species <dictionary<values=string, indices=int8>> #> #> See $metadata for additional Schema metadata future_lapply(1, \(x) as_arrow_table(iris)) #> [[1]] #> Table #> Error: Invalid <Table>, external pointer to null ``` --- This failure with `future_lapply()` was originally found in `polars`: https://github.com/pola-rs/r-polars/issues/851 cc @eitsupi ### Component(s) R -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
