[
https://issues.apache.org/jira/browse/ARROW-16421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17531338#comment-17531338
]
Weston Pace commented on ARROW-16421:
-------------------------------------
The scanner does need to close its files. It takes care of this itself as it
finishes scanning. One issue we have today (which may be related) is that the
scanner reports that it is finished (and the exec plan too if you are using
one) before it has completely finished all close/cleanup tasks so there is some
possibility that the close happens after the scanner is finished. That
behavior will change in a month or two as part of ARROW-15732
> [R] Permission error on Windows when deleting file in dataset
> -------------------------------------------------------------
>
> Key: ARROW-16421
> URL: https://issues.apache.org/jira/browse/ARROW-16421
> Project: Apache Arrow
> Issue Type: Improvement
> Components: R
> Affects Versions: 7.0.0
> Reporter: Will Jones
> Assignee: Will Jones
> Priority: Major
>
> On Windows this fails:
> {code:R}
> library(arrow)
> write_dataset(iris, "test_dataset")
> # Original example was with DuckDB, but that's not necessarily the issue
> # con <- open_dataset("test_dataset") |> to_duckdb()
> con <- open_dataset("test_dataset")$NewScan()$Finish()$ToRecordBatchReader()
> file.remove("test_dataset/part-0.parquet")
> #> Warning in file.remove("test_dataset/part-0.parquet"): cannot remove file
> #> 'test_dataset/part-0.parquet', reason 'Permission denied'
> #> [1] FALSE
> {code}
> But on MacOS it does not:
> {code:r}
> library(arrow)
> write_dataset(iris, "test_dataset")
> # Original example was with DuckDB, but that's not necessarily the issue
> # con <- open_dataset("test_dataset") |> to_duckdb()
> con <- open_dataset("test_dataset")$NewScan()$Finish()$ToRecordBatchReader()
> file.remove("test_dataset/part-0.parquet")
> #> [1] TRUE
> {code}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
