alamb opened a new issue, #7546: URL: https://github.com/apache/arrow-datafusion/issues/7546
### Describe the bug After https://github.com/apache/arrow-datafusion/pull/7510, we saw intermittent errors in a test reading spill files: https://github.com/apache/arrow-datafusion/issues/7537 and https://github.com/apache/arrow-datafusion/issues/7523 Some great debugging by @sarutak and @viirya (see description on https://github.com/apache/arrow-datafusion/pull/7538#issue-1893761145) found that the `NamedTempFiles` returned by [`DiskManager::create_tmp_file`](https://docs.rs/datafusion/latest/datafusion/execution/struct.DiskManager.html#method.create_tmp_file) can be deleted too soon, if the DiskManager is dropped while the plan is still outstanding ### To Reproduce Run a plan and drop the disk manager ### Expected behavior It should not be possible to have the NamedtempFiles deleted out from under a running plan. There are a bunch of various proposals on https://github.com/apache/arrow-datafusion/pull/7538 ### Additional context _No response_ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
