alamb opened a new pull request #1695: URL: https://github.com/apache/arrow-datafusion/pull/1695
# Which issue does this PR close? Related https://github.com/apache/arrow-datafusion/issues/1690 First part of https://github.com/apache/arrow-datafusion/issues/1690: only do IO / `TempDir` creation when tempfiles are actually needed in a plan I plan a second PR to avoid creating so many `DiskManager` instances in the first place # Rationale for this change Creating temp files is expensive and now DataFusion is doing it frequently during processing (see https://github.com/apache/arrow-datafusion/issues/1690 for more backstory) # What changes are included in this PR? Changes; If the user doesn't specify explicit temp directories, do not create a system assigned tempfile unless it is actually requested # Are there any user-facing changes? less tempfiles! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
