alamb opened a new pull request #1668: URL: https://github.com/apache/arrow-datafusion/pull/1668
# Which issue does this PR close? Resolves https://github.com/apache/arrow-datafusion/issues/1636 # Rationale for this change Creation of an `RuntimeEnv` creates a temporary directory, even when it is not used. This is unecessary per-query overhead Also, since there is no way to share `MemoryManger` and `DiskManager`s across `RuntimeEnv's` it is not possible to get a "global" view of memory / disk use across multiple queries running concurrently. Also, certain invalid memory configuration values will panic, rather than error when set incorrectly # What changes are included in this PR? Changes: 1. Add `DiskManagerConfig` and `MemoryManagerConfig` for configuring how disk and memory are managed 2. Adds proper error checking (rather than asserts/ errors) to ensure reasonable values for memory config 2. Allow re-using existing `MemoryManager` and `DiskManager` rather than always creating them new # Are there any user-facing changes? Will be a change to anyone who was using the RuntimeConfig but since it was introduced recently I don't think that is very many -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
