[
https://issues.apache.org/jira/browse/IMPALA-12971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yida Wu updated IMPALA-12971:
-----------------------------
Epic Link: IMPALA-9828
> Incorrect capacity of remote scratch dir when both local and remote scratch
> dir are configured
> ----------------------------------------------------------------------------------------------
>
> Key: IMPALA-12971
> URL: https://issues.apache.org/jira/browse/IMPALA-12971
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
> Reporter: Yida Wu
> Assignee: Yida Wu
> Priority: Major
>
> The current implementation in AllocateRemoteSpace()
> (https://github.com/apache/impala/blob/4be5fd8896dcd445a6379bdcda4bdcf318f24511/be/src/runtime/tmp-file-mgr.cc#L1391)
> has a bug when both local and remote scratch directories are configured.
> Currently, the function utilizes current_bytes_allocated_ to determine
> whether to allocate space based on the remote directory limit (capacity).
> However, this approach is flawed when the system allows spilling to other
> local directories, as current_bytes_allocated_ represents the total used
> scratch space rather than the usage of the remote directory alone,
> potentially causing allocation failures when the combined usage (local +
> remote) exceeds the remote limit.
> This bug happens when both remote and local scratch space are configured, and
> a limit is set to the remote directory (normally remote capacity is set to
> unlimited)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
