Shilun Fan created HUDI-6451:
--------------------------------
Summary: Randomly obtain a path in
HoodieMemoryConfig#getDefaultSpillableMapBasePath
Key: HUDI-6451
URL: https://issues.apache.org/jira/browse/HUDI-6451
Project: Apache Hudi
Issue Type: Improvement
Components: cli
Reporter: Shilun Fan
The HoodieMemoryConfig#getDefaultSpillableMapBasePath method retrieves a path
from YARN's LOCAL_DIRS environment variable. However, the order of LOCAL_DIRS
concatenation in YARN is typically fixed, resulting in the
DefaultSpillableMapBasePath being a fixed value. Considering the disk load
perspective, we should randomize the selection of a path from the array.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
