Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/24588 )
Change subject: IMPALA-15117: Optimize getFirstLevelAcidDirPath ...................................................................... IMPALA-15117: Optimize getFirstLevelAcidDirPath In the original implementation of AcidUtils.getFirstLevelAcidDirPath(), the FileSystem.isDirectory() API was called recursively for each path component. This introduced unnecessary RPC overhead during table loading. This change refactors the logic to leverage Path string operations by traversing upwards from the file's parent directory to detect the ACID directory structure. 1. Eliminated FileSystem.isDirectory() RPCs: Since the input filePath is guaranteed to be a valid data file during metadata load, we can assume that all of its parent paths are directories 2. Preserved backward compatibility: The original recursive behavior handled nested subdirectories inside ACID folders. To maintain the same behavior, this change preserves the exact same path-merging logic (e.g., returning "delta_000001_000001_0000/sub_dir" for a file path like "partition_dir/delta_000001_000001_0000/sub_dir/filename") Testing: - Passed core tests. - Added AcidUtilsTest.testGetFirstLevelAcidDirPath. Change-Id: I390183039181bb62d6818c8e869738ddf01c2f6b Reviewed-on: http://gerrit.cloudera.org:8080/24588 Reviewed-by: Impala Public Jenkins <[email protected]> Tested-by: Impala Public Jenkins <[email protected]> --- M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java M fe/src/main/java/org/apache/impala/util/AcidUtils.java M fe/src/test/java/org/apache/impala/util/AcidUtilsTest.java 3 files changed, 81 insertions(+), 38 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/24588 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I390183039181bb62d6818c8e869738ddf01c2f6b Gerrit-Change-Number: 24588 Gerrit-PatchSet: 7 Gerrit-Owner: Jiyoung Yoo <[email protected]> Gerrit-Reviewer: Csaba Ringhofer <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Jiyoung Yoo <[email protected]> Gerrit-Reviewer: Michael Smith <[email protected]> Gerrit-Reviewer: Sai Hemanth Gantasala <[email protected]> Gerrit-Reviewer: Yida Wu <[email protected]>
