Quanlong Huang has uploaded this change for review. ( http://gerrit.cloudera.org:8080/21680
Change subject: IMPALA-13303: FileSystemUtil.listFiles() should handle non-recursive case ...................................................................... IMPALA-13303: FileSystemUtil.listFiles() should handle non-recursive case FileSystemUtil.listFiles() is used in FileMetadataLoader#loadInternal() to list the files with block locations. When table property "impala.disable.recursive.listing" is set to true, it's supposed to skip files in the sub dirs. However, for FileSystems that don't support recursive listFiles(), we always create a RecursingIterator and don't respect the 'recursive' argument. This patch fixes the issue by adding the check for the 'recursive' argument and use the non-recursive iterator when it's false. Tests - Add test in test_recursive_listing.py to reveal the issue Change-Id: Ia930e6071963d53561ce79896bff9d19720468a4 --- M fe/src/main/java/org/apache/impala/common/FileSystemUtil.java M tests/metadata/test_recursive_listing.py 2 files changed, 7 insertions(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/80/21680/1 -- To view, visit http://gerrit.cloudera.org:8080/21680 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Ia930e6071963d53561ce79896bff9d19720468a4 Gerrit-Change-Number: 21680 Gerrit-PatchSet: 1 Gerrit-Owner: Quanlong Huang <[email protected]>
