Vihang Karajgaonkar has posted comments on this change. ( http://gerrit.cloudera.org:8080/13665 )
Change subject: IMPALA-8663 : FileMetadataLoader should skip hidden and tmp directories ...................................................................... Patch Set 8: (1 comment) http://gerrit.cloudera.org:8080/#/c/13665/8/fe/src/main/java/org/apache/impala/common/FileSystemUtil.java File fe/src/main/java/org/apache/impala/common/FileSystemUtil.java: http://gerrit.cloudera.org:8080/#/c/13665/8/fe/src/main/java/org/apache/impala/common/FileSystemUtil.java@545 PS8, Line 545: all underlying files (except which are : * in the ignored directories) > hrm, were the semantics already broken then? See the unit test AcidUtilsTes yeah, it looks like its broken currently for this case. I will file a JIRA. The AcidUtils test works because it works off the List<FileStatus> not the ones which are actually returned by the FileMetadataLoader. Specifically, if the table is transactional we cannot rely on using listFiles API since it will skip empty directories (Or we should implement a version of it for own purposes). Not sure how to handle the S3 recursive listFiles case though. Created https://issues.apache.org/jira/browse/IMPALA-8739 as suggested -- To view, visit http://gerrit.cloudera.org:8080/13665 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I2c4a22908304fe9e377d77d6c18d401c3f3294aa Gerrit-Change-Number: 13665 Gerrit-PatchSet: 8 Gerrit-Owner: Vihang Karajgaonkar <[email protected]> Gerrit-Reviewer: Bharath Vissapragada <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Todd Lipcon <[email protected]> Gerrit-Reviewer: Vihang Karajgaonkar <[email protected]> Gerrit-Comment-Date: Wed, 03 Jul 2019 20:45:32 +0000 Gerrit-HasComments: Yes
