Hello Impala Public Jenkins, I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/17171 to look at the new patch set (#3). Change subject: IMPALA-10579: Fix usage of RemoteIterator in FileSystemUtil ...................................................................... IMPALA-10579: Fix usage of RemoteIterator in FileSystemUtil HDFS FileSystem provides a listStatusIterator() API for listing remote storage using a RemoteIterator. We use it to list files when loading table file metadata. It's not guaranteed that a RemoteIterator can survive when its hasNext() or next() throws IOExceptions. We should stop the loop in this case. Otherwise, we may go into a dead loop. Without HADOOP-16685, it's also not guaranteed that FileSystem.listStatusIterator() will thrown a FileNotFoundException when the path doesn't exist. This patch refactors the file listing iterators so we don't need to depend on these two assumptions. Tests: - Loop test_insert_stress.py 100 times. Verified the non-existing subdirs are skipped and inserts are stable in a high concurrency. Change-Id: I859bd4f976c51a34eb6a03cefd2ddcdf11656cea --- M fe/src/main/java/org/apache/impala/common/FileSystemUtil.java 1 file changed, 100 insertions(+), 31 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/71/17171/3 -- To view, visit http://gerrit.cloudera.org:8080/17171 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I859bd4f976c51a34eb6a03cefd2ddcdf11656cea Gerrit-Change-Number: 17171 Gerrit-PatchSet: 3 Gerrit-Owner: Quanlong Huang <huangquanl...@gmail.com> Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com>