Zoltan Borok-Nagy has uploaded this change for review. ( http://gerrit.cloudera.org:8080/17316
Change subject: IMPALA-10658: LOAD DATA INPATH silently fails between HDFS and Azure ABFS ...................................................................... IMPALA-10658: LOAD DATA INPATH silently fails between HDFS and Azure ABFS LOAD DATA INPATH silently fails when Impala tries to move files from HDFS to ABFS. The problem is that we use FileSystem.makeQualified(Path) to decide if path is on a given filesystem. We expect to get an IllegalArgumentException if path is on a different filesystem. However, the Azure FileSystem implementation doesn't throw this exception. Because of that Impala thinks that an 'hdfs://' path and an 'abfs://' path is on the same filesystem, so it tries to move files with FileSystem.rename(). In case of errors rename() might throw an Exception, or return false. Impala doesn't check the return value, therefore if rename() returns false then the error remains silent. This patch fixes Impala's isPathOnFileSystem() and also adds a check for the return value of rename(). Testing: * tested manually between HDFS and Azure ABFS. Change-Id: Id807e8a200b83283a09d3a917185cabab930017d --- M fe/src/main/java/org/apache/impala/common/FileSystemUtil.java 1 file changed, 8 insertions(+), 6 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/16/17316/1 -- To view, visit http://gerrit.cloudera.org:8080/17316 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Id807e8a200b83283a09d3a917185cabab930017d Gerrit-Change-Number: 17316 Gerrit-PatchSet: 1 Gerrit-Owner: Zoltan Borok-Nagy <[email protected]>
