Yida Wu has uploaded this change for review. ( http://gerrit.cloudera.org:8080/17901
Change subject: IMPALA-10945: Fix S3 scratch path behavior ...................................................................... IMPALA-10945: Fix S3 scratch path behavior IMPALA-10429 "Support Spill to HDFS" introduces a new behavior to S3 scratch path. It added a path verification for the S3 path, however, the HdfsFsCache::GetNameNodeFromPath in the verification forces the input to have at least a directory after the authority, like "s3a://authority/dir", otherwise it will return an error and lead to a failure on the TmpFileMgr initialization. Therefore, it changes the previous behavior which was able to support "s3a://authority", and may affect current users. This patch resumes the behavior of the s3 scratch path to allow "s3a://authority". The solution is to pass the path of the user's input combined with a scratch suffix "/impala-scratch" to the verification function, therefore, at least one directory is contained in the path. Tests: Ran core tests. Added logic to run two types of path format in TmpFileMgrTest: "s3a://authority" and "s3a://authority/dir". Change-Id: I028f375b9f535f8641261cc02f921497e076aa9b --- M be/src/runtime/tmp-file-mgr-test.cc M be/src/runtime/tmp-file-mgr.cc 2 files changed, 83 insertions(+), 76 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/01/17901/1 -- To view, visit http://gerrit.cloudera.org:8080/17901 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I028f375b9f535f8641261cc02f921497e076aa9b Gerrit-Change-Number: 17901 Gerrit-PatchSet: 1 Gerrit-Owner: Yida Wu <[email protected]>
