Yida Wu has uploaded a new patch set (#2). ( http://gerrit.cloudera.org:8080/17901 )
Change subject: IMPALA-10945: Fix S3 scratch path behavior ...................................................................... IMPALA-10945: Fix S3 scratch path behavior IMPALA-10429 "Support Spill to HDFS" introduces a new behavior to S3 scratch path. It added a path verification for the S3 path, however, the HdfsFsCache::GetNameNodeFromPath in the verification forces the input to have at least a directory after the authority, like "s3a://authority/dir", otherwise it will return an error and lead to a failure on the TmpFileMgr initialization. Therefore, it changes the previous behavior which was able to support "s3a://authority", and may affect current users. This patch resumes the behavior of the s3 scratch path to allow "s3a://authority". The solution is to pass the path of the user's input combined with a scratch suffix "/impala-scratch" to the verification function, therefore, at least one directory is contained in the path. Tests: Ran core tests. Added logic to run two types of path format in TmpFileMgrTest: "s3a://authority" and "s3a://authority/dir". Change-Id: I028f375b9f535f8641261cc02f921497e076aa9b --- M be/src/runtime/tmp-file-mgr-test.cc M be/src/runtime/tmp-file-mgr.cc 2 files changed, 85 insertions(+), 78 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/01/17901/2 -- To view, visit http://gerrit.cloudera.org:8080/17901 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I028f375b9f535f8641261cc02f921497e076aa9b Gerrit-Change-Number: 17901 Gerrit-PatchSet: 2 Gerrit-Owner: Yida Wu <[email protected]> Gerrit-Reviewer: Abhishek Rawat <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]>
