anujphadke has posted comments on this change. Change subject: IMPALA-1850: Allow fs.defaultFS to be set to a non-HDFS filesystem ......................................................................
Patch Set 20: (1 comment) http://gerrit.cloudera.org:8080/#/c/1121/20/be/src/util/hdfs-util.cc File be/src/util/hdfs-util.cc: Line 75: return strncmp(path, "s3a://", 6) == 0; > shouldn't we do the same default_fs check here? how are both of these case Yes, I will make the change here too. I am not sure how to create a setup where I can generate different combinations of qualified and unqualified paths for s3 and hdfs to verify the API change So I hardcoded different combinations of FS paths and set the configured_defaultFs to s3a/hdfs and verified this I0420 12:29:25.167100 25288 exec-env.cc:425] Configured defaultFS: hdfs://localhost:20500 I0420 12:29:25.167109 25288 exec-env.cc:426] hdfs://test : 1 I0420 12:29:25.167112 25288 exec-env.cc:427] test : 1 I0420 12:29:25.167114 25288 exec-env.cc:428] testFile : 1 I0420 12:29:25.167115 25288 exec-env.cc:429] s3a://test : 0 I0420 12:29:25.167116 25288 exec-env.cc:431] Configured defaultFS: s3a:// I0420 12:29:25.167117 25288 exec-env.cc:432] hdfs://test : 1 I0420 12:29:25.167119 25288 exec-env.cc:433] test : 0 I0420 12:29:25.167120 25288 exec-env.cc:434] testFile : 0 I0420 12:29:25.167121 25288 exec-env.cc:435] s3a://test : 0. On my private build before running the tests , I also added logs where these functions were invoked to check the return value. It worked fine since for s3 the paths were always appended with the s3a:// -- To view, visit http://gerrit.cloudera.org:8080/1121 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I2f45bef6c94ece634045acb906d12591587ccfed Gerrit-PatchSet: 20 Gerrit-Project: Impala Gerrit-Branch: cdh5-trunk Gerrit-Owner: anujphadke <[email protected]> Gerrit-Reviewer: Alex Behm <[email protected]> Gerrit-Reviewer: Casey Ching <[email protected]> Gerrit-Reviewer: Dan Hecht <[email protected]> Gerrit-Reviewer: Henry Robinson <[email protected]> Gerrit-Reviewer: Juan Yu <[email protected]> Gerrit-Reviewer: Sailesh Mukil <[email protected]> Gerrit-Reviewer: anujphadke <[email protected]> Gerrit-HasComments: Yes
