> On Jan. 10, 2017, 4:42 p.m., Jonathan Hurley wrote: > > ambari-common/src/main/python/resource_management/libraries/providers/hdfs_resource.py, > > line 168 > > <https://reviews.apache.org/r/54874/diff/1/?file=1589085#file1589085line168> > > > > Does thie curl command work when Kerberos / SSL is being used? Should > > you instead use curl_krb_request? > > Tim Thorpe wrote: > This is the same curl command which has been used for creating files in > HDFS, upload, querying the file status, etc... > I would assume this has been tested many times with Kerberos and SSL. I > don't see why changing the URL to be an OPEN operation would impact that but > I can test Kerberos/SSL. > > Jonathan Hurley wrote: > Great - it might not be an issue, but I thought I'd check. If you can > confirm it works with Kerberos, then you can drop the issue. > > Tim Thorpe wrote: > under def action_execute(self, main_resource), it runs the kinit: > if security_enabled: > main_resource.kinit() > This avoids doing a kinit for each call to HDFS. The HDFSResource is > gathering a queue of actions to be performed to avoid all the expensive > operations of opening connections. Using curl_krb_request would defeat the > point. > > The actual curl command handles the rest of the security in the > run_command function (where the curl command you listed is found): > if self.security_enabled: > cmd += ["--negotiate", "-u", ":"] > if self.is_https_enabled: > cmd += ["-k"]
Yep - the -k and --negotiate params were what I didn't see. Thanks! - Jonathan ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/54874/#review161127 ----------------------------------------------------------- On Dec. 19, 2016, 4:24 p.m., Tim Thorpe wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/54874/ > ----------------------------------------------------------- > > (Updated Dec. 19, 2016, 4:24 p.m.) > > > Review request for Ambari, Alejandro Fernandez, Andrew Onischuk, Di Li, > Jonathan Hurley, Jayush Luniya, and Sumit Mohanty. > > > Bugs: AMBARI-19241 > https://issues.apache.org/jira/browse/AMBARI-19241 > > > Repository: ambari > > > Description > ------- > > Currently the hdfs_resources.py supports > 1) create files/directories in HDFS > 2) delete files/directories in HDFS > 3) upload files/directories to HDFS > > We should also support download of files/directories from HDFS. This will > help particularly in cloud environments where the users of the cluster don't > necessarily have write access to the local file system. > > > Diffs > ----- > > > ambari-common/src/main/python/resource_management/libraries/functions/download_from_hdfs.py > PRE-CREATION > > ambari-common/src/main/python/resource_management/libraries/providers/hdfs_resource.py > f1aa3e1 > > ambari-common/src/main/python/resource_management/libraries/resources/hdfs_resource.py > 5761fd6 > > contrib/fast-hdfs-resource/src/main/java/org/apache/ambari/fast_hdfs_resource/Resource.java > da5a706 > > contrib/fast-hdfs-resource/src/main/java/org/apache/ambari/fast_hdfs_resource/Runner.java > e4656c7 > > Diff: https://reviews.apache.org/r/54874/diff/ > > > Testing > ------- > > Manual testing with and without web hdfs. Tested download files and > directories both when they existed locally and when they didn't. > > > Thanks, > > Tim Thorpe > >
