> On Jan. 10, 2017, 9:42 p.m., Jonathan Hurley wrote:
> > ambari-common/src/main/python/resource_management/libraries/providers/hdfs_resource.py,
> > line 168
> > <https://reviews.apache.org/r/54874/diff/1/?file=1589085#file1589085line168>
> >
> > Does thie curl command work when Kerberos / SSL is being used? Should
> > you instead use curl_krb_request?
>
> Tim Thorpe wrote:
> This is the same curl command which has been used for creating files in
> HDFS, upload, querying the file status, etc...
> I would assume this has been tested many times with Kerberos and SSL. I
> don't see why changing the URL to be an OPEN operation would impact that but
> I can test Kerberos/SSL.
>
> Jonathan Hurley wrote:
> Great - it might not be an issue, but I thought I'd check. If you can
> confirm it works with Kerberos, then you can drop the issue.
under def action_execute(self, main_resource), it runs the kinit:
if security_enabled:
main_resource.kinit()
This avoids doing a kinit for each call to HDFS. The HDFSResource is gathering
a queue of actions to be performed to avoid all the expensive operations of
opening connections. Using curl_krb_request would defeat the point.
The actual curl command handles the rest of the security in the run_command
function (where the curl command you listed is found):
if self.security_enabled:
cmd += ["--negotiate", "-u", ":"]
if self.is_https_enabled:
cmd += ["-k"]
- Tim
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54874/#review161127
-----------------------------------------------------------
On Dec. 19, 2016, 9:24 p.m., Tim Thorpe wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54874/
> -----------------------------------------------------------
>
> (Updated Dec. 19, 2016, 9:24 p.m.)
>
>
> Review request for Ambari, Alejandro Fernandez, Andrew Onischuk, Di Li,
> Jonathan Hurley, Jayush Luniya, and Sumit Mohanty.
>
>
> Bugs: AMBARI-19241
> https://issues.apache.org/jira/browse/AMBARI-19241
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Currently the hdfs_resources.py supports
> 1) create files/directories in HDFS
> 2) delete files/directories in HDFS
> 3) upload files/directories to HDFS
>
> We should also support download of files/directories from HDFS. This will
> help particularly in cloud environments where the users of the cluster don't
> necessarily have write access to the local file system.
>
>
> Diffs
> -----
>
>
> ambari-common/src/main/python/resource_management/libraries/functions/download_from_hdfs.py
> PRE-CREATION
>
> ambari-common/src/main/python/resource_management/libraries/providers/hdfs_resource.py
> f1aa3e1
>
> ambari-common/src/main/python/resource_management/libraries/resources/hdfs_resource.py
> 5761fd6
>
> contrib/fast-hdfs-resource/src/main/java/org/apache/ambari/fast_hdfs_resource/Resource.java
> da5a706
>
> contrib/fast-hdfs-resource/src/main/java/org/apache/ambari/fast_hdfs_resource/Runner.java
> e4656c7
>
> Diff: https://reviews.apache.org/r/54874/diff/
>
>
> Testing
> -------
>
> Manual testing with and without web hdfs. Tested download files and
> directories both when they existed locally and when they didn't.
>
>
> Thanks,
>
> Tim Thorpe
>
>
