[
https://issues.apache.org/jira/browse/KNOX-1000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16139993#comment-16139993
]
Larry McCay commented on KNOX-1000:
-----------------------------------
JIRA #1000!!! :)
[~jeffreyr97] - I think that this JIRA is targeting the use of the hdfs
filesystem cli commands where you can provide a webhdfs URL today.
In addition, it seems like maybe you want to be able to write additional apps
that leverage the hadoop filesystem abstraction for access to resources through
Knox.
Is this correct?
I have played around with this a few times.
IIRC - we will need to modify the hadoop java client to be able to deal with a
SPNEGO challenge following the redirect from NN to DN for access to the actual
resource.
We need to clearly articulate the usecase and It would be a good idea to start
a KIP to detail the subtasks required for support.
It's pretty easy to get it almost there but not quite making it end-to-end with
these sorts of things.
One usecase to consider is that of distcp using Knox as a source or destination
or both with some combination of using the hadoop auth provider and not.
> Hadoop Java Filesystem API support through Knox
> -----------------------------------------------
>
> Key: KNOX-1000
> URL: https://issues.apache.org/jira/browse/KNOX-1000
> Project: Apache Knox
> Issue Type: Bug
> Affects Versions: 0.9.1, 0.11.0, 0.12.0, 0.13.0
> Reporter: Jeffrey E Rodriguez
> Fix For: 0.14.0
>
>
> This Jira is to re-open a discussion on this issue from:
> https://mail-archives.apache.org/mod_mbox/knox-user/201511.mbox/%3CCACRbFyiy77fc5EHQ5c5oOQM=mz_upmevsoh7ermdreuoovn...@mail.gmail.com%3E
> 1. Hadoop FileSystem api needs to provide a "proxy" mechanism where we can
> add the user credentials. (same issue as any library or API try to access
> Knox). We may need to open a Hadoop Jira for this.
> 2. Knox Kerberos support (http client access to webhdfs) doesn't support
> "file access token". (whatever this means).
> 3. Use of Transfer-Encoding header in Knox.
> Ideally we should be able to use Knox URL through the Hadoop FileSystem api
> so we call from a Jupyter or Zeppelin notebook. Also it would help to
> programtically use the Hadoop Filesystem api through Knox.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
