[
https://issues.apache.org/jira/browse/NIFI-1924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15307752#comment-15307752
]
Bryan Bende commented on NIFI-1924:
-----------------------------------
[~trixpan] I don't really have any experience with WebHDFS... all I can say is
that the hadoop processors use the UserGroupInformation class to login with the
principal & keytab provided in the processor:
{code}
UserGroupInformation ugi =
UserGroupInformation.loginUserFromKeytabAndReturnUGI(principal.trim(),
keyTab.trim());
{code}
then obtain a file system reference like this:
{code}
protected FileSystem getFileSystemAsUser(final Configuration config,
UserGroupInformation ugi) throws IOException {
try {
return ugi.doAs(new PrivilegedExceptionAction<FileSystem>() {
@Override
public FileSystem run() throws Exception {
return FileSystem.get(config);
}
});
} catch (InterruptedException e) {
throw new IOException("Unable to create file system: " +
e.getMessage());
}
}
{code}
I'm not sure if that would be expected work over the WebHDFS REST calls.
> Create set of WebHDFS processors
> --------------------------------
>
> Key: NIFI-1924
> URL: https://issues.apache.org/jira/browse/NIFI-1924
> Project: Apache NiFi
> Issue Type: Improvement
> Reporter: Andre
>
> Create processors to handle,
> Supported commands for WebHDFS V1
> * GET -> GetWebHDFS
> * OPEN
> * GETFILESTATUS
> * LISTSTATUS
> * GETCONTENTSUMMARy
> * GETFILECHECKSUM
> * GETHOMEDIRECTORY
> * GETDELEGATIONTOKEN
> * GETDELEGATIONTOKENS
> * GETXATTRS
> * LISTXATTRS
> * CHECKACCESS
> * PUT -> PutWebHDFS
> * CREATE
> * MKDIRS
> * CREATESYMLINK
> * RENAME
> * SETREPLICATION
> * SETOWNER
> * SETPERMISSION
> * SETTIMES
> * RENEWDELEGATIONTOKEN
> * CANCELDELEGATIONTOKEN
> * CREATESNAPSHOT
> * RENAMESNAPSHOT
> * SETXATTR
> * REMOVEXATTR
> * POST -> PostWebHDFS
> * APPEND
> * CONCAT
> * TRUNCATE
> * DELETE -> DeleteWebHDFS
> * DELETE
> * DELETESNAPSHOT
> operations against WebHDFS as discussed here:
> http://mail-archives.apache.org/mod_mbox/nifi-users/201604.mbox/%3CCABtjSZk2KLJJVk%2Byk-ySioPxVUV5SLHcN%2BaWMZs1ARpkfy%2B0Ow%40mail.gmail.com%3E
> Required Processor Properties:
> * Namenode Host
> * WebHDFS port
> * WebHDFS API version. Default to v1. Dropdown list of available options.
> Currently there is only v1 but would be good to have that in place to show
> our intentions.
> * Hadoop configuration files. core-site.xml and hdfs-site.xml. These
> configurations files should not be required to keep things lite. However they
> would be required to access a kerberized Hadoop cluster. If these
> configuration files are present they should take precedence over the
> properties set in NameNode host and WebHDFS port.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
