[
https://issues.apache.org/jira/browse/FLINK-14360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16950842#comment-16950842
]
Andrey Zagrebin commented on FLINK-14360:
-----------------------------------------
Thanks for the details [~shenyinjie]
As I understand there should be some proxy service/router which resolves the
requests to the current paths (basically to fs.defaultFS which is probably
virtual in this case).
Eventually they go to the concrete clusters. To me it sounds more like a
responsibility of this proxy to take care about the tokens because Flink now is
supposed to work only with one Yarn cluster.
Having the real clusters configurable seems to add more responsibility to Flink
and looks hacky, especially if we completely substitute the existing path
resolving like I see in the PR, not sure about all consequences of it.
What about configuring the RBF for security? E.g. this issue
[HDFS-13532|https://jira.apache.org/jira/browse/HDFS-13532] looks like the
implementation of the security feature for the proxy, would it work?
> Flink on yarn should support obtain delegation tokens of multi hdfs namespaces
> ------------------------------------------------------------------------------
>
> Key: FLINK-14360
> URL: https://issues.apache.org/jira/browse/FLINK-14360
> Project: Flink
> Issue Type: Improvement
> Components: Deployment / YARN
> Reporter: Shen Yinjie
> Priority: Major
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> There's a scenario when deploy flink on yarn with multi hdfs cluster or hdfs
> federation, Flink need to get delegation tokens of all the namespaces before
> start appmaster.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
