[
https://issues.apache.org/jira/browse/HDFS-13532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16784687#comment-16784687
]
Íñigo Goiri commented on HDFS-13532:
------------------------------------
{quote}
(1) router takes over delegation tokens management from namenodes at all, (2)
namenode only maintain delegation token request from router. right? IIUC, maybe
there are no graceful gray solution to upgrade clients
{quote}
You can set it up so that clients can still directly work against the Namenodes.
Actually, for you to disable the Namenodes from authenticating clients, you
have to configure it that way and restrict it so only the Routers can get
authenticated.
{quote}
Consider about one job submit to YARN from client which is upgrade to support
RBF, and all delegation tokens are distributed from router, but if yarn still
not upgrade, all executors will authenticate fail to namenode since delegation
token is not matching. Of course this issue is also true if upgrade yarn first
then client.
{quote}
If the job is submitted against the Router, then the job can only access data
through RBF.
However, I think this is OK; as I mentioned before you could still have jobs
that query the Namenodes directly.
Both types of jobs can coexist, it would just be a matter of having a different
{{fs.defaultFS}} for each of them.
For the RM itself, you can transition it from using RBF or not whenever you
want.
Both modes should work at the same time here too.
{quote}
2. any performance test results about zookeeper which manage massive delegation
tokens? I am not very familiar with zookeeper, and if there are obvious
performance differences between zookeeper and memory at namenode before RBF. If
no evaluation, I would like to test it later.
3. if znode number impact performance of delegation token request in zookeeper?
delegation token request ops is very high for a large cluster, for instance,
1000K jobs every day and the maximum lifetime for which a delegation token is
valid set default by 7 days, in the worst case, it will backlog 7000K znodes at
all. some risk for more large cluster?
{quote}
The scale we have used internally with ZK is not large enough for me to give
you a proper answer here.
I think [~crh] had some larger tests.
However, the State Store is pluggable so one could use other approaches like:
HDFS-13245 (still under review) or HDFS-10630.
The secret manager is only in ZK ({{ZKDelegationTokenSecretManagerImpl}} based
on {{ZKDelegationTokenSecretManager}}) but should be easy to extend.
> RBF: Adding security
> --------------------
>
> Key: HDFS-13532
> URL: https://issues.apache.org/jira/browse/HDFS-13532
> Project: Hadoop HDFS
> Issue Type: New Feature
> Reporter: Íñigo Goiri
> Assignee: CR Hota
> Priority: Major
> Attachments: RBF _ Security delegation token thoughts.pdf, RBF _
> Security delegation token thoughts_updated.pdf, RBF _ Security delegation
> token thoughts_updated_2.pdf, RBF-DelegationToken-Approach1b.pdf, RBF_
> Security delegation token thoughts_updated_3.pdf, Security_for_Router-based
> Federation_design_doc.pdf
>
>
> HDFS Router based federation should support security. This includes
> authentication and delegation tokens.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
