[
https://issues.apache.org/jira/browse/HDFS-13532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16787099#comment-16787099
]
CR Hota commented on HDFS-13532:
--------------------------------
[~hexiaoqiao] [~elgoiri]
I think the confusion is arising as I said earlier is with default FS. In our
set-up, defaultFS is still main namenode. Clients(RM or generic clients) trying
to access RBF need to specify fully qualified name
hdfs://router-nameservice/mydata. Yarn nodes should already have the changes in
hdfs-site.xml to expose this new nameservice.
Our migration steps are
1. Expose router endpoints to all services first, workflow scheduler, yarn,
hive, presto, spark etc WITHOUT making router as the defaultFS.
2. Change ingestion to update hive metastore and point table and partition
location to hdfs://router-nameservice instead of hdfs://main-service.
3. Query engines now accessing data that is onboarded on top of router, pass
fully qualified hdfs uri let yarn know where to connect to for both read and
write (delegation token calls are on a simpler level just write RPC calls such
as create, append etc etc).
4. We haven't yet changed defaultFS and will do that much later, probably
towards end of q3.
We never spend time in investigating how defaultFS can be changed, that would
be quite complex honestly as far as i can think.
Let's park this discussion till we get more insights from you when you try the
actual migration at your end. Share failure logs etc and we can re-visit this.
> RBF: Adding security
> --------------------
>
> Key: HDFS-13532
> URL: https://issues.apache.org/jira/browse/HDFS-13532
> Project: Hadoop HDFS
> Issue Type: New Feature
> Reporter: Íñigo Goiri
> Assignee: CR Hota
> Priority: Major
> Attachments: RBF _ Security delegation token thoughts.pdf, RBF _
> Security delegation token thoughts_updated.pdf, RBF _ Security delegation
> token thoughts_updated_2.pdf, RBF-DelegationToken-Approach1b.pdf, RBF_
> Security delegation token thoughts_updated_3.pdf, Security_for_Router-based
> Federation_design_doc.pdf
>
>
> HDFS Router based federation should support security. This includes
> authentication and delegation tokens.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
