[jira] [Commented] (YARN-10382) Non-secure YARN access secure HDFS
[ https://issues.apache.org/jira/browse/YARN-10382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17200764#comment-17200764 ] bianqi commented on YARN-10382: --- stackoverflow issues: https://stackoverflow.com/questions/42650562/access-a-secured-hive-when-running-spark-in-an-unsecured-yarn-cluster > Non-secure YARN access secure HDFS > -- > > Key: YARN-10382 > URL: https://issues.apache.org/jira/browse/YARN-10382 > Project: Hadoop YARN > Issue Type: New Feature > Components: yarn >Reporter: bianqi >Priority: Minor > > In our production environment, yarn cannot enable kerberos due to yarn > environment problems, but our hdfs is to enable kerberos, and now we need > non-secure yarn to access secure hdfs. > It is known that yarn and hdfs are both safe after security is turned on. > I hope that after enabling hdfs security, you can use non-secure yarn to > access secure hdfs, or use secure yarn to access non-secure hdfs. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10382) Non-secure YARN access secure HDFS
[
https://issues.apache.org/jira/browse/YARN-10382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17183273#comment-17183273
]
bianqi commented on YARN-10382:
---
[~aajisaka]
{quote}Everyone who can access to the RM can submit jobs to insecure YARN
cluster as HDFS superuser. That way they can access HDFS as superuser.
{quote}
We do not use superuser access HDFS.What you said does not fit our scenario.
> Non-secure YARN access secure HDFS
> --
>
> Key: YARN-10382
> URL: https://issues.apache.org/jira/browse/YARN-10382
> Project: Hadoop YARN
> Issue Type: New Feature
> Components: yarn
>Reporter: bianqi
>Priority: Minor
>
> In our production environment, yarn cannot enable kerberos due to yarn
> environment problems, but our hdfs is to enable kerberos, and now we need
> non-secure yarn to access secure hdfs.
> It is known that yarn and hdfs are both safe after security is turned on.
> I hope that after enabling hdfs security, you can use non-secure yarn to
> access secure hdfs, or use secure yarn to access non-secure hdfs.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10382) Non-secure yarn access secure hdfs
[
https://issues.apache.org/jira/browse/YARN-10382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17183269#comment-17183269
]
bianqi commented on YARN-10382:
---
[~ste...@apache.org] Thank you for your reply.
{quote}You could somehow cheat the configs to name some kerberos principal
(yourself?)
{quote}
I tried to customize the kerberos principal, but it still cannot be
resolved.I think it may be necessary to modify the server source code.
> Non-secure yarn access secure hdfs
> --
>
> Key: YARN-10382
> URL: https://issues.apache.org/jira/browse/YARN-10382
> Project: Hadoop YARN
> Issue Type: New Feature
> Components: yarn
>Reporter: bianqi
>Priority: Minor
>
> In our production environment, yarn cannot enable kerberos due to yarn
> environment problems, but our hdfs is to enable kerberos, and now we need
> non-secure yarn to access secure hdfs.
> It is known that yarn and hdfs are both safe after security is turned on.
> I hope that after enabling hdfs security, you can use non-secure yarn to
> access secure hdfs, or use secure yarn to access non-secure hdfs.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10382) Non-secure yarn access secure hdfs
[ https://issues.apache.org/jira/browse/YARN-10382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17174253#comment-17174253 ] Steve Loughran commented on YARN-10382: --- Problem there is that the code wants to know who the YARN principal of the resource manager is so that it can send messages to HDFS saying "renew these delegation tokens". Your insecure YARN RM doesn't have a kerberos principal, so secure HDFS will not issue delegation tokens to it. You could somehow cheat the configs to name some kerberos principal (yourself?) as the RM principal -no idea what happens then. I would personally like YARN To collect tokens from services even when Kerberos is disabled, though not for your use case - I want to be able to collect tokens for the object stores. But I've avoiding going near the code as (a) I'm scared and (b) applications like Spark do their own checks against UserGroupInformation.isSecurityEnabled() which still wouldn't work > Non-secure yarn access secure hdfs > -- > > Key: YARN-10382 > URL: https://issues.apache.org/jira/browse/YARN-10382 > Project: Hadoop YARN > Issue Type: New Feature > Components: yarn >Reporter: bianqi >Priority: Minor > > In our production environment, yarn cannot enable kerberos due to yarn > environment problems, but our hdfs is to enable kerberos, and now we need > non-secure yarn to access secure hdfs. > It is known that yarn and hdfs are both safe after security is turned on. > I hope that after enabling hdfs security, you can use non-secure yarn to > access secure hdfs, or use secure yarn to access non-secure hdfs. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10382) Non-secure yarn access secure hdfs
[ https://issues.apache.org/jira/browse/YARN-10382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17169846#comment-17169846 ] bianqi commented on YARN-10382: --- Thank you for your reply. Our scenario is that there are two big data clusters, one of which is a non-secure cluster and the other is a secure cluster. Now there is a unified client who wants to submit tasks to the non-secure yarn cluster for access Safe HDFS. Non-secure HDFS and Secure yarn are not the same cluster. > Non-secure yarn access secure hdfs > -- > > Key: YARN-10382 > URL: https://issues.apache.org/jira/browse/YARN-10382 > Project: Hadoop YARN > Issue Type: New Feature > Components: yarn >Reporter: bianqi >Priority: Major > > In our production environment, yarn cannot enable kerberos due to yarn > environment problems, but our hdfs is to enable kerberos, and now we need > non-secure yarn to access secure hdfs. > It is known that yarn and hdfs are both safe after security is turned on. > I hope that after enabling hdfs security, you can use non-secure yarn to > access secure hdfs, or use secure yarn to access non-secure hdfs. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10382) Non-secure yarn access secure hdfs
[ https://issues.apache.org/jira/browse/YARN-10382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17169841#comment-17169841 ] Akira Ajisaka commented on YARN-10382: -- Everyone who can access to the RM can submit jobs to insecure YARN cluster as HDFS superuser. That way they can access HDFS as superuser. > Non-secure yarn access secure hdfs > -- > > Key: YARN-10382 > URL: https://issues.apache.org/jira/browse/YARN-10382 > Project: Hadoop YARN > Issue Type: New Feature > Components: yarn >Reporter: bianqi >Priority: Major > > In our production environment, yarn cannot enable kerberos due to yarn > environment problems, but our hdfs is to enable kerberos, and now we need > non-secure yarn to access secure hdfs. > It is known that yarn and hdfs are both safe after security is turned on. > I hope that after enabling hdfs security, you can use non-secure yarn to > access secure hdfs, or use secure yarn to access non-secure hdfs. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10382) Non-secure yarn access secure hdfs
[ https://issues.apache.org/jira/browse/YARN-10382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17169839#comment-17169839 ] Akira Ajisaka commented on YARN-10382: -- -1 > Non-secure yarn access secure hdfs > -- > > Key: YARN-10382 > URL: https://issues.apache.org/jira/browse/YARN-10382 > Project: Hadoop YARN > Issue Type: New Feature > Components: yarn >Reporter: bianqi >Priority: Major > > In our production environment, yarn cannot enable kerberos due to yarn > environment problems, but our hdfs is to enable kerberos, and now we need > non-secure yarn to access secure hdfs. > It is known that yarn and hdfs are both safe after security is turned on. > I hope that after enabling hdfs security, you can use non-secure yarn to > access secure hdfs, or use secure yarn to access non-secure hdfs. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
