xingchen1997 commented on issue #3598:
URL: https://github.com/apache/amoro/issues/3598#issuecomment-2952323538
Problem Analysis:
The issue occurred because spark-shell was launched under the root user.
Hadoop’s security mechanism prevents the root user from impersonating the
hadoop user (or other users/groups) to operate YARN resources. This requires
explicit proxy permission configuration in Hadoop.
Solution:
Add the following properties to Hadoop’s core-site.xml to allow the root
user to proxy the hadoop user (or groups):
<property>
<name>hadoop.proxyuser.root.hosts</name>
<value>*</value>
</property>
<property>
<name>hadoop.proxyuser.root.groups</name>
<value>*</value>
</property>
Activation Steps:
Distribute the updated core-site.xml to all Hadoop nodes.
Restart HDFS/YARN services to apply the changes.
This configuration grants the root user permission to impersonate (proxy)
any host (`) and any group (`), resolving the authorization conflict.
是因为我在命令行启动 spark-shell，此时的用户是 root，Hadoop 的安全机制阻止了 root 用户代理 hadoop 用户操作
YARN 资源，需在 Hadoop 配置中明确授权代理关系
解决方案：需要修改 Hadoop 的 core-site.xml 配置，添加以下配置，允许 root 用户代理 hadoop 用户（或用户组）：
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
