[
https://issues.apache.org/jira/browse/HDFS-15753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
lujie updated HDFS-15753:
-------------------------
Description:
while we enable kerberos, we found that "hdfs fsck /path -move" always failed.
After checking the log, we find a WARN message:
2020-12-25 13:51:30,485 WARN org.apache.hadoop.ipc.Client: Exception
encountered while connecting to the server :
org.apache.hadoop.security.AccessControlException: Client cannot authenticate
via:[TOKEN, KERBEROS]
2020-12-25 13:51:30,490 WARN org.apache.hadoop.hdfs.server.namenode.NameNode:
Cannot initialize /lost+found.
2020-12-25 13:51:30,491 ERROR org.apache.hadoop.hdfs.server.namenode.NameNode:
copyBlocksToLostFound: error processing /private/file_name_sensitive.txt
java.io.IOException: failed to initialize lost+found
at
org.apache.hadoop.hdfs.server.namenode.NamenodeFsck.copyBlocksToLostFound(NamenodeFsck.java:772)
at
org.apache.hadoop.hdfs.server.namenode.NamenodeFsck.collectBlocksSummary(NamenodeFsck.java:718)
The root cause is Fsck use DFSClient to do operation like mkdir or create. But
once kerberos is enabled, the client can't do authentication due to it is now
on NameNode, not the client node.
Fixing the root cause is hard, my suggestion we should disable it while
KERBEROS is enabled, or enable it only when authorization is SIMPLE.
was:
while we enable kerberos, we found that "hdfs fsck /path -move" always failed.
After checking the log, we find a WARN message:
2020-12-25 13:51:30,485 WARN org.apache.hadoop.ipc.Client: Exception
encountered while connecting to the server :
org.apache.hadoop.security.AccessControlException: Client cannot authenticate
via:[TOKEN, KERBEROS]
2020-12-25 13:51:30,490 WARN org.apache.hadoop.hdfs.server.namenode.NameNode:
Cannot initialize /lost+found.
2020-12-25 13:51:30,491 ERROR org.apache.hadoop.hdfs.server.namenode.NameNode:
copyBlocksToLostFound: error processing /private/file_name_sensitive.txt
java.io.IOException: failed to initialize lost+found
at
org.apache.hadoop.hdfs.server.namenode.NamenodeFsck.copyBlocksToLostFound(NamenodeFsck.java:772)
at
org.apache.hadoop.hdfs.server.namenode.NamenodeFsck.collectBlocksSummary(NamenodeFsck.java:718)
The root cause is Fsck use DFSClient to do operation like mkdir or create. But
once kerberos is enabled, the client can't do authentication due to it is now
on NameNode, not the client node.
Fixing the root cause is hard, so we should disable it while KERBEROS is
enabled, or enable it only when authorization is SIMPLE.
> "fsck -move" does not work while enable kerberos
> ------------------------------------------------
>
> Key: HDFS-15753
> URL: https://issues.apache.org/jira/browse/HDFS-15753
> Project: Hadoop HDFS
> Issue Type: Bug
> Reporter: lujie
> Priority: Major
>
> while we enable kerberos, we found that "hdfs fsck /path -move" always failed.
> After checking the log, we find a WARN message:
>
> 2020-12-25 13:51:30,485 WARN org.apache.hadoop.ipc.Client: Exception
> encountered while connecting to the server :
> org.apache.hadoop.security.AccessControlException: Client cannot authenticate
> via:[TOKEN, KERBEROS]
> 2020-12-25 13:51:30,490 WARN
> org.apache.hadoop.hdfs.server.namenode.NameNode: Cannot initialize
> /lost+found.
> 2020-12-25 13:51:30,491 ERROR
> org.apache.hadoop.hdfs.server.namenode.NameNode: copyBlocksToLostFound: error
> processing /private/file_name_sensitive.txt
> java.io.IOException: failed to initialize lost+found
> at
> org.apache.hadoop.hdfs.server.namenode.NamenodeFsck.copyBlocksToLostFound(NamenodeFsck.java:772)
> at
> org.apache.hadoop.hdfs.server.namenode.NamenodeFsck.collectBlocksSummary(NamenodeFsck.java:718)
>
> The root cause is Fsck use DFSClient to do operation like mkdir or create.
> But once kerberos is enabled, the client can't do authentication due to it is
> now on NameNode, not the client node.
> Fixing the root cause is hard, my suggestion we should disable it while
> KERBEROS is enabled, or enable it only when authorization is SIMPLE.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
