Clint Heath created HBASE-8644:
----------------------------------
Summary: "hbck -fixReferenceFiles" checks for write permissions in
/ even if sidelineDir is specified
Key: HBASE-8644
URL: https://issues.apache.org/jira/browse/HBASE-8644
Project: HBase
Issue Type: Bug
Components: hbck
Affects Versions: 0.94.2
Reporter: Clint Heath
Priority: Minor
We found a situation where a temporary reference file got left in META's
recovered.edits directory and this was causing hbck to report inconsistency
with HBase. When trying to repair it with the "hbck -fixReferenceFiles"
command, an access control exception was thrown, because hbck was trying to put
the sidelined reference file in the root of HDFS and the hbase user did not
have permissions. But even after specifying a "sidelineDir" (hbck
-fixReferenceFiles -sidelineDir hdfs://<namespace>/tmp), the same exception was
thrown (logs obfuscated below):
INFO util.HBaseFsck: Trying to sildeline reference
filehdfs://<namespace>/hbase/.META./1028685194/recovered.edits/0000000000000000016.temp
to hdfs://<namespace>/0000000000000000016.temp
Exception in thread "main" org.apache.hadoop.security.AccessControlException:
Permission denied: user=hbase, access=WRITE, inode="/":hdfs:hadoop:drwxr-xr-x
Since the operation requires moving data around in HDFS, we tried the same
command as the "hdfs" user which should have no permission problems, but an
exception was thrown nonetheless:
...
WARN util.HBaseFsck: Got AccessControlException when preCheckPermission
org.apache.hadoop.security.AccessControlException: Permission denied:
action=WRITE path=hdfs://<namespace>/hbase/-ROOT- user=hdfs
Ultimately in order to get this working, we ran this command to tell hbck to
not run it's prechecks:
hbase hbck -fixReferenceFiles -sidelineDir hdfs://<namespace>/tmp
-ignorePreCheckPermission
A few thoughts here:
1) it seems that / is a bad default location to attempt to sideline files
2) shouldn't hbck be run as the hbase user? If so, it should sideline files to
a directory it has write permissions to.
3) if we expect hbck to be run as the hdfs user, permissions should not be
denied that user when HDFS itself will not deny them.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
