[
https://issues.apache.org/jira/browse/HBASE-17861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15969342#comment-15969342
]
Umesh Agashe commented on HBASE-17861:
--------------------------------------
Thanks to [~waterlx] for pointing this JIRA to me. Sometime back when I tried
running HBase on S3 I got the same issue. I discussed it with my colleague. All
these filesystems will differ in what typical FS features they support. I think
this needs to be handled at broader level than hard-coded list. We can have a
configuration toggle for checking permissions and when the exception is thrown
due to underlying filesystem limitations, we can point to the configuration
that needs to be toggled. This can be done for all similar FS issues that we
may come across in future.
Eventually we can document all configurations options in a reference doc to
support certain filesystem. Let me know your thoughts. Thanks!
> Regionserver down when checking the permission of staging dir if
> hbase.rootdir is on S3
> ---------------------------------------------------------------------------------------
>
> Key: HBASE-17861
> URL: https://issues.apache.org/jira/browse/HBASE-17861
> Project: HBase
> Issue Type: Bug
> Affects Versions: 1.4.0
> Reporter: Yi Liang
> Assignee: Yi Liang
> Labels: filesystem, s3, wal
> Fix For: 1.4.0
>
> Attachments: HBASE-17861.branch-1.V1.patch,
> HBASE-17861.branch-1.V2.patch, HBASE-17861.branch-1.V3.patch,
> HBASE-17861.branch-1.V4.patch, HBASE-17861-V1.patch
>
>
> Found some issue, when set up HBASE-17437: Support specifying a WAL directory
> outside of the root directory.
> The region server are showdown when I add following config into
> hbase-site.xml
> hbase.rootdir = s3a://xx//xx
> hbase.wal.dir = hdfs://xx/xx
> hbase.coprocessor.region.classes =
> org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint
> Error is below
> {noformat}
> org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint threw
> java.lang.IllegalStateException: Directory already exists but permissions
> aren't set to '-rwx--x--x'
> {noformat}
> The reason is that, when hbase enable securebulkload, hbase will create a
> folder in s3, it can not set above permission, because in s3, all files are
> listed as having full read/write permissions and all directories appear to
> have full rwx permissions. See Object stores have differerent authorization
> models in
> https://hadoop.apache.org/docs/current3/hadoop-aws/tools/hadoop-aws/index.html
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
