[jira] [Updated] (HBASE-17861) Regionserver down when checking the permission of staging dir if hbase.rootdir is on S3

2018-08-19 Thread Sean Busbey (JIRA) 


 [ 
https://issues.apache.org/jira/browse/HBASE-17861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean Busbey updated HBASE-17861:

Fix Version/s: 1.3.3
   1.2.7

> Regionserver down when checking the permission of staging dir if 
> hbase.rootdir is on S3
> ---
>
> Key: HBASE-17861
> URL: https://issues.apache.org/jira/browse/HBASE-17861
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 1.4.0
>Reporter: Yi Liang
>Assignee: Yi Liang
>Priority: Major
>  Labels: filesystem, s3, wal
> Fix For: 1.4.0, 1.2.7, 1.3.3
>
> Attachments: HBASE-17861-V1.patch, HBASE-17861.branch-1.V1.patch, 
> HBASE-17861.branch-1.V2.patch, HBASE-17861.branch-1.V3.patch, 
> HBASE-17861.branch-1.V4.patch
>
>
> Found some issue, when set up HBASE-17437: Support specifying a WAL directory 
> outside of the root directory.
> The region server are  showdown when I add following config into 
> hbase-site.xml 
> hbase.rootdir =  s3a://xx//xx
> hbase.wal.dir = hdfs://xx/xx
> hbase.coprocessor.region.classes = 
> org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint
> Error is below
> {noformat}
> org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint threw 
> java.lang.IllegalStateException: Directory already exists but permissions 
> aren't set to '-rwx--x--x'
> {noformat}
> The reason is that, when hbase enable securebulkload, hbase will create a 
> folder in s3, it can not set above permission, because in s3, all files are 
> listed as having full read/write permissions and all directories appear to 
> have full rwx permissions. See Object stores have differerent authorization 
> models in 
> https://hadoop.apache.org/docs/current3/hadoop-aws/tools/hadoop-aws/index.html



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-17861) Regionserver down when checking the permission of staging dir if hbase.rootdir is on S3

2017-04-04 Thread Ted Yu (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HBASE-17861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ted Yu updated HBASE-17861:
---
Resolution: Fixed
Status: Resolved  (was: Patch Available)

Thanks for the patch, Li.

Thanks for the reviews, Zach and Jerry.

> Regionserver down when checking the permission of staging dir if 
> hbase.rootdir is on S3
> ---
>
> Key: HBASE-17861
> URL: https://issues.apache.org/jira/browse/HBASE-17861
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 1.4.0
>Reporter: Yi Liang
>Assignee: Yi Liang
>  Labels: filesystem, s3, wal
> Fix For: 1.4.0
>
> Attachments: HBASE-17861.branch-1.V1.patch, 
> HBASE-17861.branch-1.V2.patch, HBASE-17861.branch-1.V3.patch, 
> HBASE-17861.branch-1.V4.patch, HBASE-17861-V1.patch
>
>
> Found some issue, when set up HBASE-17437: Support specifying a WAL directory 
> outside of the root directory.
> The region server are  showdown when I add following config into 
> hbase-site.xml 
> hbase.rootdir =  s3a://xx//xx
> hbase.wal.dir = hdfs://xx/xx
> hbase.coprocessor.region.classes = 
> org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint
> Error is below
> {noformat}
> org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint threw 
> java.lang.IllegalStateException: Directory already exists but permissions 
> aren't set to '-rwx--x--x'
> {noformat}
> The reason is that, when hbase enable securebulkload, hbase will create a 
> folder in s3, it can not set above permission, because in s3, all files are 
> listed as having full read/write permissions and all directories appear to 
> have full rwx permissions. See Object stores have differerent authorization 
> models in 
> https://hadoop.apache.org/docs/current3/hadoop-aws/tools/hadoop-aws/index.html



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (HBASE-17861) Regionserver down when checking the permission of staging dir if hbase.rootdir is on S3

2017-04-04 Thread Yi Liang (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HBASE-17861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yi Liang updated HBASE-17861:
-
Release Note: Some object store does not support unix style permission, 
this jira fix the permission check issue when specify staging dir in different 
file system. Currently it covers s3, wasb, swift. 

> Regionserver down when checking the permission of staging dir if 
> hbase.rootdir is on S3
> ---
>
> Key: HBASE-17861
> URL: https://issues.apache.org/jira/browse/HBASE-17861
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 1.4.0
>Reporter: Yi Liang
>Assignee: Yi Liang
>  Labels: filesystem, s3, wal
> Fix For: 1.4.0
>
> Attachments: HBASE-17861.branch-1.V1.patch, 
> HBASE-17861.branch-1.V2.patch, HBASE-17861.branch-1.V3.patch, 
> HBASE-17861.branch-1.V4.patch, HBASE-17861-V1.patch
>
>
> Found some issue, when set up HBASE-17437: Support specifying a WAL directory 
> outside of the root directory.
> The region server are  showdown when I add following config into 
> hbase-site.xml 
> hbase.rootdir =  s3a://xx//xx
> hbase.wal.dir = hdfs://xx/xx
> hbase.coprocessor.region.classes = 
> org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint
> Error is below
> {noformat}
> org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint threw 
> java.lang.IllegalStateException: Directory already exists but permissions 
> aren't set to '-rwx--x--x'
> {noformat}
> The reason is that, when hbase enable securebulkload, hbase will create a 
> folder in s3, it can not set above permission, because in s3, all files are 
> listed as having full read/write permissions and all directories appear to 
> have full rwx permissions. See Object stores have differerent authorization 
> models in 
> https://hadoop.apache.org/docs/current3/hadoop-aws/tools/hadoop-aws/index.html



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (HBASE-17861) Regionserver down when checking the permission of staging dir if hbase.rootdir is on S3

2017-04-04 Thread Ted Yu (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HBASE-17861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ted Yu updated HBASE-17861:
---
Hadoop Flags: Reviewed

+1

Please fill out release note.

> Regionserver down when checking the permission of staging dir if 
> hbase.rootdir is on S3
> ---
>
> Key: HBASE-17861
> URL: https://issues.apache.org/jira/browse/HBASE-17861
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 1.4.0
>Reporter: Yi Liang
>Assignee: Yi Liang
>  Labels: filesystem, s3, wal
> Fix For: 1.4.0
>
> Attachments: HBASE-17861.branch-1.V1.patch, 
> HBASE-17861.branch-1.V2.patch, HBASE-17861.branch-1.V3.patch, 
> HBASE-17861.branch-1.V4.patch, HBASE-17861-V1.patch
>
>
> Found some issue, when set up HBASE-17437: Support specifying a WAL directory 
> outside of the root directory.
> The region server are  showdown when I add following config into 
> hbase-site.xml 
> hbase.rootdir =  s3a://xx//xx
> hbase.wal.dir = hdfs://xx/xx
> hbase.coprocessor.region.classes = 
> org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint
> Error is below
> {noformat}
> org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint threw 
> java.lang.IllegalStateException: Directory already exists but permissions 
> aren't set to '-rwx--x--x'
> {noformat}
> The reason is that, when hbase enable securebulkload, hbase will create a 
> folder in s3, it can not set above permission, because in s3, all files are 
> listed as having full read/write permissions and all directories appear to 
> have full rwx permissions. See Object stores have differerent authorization 
> models in 
> https://hadoop.apache.org/docs/current3/hadoop-aws/tools/hadoop-aws/index.html



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (HBASE-17861) Regionserver down when checking the permission of staging dir if hbase.rootdir is on S3

2017-04-03 Thread Yi Liang (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HBASE-17861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yi Liang updated HBASE-17861:
-
Attachment: HBASE-17861.branch-1.V4.patch

> Regionserver down when checking the permission of staging dir if 
> hbase.rootdir is on S3
> ---
>
> Key: HBASE-17861
> URL: https://issues.apache.org/jira/browse/HBASE-17861
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 1.4.0
>Reporter: Yi Liang
>Assignee: Yi Liang
>  Labels: filesystem, s3, wal
> Fix For: 1.4.0
>
> Attachments: HBASE-17861.branch-1.V1.patch, 
> HBASE-17861.branch-1.V2.patch, HBASE-17861.branch-1.V3.patch, 
> HBASE-17861.branch-1.V4.patch, HBASE-17861-V1.patch
>
>
> Found some issue, when set up HBASE-17437: Support specifying a WAL directory 
> outside of the root directory.
> The region server are  showdown when I add following config into 
> hbase-site.xml 
> hbase.rootdir =  s3a://xx//xx
> hbase.wal.dir = hdfs://xx/xx
> hbase.coprocessor.region.classes = 
> org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint
> Error is below
> {noformat}
> org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint threw 
> java.lang.IllegalStateException: Directory already exists but permissions 
> aren't set to '-rwx--x--x'
> {noformat}
> The reason is that, when hbase enable securebulkload, hbase will create a 
> folder in s3, it can not set above permission, because in s3, all files are 
> listed as having full read/write permissions and all directories appear to 
> have full rwx permissions. See Object stores have differerent authorization 
> models in 
> https://hadoop.apache.org/docs/current3/hadoop-aws/tools/hadoop-aws/index.html



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (HBASE-17861) Regionserver down when checking the permission of staging dir if hbase.rootdir is on S3

2017-04-03 Thread Yi Liang (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HBASE-17861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yi Liang updated HBASE-17861:
-
Attachment: HBASE-17861.branch-1.V3.patch

> Regionserver down when checking the permission of staging dir if 
> hbase.rootdir is on S3
> ---
>
> Key: HBASE-17861
> URL: https://issues.apache.org/jira/browse/HBASE-17861
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 1.4.0
>Reporter: Yi Liang
>Assignee: Yi Liang
>  Labels: filesystem, s3, wal
> Fix For: 1.4.0
>
> Attachments: HBASE-17861.branch-1.V1.patch, 
> HBASE-17861.branch-1.V2.patch, HBASE-17861.branch-1.V3.patch, 
> HBASE-17861-V1.patch
>
>
> Found some issue, when set up HBASE-17437: Support specifying a WAL directory 
> outside of the root directory.
> The region server are  showdown when I add following config into 
> hbase-site.xml 
> hbase.rootdir =  s3a://xx//xx
> hbase.wal.dir = hdfs://xx/xx
> hbase.coprocessor.region.classes = 
> org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint
> Error is below
> {noformat}
> org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint threw 
> java.lang.IllegalStateException: Directory already exists but permissions 
> aren't set to '-rwx--x--x'
> {noformat}
> The reason is that, when hbase enable securebulkload, hbase will create a 
> folder in s3, it can not set above permission, because in s3, all files are 
> listed as having full read/write permissions and all directories appear to 
> have full rwx permissions. See Object stores have differerent authorization 
> models in 
> https://hadoop.apache.org/docs/current3/hadoop-aws/tools/hadoop-aws/index.html



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (HBASE-17861) Regionserver down when checking the permission of staging dir if hbase.rootdir is on S3

2017-04-03 Thread Yi Liang (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HBASE-17861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yi Liang updated HBASE-17861:
-
Attachment: (was: HBASE-17861.branch-1.V2.patch)

> Regionserver down when checking the permission of staging dir if 
> hbase.rootdir is on S3
> ---
>
> Key: HBASE-17861
> URL: https://issues.apache.org/jira/browse/HBASE-17861
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 1.4.0
>Reporter: Yi Liang
>Assignee: Yi Liang
>  Labels: filesystem, s3, wal
> Fix For: 1.4.0
>
> Attachments: HBASE-17861.branch-1.V1.patch, 
> HBASE-17861.branch-1.V2.patch, HBASE-17861-V1.patch
>
>
> Found some issue, when set up HBASE-17437: Support specifying a WAL directory 
> outside of the root directory.
> The region server are  showdown when I add following config into 
> hbase-site.xml 
> hbase.rootdir =  s3a://xx//xx
> hbase.wal.dir = hdfs://xx/xx
> hbase.coprocessor.region.classes = 
> org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint
> Error is below
> {noformat}
> org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint threw 
> java.lang.IllegalStateException: Directory already exists but permissions 
> aren't set to '-rwx--x--x'
> {noformat}
> The reason is that, when hbase enable securebulkload, hbase will create a 
> folder in s3, it can not set above permission, because in s3, all files are 
> listed as having full read/write permissions and all directories appear to 
> have full rwx permissions. See Object stores have differerent authorization 
> models in 
> https://hadoop.apache.org/docs/current3/hadoop-aws/tools/hadoop-aws/index.html



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (HBASE-17861) Regionserver down when checking the permission of staging dir if hbase.rootdir is on S3

2017-04-03 Thread Yi Liang (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HBASE-17861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yi Liang updated HBASE-17861:
-
Attachment: HBASE-17861.branch-1.V2.patch
HBASE-17861.branch-1.V2.patch

> Regionserver down when checking the permission of staging dir if 
> hbase.rootdir is on S3
> ---
>
> Key: HBASE-17861
> URL: https://issues.apache.org/jira/browse/HBASE-17861
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 1.4.0
>Reporter: Yi Liang
>Assignee: Yi Liang
>  Labels: filesystem, s3, wal
> Fix For: 1.4.0
>
> Attachments: HBASE-17861.branch-1.V1.patch, 
> HBASE-17861.branch-1.V2.patch, HBASE-17861-V1.patch
>
>
> Found some issue, when set up HBASE-17437: Support specifying a WAL directory 
> outside of the root directory.
> The region server are  showdown when I add following config into 
> hbase-site.xml 
> hbase.rootdir =  s3a://xx//xx
> hbase.wal.dir = hdfs://xx/xx
> hbase.coprocessor.region.classes = 
> org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint
> Error is below
> {noformat}
> org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint threw 
> java.lang.IllegalStateException: Directory already exists but permissions 
> aren't set to '-rwx--x--x'
> {noformat}
> The reason is that, when hbase enable securebulkload, hbase will create a 
> folder in s3, it can not set above permission, because in s3, all files are 
> listed as having full read/write permissions and all directories appear to 
> have full rwx permissions. See Object stores have differerent authorization 
> models in 
> https://hadoop.apache.org/docs/current3/hadoop-aws/tools/hadoop-aws/index.html



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (HBASE-17861) Regionserver down when checking the permission of staging dir if hbase.rootdir is on S3

2017-04-03 Thread Yi Liang (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HBASE-17861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yi Liang updated HBASE-17861:
-
Attachment: (was: HBASE-17861.branch-1.V2.patch)

> Regionserver down when checking the permission of staging dir if 
> hbase.rootdir is on S3
> ---
>
> Key: HBASE-17861
> URL: https://issues.apache.org/jira/browse/HBASE-17861
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 1.4.0
>Reporter: Yi Liang
>Assignee: Yi Liang
>  Labels: filesystem, s3, wal
> Fix For: 1.4.0
>
> Attachments: HBASE-17861.branch-1.V1.patch, 
> HBASE-17861.branch-1.V2.patch, HBASE-17861-V1.patch
>
>
> Found some issue, when set up HBASE-17437: Support specifying a WAL directory 
> outside of the root directory.
> The region server are  showdown when I add following config into 
> hbase-site.xml 
> hbase.rootdir =  s3a://xx//xx
> hbase.wal.dir = hdfs://xx/xx
> hbase.coprocessor.region.classes = 
> org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint
> Error is below
> {noformat}
> org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint threw 
> java.lang.IllegalStateException: Directory already exists but permissions 
> aren't set to '-rwx--x--x'
> {noformat}
> The reason is that, when hbase enable securebulkload, hbase will create a 
> folder in s3, it can not set above permission, because in s3, all files are 
> listed as having full read/write permissions and all directories appear to 
> have full rwx permissions. See Object stores have differerent authorization 
> models in 
> https://hadoop.apache.org/docs/current3/hadoop-aws/tools/hadoop-aws/index.html



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (HBASE-17861) Regionserver down when checking the permission of staging dir if hbase.rootdir is on S3

2017-04-03 Thread Yi Liang (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HBASE-17861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yi Liang updated HBASE-17861:
-
Attachment: HBASE-17861.branch-1.V2.patch

> Regionserver down when checking the permission of staging dir if 
> hbase.rootdir is on S3
> ---
>
> Key: HBASE-17861
> URL: https://issues.apache.org/jira/browse/HBASE-17861
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 1.4.0
>Reporter: Yi Liang
>Assignee: Yi Liang
>  Labels: filesystem, s3, wal
> Fix For: 1.4.0
>
> Attachments: HBASE-17861.branch-1.V1.patch, 
> HBASE-17861.branch-1.V2.patch, HBASE-17861-V1.patch
>
>
> Found some issue, when set up HBASE-17437: Support specifying a WAL directory 
> outside of the root directory.
> The region server are  showdown when I add following config into 
> hbase-site.xml 
> hbase.rootdir =  s3a://xx//xx
> hbase.wal.dir = hdfs://xx/xx
> hbase.coprocessor.region.classes = 
> org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint
> Error is below
> {noformat}
> org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint threw 
> java.lang.IllegalStateException: Directory already exists but permissions 
> aren't set to '-rwx--x--x'
> {noformat}
> The reason is that, when hbase enable securebulkload, hbase will create a 
> folder in s3, it can not set above permission, because in s3, all files are 
> listed as having full read/write permissions and all directories appear to 
> have full rwx permissions. See Object stores have differerent authorization 
> models in 
> https://hadoop.apache.org/docs/current3/hadoop-aws/tools/hadoop-aws/index.html



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (HBASE-17861) Regionserver down when checking the permission of staging dir if hbase.rootdir is on S3

2017-04-02 Thread Yi Liang (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HBASE-17861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yi Liang updated HBASE-17861:
-
Description: 
Found some issue, when set up HBASE-17437: Support specifying a WAL directory 
outside of the root directory.

The region server are  showdown when I add following config into hbase-site.xml 
hbase.rootdir =  s3a://xx//xx
hbase.wal.dir = hdfs://xx/xx
hbase.coprocessor.region.classes = 
org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint
Error is below
{noformat}
org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint threw 
java.lang.IllegalStateException: Directory already exists but permissions 
aren't set to '-rwx--x--x'
{noformat}

The reason is that, when hbase enable securebulkload, hbase will create a 
folder in s3, it can not set above permission, because in s3, all files are 
listed as having full read/write permissions and all directories appear to have 
full rwx permissions. See Object stores have differerent authorization models 
in 
https://hadoop.apache.org/docs/current3/hadoop-aws/tools/hadoop-aws/index.html




  was:
Found some issue, when set up HBASE-17437: Support specifying a WAL directory 
outside of the root directory.

The region server are  showdown when I add following config into hbase-site.xml 
hbase.rootdir = hdfs://xx/xx
hbase.wal.dir = s3a://xx//xx
hbase.coprocessor.region.classes = 
org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint
Error is below
{noformat}
org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint threw 
java.lang.IllegalStateException: Directory already exists but permissions 
aren't set to '-rwx--x--x'
{noformat}

The reason is that, when hbase enable securebulkload, hbase will create a 
folder in s3, it can not set above permission, because in s3, all files are 
listed as having full read/write permissions and all directories appear to have 
full rwx permissions. See Object stores have differerent authorization models 
in 
https://hadoop.apache.org/docs/current3/hadoop-aws/tools/hadoop-aws/index.html




> Regionserver down when checking the permission of staging dir if 
> hbase.rootdir is on S3
> ---
>
> Key: HBASE-17861
> URL: https://issues.apache.org/jira/browse/HBASE-17861
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 1.4.0
>Reporter: Yi Liang
>Assignee: Yi Liang
>  Labels: filesystem, s3, wal
> Fix For: 1.4.0
>
> Attachments: HBASE-17861.branch-1.V1.patch, HBASE-17861-V1.patch
>
>
> Found some issue, when set up HBASE-17437: Support specifying a WAL directory 
> outside of the root directory.
> The region server are  showdown when I add following config into 
> hbase-site.xml 
> hbase.rootdir =  s3a://xx//xx
> hbase.wal.dir = hdfs://xx/xx
> hbase.coprocessor.region.classes = 
> org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint
> Error is below
> {noformat}
> org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint threw 
> java.lang.IllegalStateException: Directory already exists but permissions 
> aren't set to '-rwx--x--x'
> {noformat}
> The reason is that, when hbase enable securebulkload, hbase will create a 
> folder in s3, it can not set above permission, because in s3, all files are 
> listed as having full read/write permissions and all directories appear to 
> have full rwx permissions. See Object stores have differerent authorization 
> models in 
> https://hadoop.apache.org/docs/current3/hadoop-aws/tools/hadoop-aws/index.html



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (HBASE-17861) Regionserver down when checking the permission of staging dir if hbase.rootdir is on S3

2017-03-31 Thread Yi Liang (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HBASE-17861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yi Liang updated HBASE-17861:
-
Attachment: HBASE-17861.branch-1.V1.patch

> Regionserver down when checking the permission of staging dir if 
> hbase.rootdir is on S3
> ---
>
> Key: HBASE-17861
> URL: https://issues.apache.org/jira/browse/HBASE-17861
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 1.4.0
>Reporter: Yi Liang
>Assignee: Yi Liang
>  Labels: filesystem, s3, wal
> Fix For: 1.4.0
>
> Attachments: HBASE-17861.branch-1.V1.patch, HBASE-17861-V1.patch
>
>
> Found some issue, when set up HBASE-17437: Support specifying a WAL directory 
> outside of the root directory.
> The region server are  showdown when I add following config into 
> hbase-site.xml 
> hbase.rootdir = hdfs://xx/xx
> hbase.wal.dir = s3a://xx//xx
> hbase.coprocessor.region.classes = 
> org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint
> Error is below
> {noformat}
> org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint threw 
> java.lang.IllegalStateException: Directory already exists but permissions 
> aren't set to '-rwx--x--x'
> {noformat}
> The reason is that, when hbase enable securebulkload, hbase will create a 
> folder in s3, it can not set above permission, because in s3, all files are 
> listed as having full read/write permissions and all directories appear to 
> have full rwx permissions. See Object stores have differerent authorization 
> models in 
> https://hadoop.apache.org/docs/current3/hadoop-aws/tools/hadoop-aws/index.html



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (HBASE-17861) Regionserver down when checking the permission of staging dir if hbase.rootdir is on S3

2017-03-31 Thread Yi Liang (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HBASE-17861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yi Liang updated HBASE-17861:
-
Description: 
Found some issue, when set up HBASE-17437: Support specifying a WAL directory 
outside of the root directory.

The region server are  showdown when I add following config into hbase-site.xml 
hbase.rootdir = hdfs://xx/xx
hbase.wal.dir = s3a://xx//xx
hbase.coprocessor.region.classes = 
org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint
Error is below
{noformat}
org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint threw 
java.lang.IllegalStateException: Directory already exists but permissions 
aren't set to '-rwx--x--x'
{noformat}

The reason is that, when hbase enable securebulkload, hbase will create a 
folder in s3, it can not set above permission, because in s3, all files are 
listed as having full read/write permissions and all directories appear to have 
full rwx permissions. See Object stores have differerent authorization models 
in 
https://hadoop.apache.org/docs/current3/hadoop-aws/tools/hadoop-aws/index.html



  was:
Found some issue, when set up HBASE-17437: Support specifying a WAL directory 
outside of the root directory.

The region server are  showdown when I add following config into hbase-site.xml 
hbase.rootdir = hdfs://xx/xx
hbase.wal.dir = s3a://xx//xx
hbase.coprocessor.region.classes = 
org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint
Error is below
{noformat}
org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint threw 
java.lang.IllegalStateException: Directory already exists but permissions 
aren't set to '-rwx--x--x'
{noformat}

The reason is that, when hbase enable securebulkload, hbase will create a 
folder in s3, it can not set above permission, because in s3, all files are 
listed as having full read/write permissions and all directories appear to have 
full rwx permissions.  See simulated permission section in 
https://docs.hortonworks.com/HDPDocuments/HDCloudAWS/HDCloudAWS-1.8.0/bk_hdcloud-aws/content/s3-s3aclient/index.html




> Regionserver down when checking the permission of staging dir if 
> hbase.rootdir is on S3
> ---
>
> Key: HBASE-17861
> URL: https://issues.apache.org/jira/browse/HBASE-17861
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 1.4.0
>Reporter: Yi Liang
>Assignee: Yi Liang
>  Labels: filesystem, s3, wal
> Fix For: 1.4.0
>
> Attachments: HBASE-17861-V1.patch
>
>
> Found some issue, when set up HBASE-17437: Support specifying a WAL directory 
> outside of the root directory.
> The region server are  showdown when I add following config into 
> hbase-site.xml 
> hbase.rootdir = hdfs://xx/xx
> hbase.wal.dir = s3a://xx//xx
> hbase.coprocessor.region.classes = 
> org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint
> Error is below
> {noformat}
> org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint threw 
> java.lang.IllegalStateException: Directory already exists but permissions 
> aren't set to '-rwx--x--x'
> {noformat}
> The reason is that, when hbase enable securebulkload, hbase will create a 
> folder in s3, it can not set above permission, because in s3, all files are 
> listed as having full read/write permissions and all directories appear to 
> have full rwx permissions. See Object stores have differerent authorization 
> models in 
> https://hadoop.apache.org/docs/current3/hadoop-aws/tools/hadoop-aws/index.html



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (HBASE-17861) Regionserver down when checking the permission of staging dir if hbase.rootdir is on S3

2017-03-31 Thread Yi Liang (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HBASE-17861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yi Liang updated HBASE-17861:
-
Status: Patch Available  (was: Open)

> Regionserver down when checking the permission of staging dir if 
> hbase.rootdir is on S3
> ---
>
> Key: HBASE-17861
> URL: https://issues.apache.org/jira/browse/HBASE-17861
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 1.4.0
>Reporter: Yi Liang
>Assignee: Yi Liang
>  Labels: filesystem, s3, wal
> Fix For: 1.4.0
>
> Attachments: HBASE-17861-V1.patch
>
>
> Found some issue, when set up HBASE-17437: Support specifying a WAL directory 
> outside of the root directory.
> The region server are  showdown when I add following config into 
> hbase-site.xml 
> hbase.rootdir = hdfs://xx/xx
> hbase.wal.dir = s3a://xx//xx
> hbase.coprocessor.region.classes = 
> org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint
> Error is below
> {noformat}
> org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint threw 
> java.lang.IllegalStateException: Directory already exists but permissions 
> aren't set to '-rwx--x--x'
> {noformat}
> The reason is that, when hbase enable securebulkload, hbase will create a 
> folder in s3, it can not set above permission, because in s3, all files are 
> listed as having full read/write permissions and all directories appear to 
> have full rwx permissions.  See simulated permission section in 
> https://docs.hortonworks.com/HDPDocuments/HDCloudAWS/HDCloudAWS-1.8.0/bk_hdcloud-aws/content/s3-s3aclient/index.html



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (HBASE-17861) Regionserver down when checking the permission of staging dir if hbase.rootdir is on S3

2017-03-31 Thread Yi Liang (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HBASE-17861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yi Liang updated HBASE-17861:
-
Labels: filesystem s3 wal  (was: )

> Regionserver down when checking the permission of staging dir if 
> hbase.rootdir is on S3
> ---
>
> Key: HBASE-17861
> URL: https://issues.apache.org/jira/browse/HBASE-17861
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 1.4.0
>Reporter: Yi Liang
>Assignee: Yi Liang
>  Labels: filesystem, s3, wal
> Fix For: 1.4.0
>
> Attachments: HBASE-17861-V1.patch
>
>
> Found some issue, when set up HBASE-17437: Support specifying a WAL directory 
> outside of the root directory.
> The region server are  showdown when I add following config into 
> hbase-site.xml 
> hbase.rootdir = hdfs://xx/xx
> hbase.wal.dir = s3a://xx//xx
> hbase.coprocessor.region.classes = 
> org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint
> Error is below
> {noformat}
> org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint threw 
> java.lang.IllegalStateException: Directory already exists but permissions 
> aren't set to '-rwx--x--x'
> {noformat}
> The reason is that, when hbase enable securebulkload, hbase will create a 
> folder in s3, it can not set above permission, because in s3, all files are 
> listed as having full read/write permissions and all directories appear to 
> have full rwx permissions.  See simulated permission section in 
> https://docs.hortonworks.com/HDPDocuments/HDCloudAWS/HDCloudAWS-1.8.0/bk_hdcloud-aws/content/s3-s3aclient/index.html



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (HBASE-17861) Regionserver down when checking the permission of staging dir if hbase.rootdir is on S3

2017-03-31 Thread Yi Liang (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HBASE-17861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yi Liang updated HBASE-17861:
-
Attachment: (was: HBase-17821-V1.patch)

> Regionserver down when checking the permission of staging dir if 
> hbase.rootdir is on S3
> ---
>
> Key: HBASE-17861
> URL: https://issues.apache.org/jira/browse/HBASE-17861
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 1.4.0
>Reporter: Yi Liang
>Assignee: Yi Liang
> Fix For: 1.4.0
>
> Attachments: HBASE-17861-V1.patch
>
>
> Found some issue, when set up HBASE-17437: Support specifying a WAL directory 
> outside of the root directory.
> The region server are  showdown when I add following config into 
> hbase-site.xml 
> hbase.rootdir = hdfs://xx/xx
> hbase.wal.dir = s3a://xx//xx
> hbase.coprocessor.region.classes = 
> org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint
> Error is below
> {noformat}
> org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint threw 
> java.lang.IllegalStateException: Directory already exists but permissions 
> aren't set to '-rwx--x--x'
> {noformat}
> The reason is that, when hbase enable securebulkload, hbase will create a 
> folder in s3, it can not set above permission, because in s3, all files are 
> listed as having full read/write permissions and all directories appear to 
> have full rwx permissions.  See simulated permission section in 
> https://docs.hortonworks.com/HDPDocuments/HDCloudAWS/HDCloudAWS-1.8.0/bk_hdcloud-aws/content/s3-s3aclient/index.html



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (HBASE-17861) Regionserver down when checking the permission of staging dir if hbase.rootdir is on S3

2017-03-31 Thread Yi Liang (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HBASE-17861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yi Liang updated HBASE-17861:
-
Attachment: HBASE-17861-V1.patch
HBase-17821-V1.patch

> Regionserver down when checking the permission of staging dir if 
> hbase.rootdir is on S3
> ---
>
> Key: HBASE-17861
> URL: https://issues.apache.org/jira/browse/HBASE-17861
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 1.4.0
>Reporter: Yi Liang
>Assignee: Yi Liang
> Fix For: 1.4.0
>
> Attachments: HBASE-17861-V1.patch
>
>
> Found some issue, when set up HBASE-17437: Support specifying a WAL directory 
> outside of the root directory.
> The region server are  showdown when I add following config into 
> hbase-site.xml 
> hbase.rootdir = hdfs://xx/xx
> hbase.wal.dir = s3a://xx//xx
> hbase.coprocessor.region.classes = 
> org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint
> Error is below
> {noformat}
> org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint threw 
> java.lang.IllegalStateException: Directory already exists but permissions 
> aren't set to '-rwx--x--x'
> {noformat}
> The reason is that, when hbase enable securebulkload, hbase will create a 
> folder in s3, it can not set above permission, because in s3, all files are 
> listed as having full read/write permissions and all directories appear to 
> have full rwx permissions.  See simulated permission section in 
> https://docs.hortonworks.com/HDPDocuments/HDCloudAWS/HDCloudAWS-1.8.0/bk_hdcloud-aws/content/s3-s3aclient/index.html



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (HBASE-17861) Regionserver down when checking the permission of staging dir if hbase.rootdir is on S3

2017-03-31 Thread Yi Liang (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HBASE-17861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yi Liang updated HBASE-17861:
-
Affects Version/s: 1.4.0

> Regionserver down when checking the permission of staging dir if 
> hbase.rootdir is on S3
> ---
>
> Key: HBASE-17861
> URL: https://issues.apache.org/jira/browse/HBASE-17861
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 1.4.0
>Reporter: Yi Liang
>Assignee: Yi Liang
> Fix For: 1.4.0
>
>
> Found some issue, when set up HBASE-17437: Support specifying a WAL directory 
> outside of the root directory.
> The region server are  showdown when I add following config into 
> hbase-site.xml 
> hbase.rootdir = hdfs://xx/xx
> hbase.wal.dir = s3a://xx//xx
> hbase.coprocessor.region.classes = 
> org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint
> Error is below
> {noformat}
> org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint threw 
> java.lang.IllegalStateException: Directory already exists but permissions 
> aren't set to '-rwx--x--x'
> {noformat}
> The reason is that, when hbase enable securebulkload, hbase will create a 
> folder in s3, it can not set above permission, because in s3, all files are 
> listed as having full read/write permissions and all directories appear to 
> have full rwx permissions.  See simulated permission section in 
> https://docs.hortonworks.com/HDPDocuments/HDCloudAWS/HDCloudAWS-1.8.0/bk_hdcloud-aws/content/s3-s3aclient/index.html



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (HBASE-17861) Regionserver down when checking the permission of staging dir if hbase.rootdir is on S3

2017-03-31 Thread Yi Liang (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HBASE-17861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yi Liang updated HBASE-17861:
-
Fix Version/s: 1.4.0

> Regionserver down when checking the permission of staging dir if 
> hbase.rootdir is on S3
> ---
>
> Key: HBASE-17861
> URL: https://issues.apache.org/jira/browse/HBASE-17861
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 1.4.0
>Reporter: Yi Liang
>Assignee: Yi Liang
> Fix For: 1.4.0
>
>
> Found some issue, when set up HBASE-17437: Support specifying a WAL directory 
> outside of the root directory.
> The region server are  showdown when I add following config into 
> hbase-site.xml 
> hbase.rootdir = hdfs://xx/xx
> hbase.wal.dir = s3a://xx//xx
> hbase.coprocessor.region.classes = 
> org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint
> Error is below
> {noformat}
> org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint threw 
> java.lang.IllegalStateException: Directory already exists but permissions 
> aren't set to '-rwx--x--x'
> {noformat}
> The reason is that, when hbase enable securebulkload, hbase will create a 
> folder in s3, it can not set above permission, because in s3, all files are 
> listed as having full read/write permissions and all directories appear to 
> have full rwx permissions.  See simulated permission section in 
> https://docs.hortonworks.com/HDPDocuments/HDCloudAWS/HDCloudAWS-1.8.0/bk_hdcloud-aws/content/s3-s3aclient/index.html



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)