[jira] [Commented] (HBASE-15790) Force "hbase" ownership on bulkload
[ https://issues.apache.org/jira/browse/HBASE-15790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15303197#comment-15303197 ] Matteo Bertozzi commented on HBASE-15790: - there are so many combinations where you can have bulk load succeed. the simples way if you want to try is have files owned by a user and then have hdfs call bulkload. file will be moved without problems and the permission will be the wrong one > Force "hbase" ownership on bulkload > --- > > Key: HBASE-15790 > URL: https://issues.apache.org/jira/browse/HBASE-15790 > Project: HBase > Issue Type: Bug > Components: regionserver >Affects Versions: 2.0.0, 1.2.1, 1.1.4, 0.98.19 >Reporter: Matteo Bertozzi >Assignee: Matteo Bertozzi >Priority: Minor > Attachments: HBASE-15790-v0.patch, HBASE-15790-v1.patch, > HBASE-15790-v2.patch, HBASE-15790-v3.patch > > > When a user different than "hbase" bulkload files, in general we end up with > files owned by a user different than hbase. sometimes this causes problems > with hbase not be able to move files around archiving/deleting. > A simple solution is probably to change the ownership of the files to "hbase" > during bulkload. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15790) Force "hbase" ownership on bulkload
[ https://issues.apache.org/jira/browse/HBASE-15790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15303181#comment-15303181 ] Jerry He commented on HBASE-15790: -- Hmm. Then how could the files be moved into hbase by bulk load? Would the bulk load fail in this case? If it is rename, hbase needs write permission on the files. If it is copy, hbase needs read, but then copy does change the owner. > Force "hbase" ownership on bulkload > --- > > Key: HBASE-15790 > URL: https://issues.apache.org/jira/browse/HBASE-15790 > Project: HBase > Issue Type: Bug > Components: regionserver >Affects Versions: 2.0.0, 1.2.1, 1.1.4, 0.98.19 >Reporter: Matteo Bertozzi >Assignee: Matteo Bertozzi >Priority: Minor > Attachments: HBASE-15790-v0.patch, HBASE-15790-v1.patch, > HBASE-15790-v2.patch, HBASE-15790-v3.patch > > > When a user different than "hbase" bulkload files, in general we end up with > files owned by a user different than hbase. sometimes this causes problems > with hbase not be able to move files around archiving/deleting. > A simple solution is probably to change the ownership of the files to "hbase" > during bulkload. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15790) Force "hbase" ownership on bulkload
[ https://issues.apache.org/jira/browse/HBASE-15790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15303145#comment-15303145 ] Matteo Bertozzi commented on HBASE-15790: - the problem we have is that people does not use SecureBulkLoad and just bulkload the files as normal users. but the files may be owned by foo with permission rw- and those file will be moved into hbase. and hbase explodes when it tries to move the files away for example after a compaction. > Force "hbase" ownership on bulkload > --- > > Key: HBASE-15790 > URL: https://issues.apache.org/jira/browse/HBASE-15790 > Project: HBase > Issue Type: Bug > Components: regionserver >Affects Versions: 2.0.0, 1.2.1, 1.1.4, 0.98.19 >Reporter: Matteo Bertozzi >Assignee: Matteo Bertozzi >Priority: Minor > Attachments: HBASE-15790-v0.patch, HBASE-15790-v1.patch, > HBASE-15790-v2.patch, HBASE-15790-v3.patch > > > When a user different than "hbase" bulkload files, in general we end up with > files owned by a user different than hbase. sometimes this causes problems > with hbase not be able to move files around archiving/deleting. > A simple solution is probably to change the ownership of the files to "hbase" > during bulkload. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15790) Force "hbase" ownership on bulkload
[ https://issues.apache.org/jira/browse/HBASE-15790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15303139#comment-15303139 ] Jerry He commented on HBASE-15790: -- Could you give a specific example of the problem on the permissions from hbase bulk load. Yes. The SecureBulkLoad is the way to go. I have a JIRA to get it unified and be default HBASE-13701. I think I should start working on it ... > Force "hbase" ownership on bulkload > --- > > Key: HBASE-15790 > URL: https://issues.apache.org/jira/browse/HBASE-15790 > Project: HBase > Issue Type: Bug > Components: regionserver >Affects Versions: 2.0.0, 1.2.1, 1.1.4, 0.98.19 >Reporter: Matteo Bertozzi >Assignee: Matteo Bertozzi >Priority: Minor > Attachments: HBASE-15790-v0.patch, HBASE-15790-v1.patch, > HBASE-15790-v2.patch, HBASE-15790-v3.patch > > > When a user different than "hbase" bulkload files, in general we end up with > files owned by a user different than hbase. sometimes this causes problems > with hbase not be able to move files around archiving/deleting. > A simple solution is probably to change the ownership of the files to "hbase" > during bulkload. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15790) Force "hbase" ownership on bulkload
[ https://issues.apache.org/jira/browse/HBASE-15790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15302724#comment-15302724 ] Matteo Bertozzi commented on HBASE-15790: - yeah, it will throw exception. any idea on how change it? I basically want to prevent the ability to bulkload files hbase is not able to interact with. as enis mentioned SecureBulkLoad is the way to go since it works in both secure and unsecure way, but some people still don't use it since it is something that we still have to enable by hand. > Force "hbase" ownership on bulkload > --- > > Key: HBASE-15790 > URL: https://issues.apache.org/jira/browse/HBASE-15790 > Project: HBase > Issue Type: Bug > Components: regionserver >Affects Versions: 2.0.0, 1.2.1, 1.1.4, 0.98.19 >Reporter: Matteo Bertozzi >Assignee: Matteo Bertozzi >Priority: Minor > Attachments: HBASE-15790-v0.patch, HBASE-15790-v1.patch, > HBASE-15790-v2.patch, HBASE-15790-v3.patch > > > When a user different than "hbase" bulkload files, in general we end up with > files owned by a user different than hbase. sometimes this causes problems > with hbase not be able to move files around archiving/deleting. > A simple solution is probably to change the ownership of the files to "hbase" > during bulkload. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15790) Force "hbase" ownership on bulkload
[ https://issues.apache.org/jira/browse/HBASE-15790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15302690#comment-15302690 ] Jerry He commented on HBASE-15790: -- On patch-v3, checking 777 is probably not good. For example, the user (say 'hive') and 'hbase' are in the same group, and the permission is rw for group. This case works currently. But after v3, we will throw exception? > Force "hbase" ownership on bulkload > --- > > Key: HBASE-15790 > URL: https://issues.apache.org/jira/browse/HBASE-15790 > Project: HBase > Issue Type: Bug > Components: regionserver >Affects Versions: 2.0.0, 1.2.1, 1.1.4, 0.98.19 >Reporter: Matteo Bertozzi >Assignee: Matteo Bertozzi >Priority: Minor > Attachments: HBASE-15790-v0.patch, HBASE-15790-v1.patch, > HBASE-15790-v2.patch, HBASE-15790-v3.patch > > > When a user different than "hbase" bulkload files, in general we end up with > files owned by a user different than hbase. sometimes this causes problems > with hbase not be able to move files around archiving/deleting. > A simple solution is probably to change the ownership of the files to "hbase" > during bulkload. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15790) Force "hbase" ownership on bulkload
[ https://issues.apache.org/jira/browse/HBASE-15790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15302618#comment-15302618 ] Enis Soztutar commented on HBASE-15790: --- bq. SecureBulkLoadEndpoint seems to always do the chmod 777. so we are good in that case even if the files are owned by someone that is not hbase. There was an issue to make SBLE to be the default way for BL even in unsecure setups. Even in unsecure setups, the SBLE does the "correct" thing in terms of passing the file down to RS. Maybe we should finish the jira that wanted to refactor SBLE to be the default method (meanly replacing the need for the coprocessor). > Force "hbase" ownership on bulkload > --- > > Key: HBASE-15790 > URL: https://issues.apache.org/jira/browse/HBASE-15790 > Project: HBase > Issue Type: Bug > Components: regionserver >Affects Versions: 2.0.0, 1.2.1, 1.1.4, 0.98.19 >Reporter: Matteo Bertozzi >Assignee: Matteo Bertozzi >Priority: Minor > Attachments: HBASE-15790-v0.patch, HBASE-15790-v1.patch, > HBASE-15790-v2.patch, HBASE-15790-v3.patch > > > When a user different than "hbase" bulkload files, in general we end up with > files owned by a user different than hbase. sometimes this causes problems > with hbase not be able to move files around archiving/deleting. > A simple solution is probably to change the ownership of the files to "hbase" > during bulkload. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15790) Force "hbase" ownership on bulkload
[
https://issues.apache.org/jira/browse/HBASE-15790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15297603#comment-15297603
]
Hadoop QA commented on HBASE-15790:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 12s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
56s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 4s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 48s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
47s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
24s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
36s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 52s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 50s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 9s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
59s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 4s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 4s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 50s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 50s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
50s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
24s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 9m
5s {color} | {color:green} Patch does not cause any errors with Hadoop 2.4.1
2.5.2 2.6.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
10s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 59s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 51s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 45s
{color} | {color:green} hbase-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 102m 31s
{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
27s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 134m 18s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12805789/HBASE-15790-v3.patch |
| JIRA Issue | HBASE-15790 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux asf907.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
[jira] [Commented] (HBASE-15790) Force "hbase" ownership on bulkload
[ https://issues.apache.org/jira/browse/HBASE-15790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15295407#comment-15295407 ] Jerry He commented on HBASE-15790: -- Ok. The UT can be explained. The build id starts the hdfs minicluster. It is the hdfs superuser already. > Force "hbase" ownership on bulkload > --- > > Key: HBASE-15790 > URL: https://issues.apache.org/jira/browse/HBASE-15790 > Project: HBase > Issue Type: Bug > Components: regionserver >Affects Versions: 2.0.0, 1.2.1, 1.1.4, 0.98.19 >Reporter: Matteo Bertozzi >Assignee: Matteo Bertozzi >Priority: Minor > Attachments: HBASE-15790-v0.patch, HBASE-15790-v1.patch, > HBASE-15790-v2.patch > > > When a user different than "hbase" bulkload files, in general we end up with > files owned by a user different than hbase. sometimes this causes problems > with hbase not be able to move files around archiving/deleting. > A simple solution is probably to change the ownership of the files to "hbase" > during bulkload. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15790) Force "hbase" ownership on bulkload
[ https://issues.apache.org/jira/browse/HBASE-15790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15295406#comment-15295406 ] Jerry He commented on HBASE-15790: -- Hmm. My impression is that only hdfs superuser can chown, or the root in the old Unix world, with some specific exceptions. But your UT passed ... > Force "hbase" ownership on bulkload > --- > > Key: HBASE-15790 > URL: https://issues.apache.org/jira/browse/HBASE-15790 > Project: HBase > Issue Type: Bug > Components: regionserver >Affects Versions: 2.0.0, 1.2.1, 1.1.4, 0.98.19 >Reporter: Matteo Bertozzi >Assignee: Matteo Bertozzi >Priority: Minor > Attachments: HBASE-15790-v0.patch, HBASE-15790-v1.patch, > HBASE-15790-v2.patch > > > When a user different than "hbase" bulkload files, in general we end up with > files owned by a user different than hbase. sometimes this causes problems > with hbase not be able to move files around archiving/deleting. > A simple solution is probably to change the ownership of the files to "hbase" > during bulkload. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15790) Force "hbase" ownership on bulkload
[ https://issues.apache.org/jira/browse/HBASE-15790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15295389#comment-15295389 ] Matteo Bertozzi commented on HBASE-15790: - why hdfs super user? does set owner has some specific permission requirement that I'm not aware of? the process should just be able to chown user:group that the /hbase dir has and that the files the process creates have > Force "hbase" ownership on bulkload > --- > > Key: HBASE-15790 > URL: https://issues.apache.org/jira/browse/HBASE-15790 > Project: HBase > Issue Type: Bug > Components: regionserver >Affects Versions: 2.0.0, 1.2.1, 1.1.4, 0.98.19 >Reporter: Matteo Bertozzi >Assignee: Matteo Bertozzi >Priority: Minor > Attachments: HBASE-15790-v0.patch, HBASE-15790-v1.patch, > HBASE-15790-v2.patch > > > When a user different than "hbase" bulkload files, in general we end up with > files owned by a user different than hbase. sometimes this causes problems > with hbase not be able to move files around archiving/deleting. > A simple solution is probably to change the ownership of the files to "hbase" > during bulkload. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15790) Force "hbase" ownership on bulkload
[ https://issues.apache.org/jira/browse/HBASE-15790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15295388#comment-15295388 ] Jerry He commented on HBASE-15790: -- Just thought of another question. Does this mean 'hbase' has to be hdfs superuser now? To use setOwner(). > Force "hbase" ownership on bulkload > --- > > Key: HBASE-15790 > URL: https://issues.apache.org/jira/browse/HBASE-15790 > Project: HBase > Issue Type: Bug > Components: regionserver >Affects Versions: 2.0.0, 1.2.1, 1.1.4, 0.98.19 >Reporter: Matteo Bertozzi >Assignee: Matteo Bertozzi >Priority: Minor > Attachments: HBASE-15790-v0.patch, HBASE-15790-v1.patch, > HBASE-15790-v2.patch > > > When a user different than "hbase" bulkload files, in general we end up with > files owned by a user different than hbase. sometimes this causes problems > with hbase not be able to move files around archiving/deleting. > A simple solution is probably to change the ownership of the files to "hbase" > during bulkload. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15790) Force "hbase" ownership on bulkload
[
https://issues.apache.org/jira/browse/HBASE-15790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15294676#comment-15294676
]
Hadoop QA commented on HBASE-15790:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 23s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
21s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 6s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 51s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
48s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
27s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
51s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 54s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 50s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 10s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
1s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 11s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 11s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 55s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 55s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
49s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
27s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 9m
38s {color} | {color:green} Patch does not cause any errors with Hadoop 2.4.1
2.5.2 2.6.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
30s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 52s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 56s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 52s
{color} | {color:green} hbase-common in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 100m 9s {color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
27s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 134m 16s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.regionserver.TestRegionServerMetrics |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12804851/HBASE-15790-v2.patch |
| JIRA Issue | HBASE-15790 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux asf900.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014
[jira] [Commented] (HBASE-15790) Force "hbase" ownership on bulkload
[ https://issues.apache.org/jira/browse/HBASE-15790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15294618#comment-15294618 ] Enis Soztutar commented on HBASE-15790: --- Does this work with the SecureBLE and the permission switching there? If so +1. > Force "hbase" ownership on bulkload > --- > > Key: HBASE-15790 > URL: https://issues.apache.org/jira/browse/HBASE-15790 > Project: HBase > Issue Type: Bug > Components: regionserver >Affects Versions: 2.0.0, 1.2.1, 1.1.4, 0.98.19 >Reporter: Matteo Bertozzi >Assignee: Matteo Bertozzi >Priority: Minor > Attachments: HBASE-15790-v0.patch, HBASE-15790-v1.patch, > HBASE-15790-v2.patch > > > When a user different than "hbase" bulkload files, in general we end up with > files owned by a user different than hbase. sometimes this causes problems > with hbase not be able to move files around archiving/deleting. > A simple solution is probably to change the ownership of the files to "hbase" > during bulkload. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15790) Force "hbase" ownership on bulkload
[ https://issues.apache.org/jira/browse/HBASE-15790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15294614#comment-15294614 ] stack commented on HBASE-15790: --- +1 Maybe add comment on your trickery getting group (as per discussion off list). The failed test is unrelated I'd say. It fail for you local? Something weird related in a servlet setup? > Force "hbase" ownership on bulkload > --- > > Key: HBASE-15790 > URL: https://issues.apache.org/jira/browse/HBASE-15790 > Project: HBase > Issue Type: Bug > Components: regionserver >Affects Versions: 2.0.0, 1.2.1, 1.1.4, 0.98.19 >Reporter: Matteo Bertozzi >Assignee: Matteo Bertozzi >Priority: Minor > Attachments: HBASE-15790-v0.patch, HBASE-15790-v1.patch, > HBASE-15790-v2.patch > > > When a user different than "hbase" bulkload files, in general we end up with > files owned by a user different than hbase. sometimes this causes problems > with hbase not be able to move files around archiving/deleting. > A simple solution is probably to change the ownership of the files to "hbase" > during bulkload. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15790) Force "hbase" ownership on bulkload
[
https://issues.apache.org/jira/browse/HBASE-15790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15290608#comment-15290608
]
Hadoop QA commented on HBASE-15790:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 19s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
21s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 11s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 51s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
55s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
26s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
39s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 54s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 50s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 10s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
7s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 5s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 5s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 50s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 50s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
46s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
25s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 9m
24s {color} | {color:green} Patch does not cause any errors with Hadoop 2.4.1
2.5.2 2.6.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
17s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 52s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 51s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 47s
{color} | {color:green} hbase-common in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 32m 33s {color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
20s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 65m 42s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Timed out junit tests | org.apache.hadoop.hbase.http.TestHttpServer |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12804851/HBASE-15790-v2.patch |
| JIRA Issue | HBASE-15790 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux penates.apache.org 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT
Wed Sep 3 21:56:12 UTC 2014 x86_64
[jira] [Commented] (HBASE-15790) Force "hbase" ownership on bulkload
[
https://issues.apache.org/jira/browse/HBASE-15790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15283417#comment-15283417
]
Hadoop QA commented on HBASE-15790:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 9s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
55s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 10s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 48s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
45s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
27s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
41s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 57s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 52s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 9s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
1s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 6s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 6s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 50s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 50s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
44s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
24s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 8m
49s {color} | {color:green} Patch does not cause any errors with Hadoop 2.4.1
2.5.2 2.6.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
17s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 55s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 50s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 43s
{color} | {color:green} hbase-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 106m 14s
{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
26s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 137m 54s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12803994/HBASE-15790-v1.patch |
| JIRA Issue | HBASE-15790 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux asf900.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
[jira] [Commented] (HBASE-15790) Force "hbase" ownership on bulkload
[ https://issues.apache.org/jira/browse/HBASE-15790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15274795#comment-15274795 ] Jerry He commented on HBASE-15790: -- lgtm > Force "hbase" ownership on bulkload > --- > > Key: HBASE-15790 > URL: https://issues.apache.org/jira/browse/HBASE-15790 > Project: HBase > Issue Type: Bug > Components: regionserver >Affects Versions: 2.0.0, 1.2.1, 1.1.4, 0.98.19 >Reporter: Matteo Bertozzi >Assignee: Matteo Bertozzi >Priority: Minor > Attachments: HBASE-15790-v0.patch > > > When a user different than "hbase" bulkload files, in general we end up with > files owned by a user different than hbase. sometimes this causes problems > with hbase not be able to move files around archiving/deleting. > A simple solution is probably to change the ownership of the files to "hbase" > during bulkload. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15790) Force "hbase" ownership on bulkload
[ https://issues.apache.org/jira/browse/HBASE-15790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15274684#comment-15274684 ] Jonathan Hsieh commented on HBASE-15790: makes sense. > Force "hbase" ownership on bulkload > --- > > Key: HBASE-15790 > URL: https://issues.apache.org/jira/browse/HBASE-15790 > Project: HBase > Issue Type: Bug > Components: regionserver >Affects Versions: 2.0.0, 1.2.1, 1.1.4, 0.98.19 >Reporter: Matteo Bertozzi >Assignee: Matteo Bertozzi >Priority: Minor > Attachments: HBASE-15790-v0.patch > > > When a user different than "hbase" bulkload files, in general we end up with > files owned by a user different than hbase. sometimes this causes problems > with hbase not be able to move files around archiving/deleting. > A simple solution is probably to change the ownership of the files to "hbase" > during bulkload. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15790) Force "hbase" ownership on bulkload
[ https://issues.apache.org/jira/browse/HBASE-15790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15274644#comment-15274644 ] Matteo Bertozzi commented on HBASE-15790: - the bulkload fail, the file is in temp. nothing committed yet. this call is before commit. > Force "hbase" ownership on bulkload > --- > > Key: HBASE-15790 > URL: https://issues.apache.org/jira/browse/HBASE-15790 > Project: HBase > Issue Type: Bug > Components: regionserver >Affects Versions: 2.0.0, 1.2.1, 1.1.4, 0.98.19 >Reporter: Matteo Bertozzi >Assignee: Matteo Bertozzi >Priority: Minor > Attachments: HBASE-15790-v0.patch > > > When a user different than "hbase" bulkload files, in general we end up with > files owned by a user different than hbase. sometimes this causes problems > with hbase not be able to move files around archiving/deleting. > A simple solution is probably to change the ownership of the files to "hbase" > during bulkload. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15790) Force "hbase" ownership on bulkload
[
https://issues.apache.org/jira/browse/HBASE-15790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15274640#comment-15274640
]
Jonathan Hsieh commented on HBASE-15790:
lgtm if the bot passes.
Seems like there could be rare race between 497/498 if there was a crash.
Probably not worth fixing, but might be worth filing a follow up for hbck to
check file owners/groups and perms to find things that seem strange. Wdyt?
{code}
public Void run() throws Exception {
496 FsPermission perms = FSUtils.getFilePermissions(fs, conf,
HConstants.DATA_FILE_UMASK_KEY);
497 fs.setOwner(dstPath, filesOwner, filesGroup);
498 fs.setPermission(dstPath, perms);
499 return null;
500 }
{code}
> Force "hbase" ownership on bulkload
> ---
>
> Key: HBASE-15790
> URL: https://issues.apache.org/jira/browse/HBASE-15790
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 2.0.0, 1.2.1, 1.1.4, 0.98.19
>Reporter: Matteo Bertozzi
>Assignee: Matteo Bertozzi
>Priority: Minor
> Attachments: HBASE-15790-v0.patch
>
>
> When a user different than "hbase" bulkload files, in general we end up with
> files owned by a user different than hbase. sometimes this causes problems
> with hbase not be able to move files around archiving/deleting.
> A simple solution is probably to change the ownership of the files to "hbase"
> during bulkload.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
