[
https://issues.apache.org/jira/browse/HBASE-17710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15889533#comment-15889533
]
Hadoop QA commented on HBASE-17710:
-----------------------------------
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 13s
{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} patch {color} | {color:blue} 0m 4s
{color} | {color:blue} The patch file was not named according to hbase's naming
conventions. Please see
https://yetus.apache.org/documentation/0.3.0/precommit-patchnames for
instructions. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 1m 16s
{color} | {color:red} root in branch-1 failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m 20s
{color} | {color:red} hbase-server in branch-1 failed with JDK v1.8.0_121.
{color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m 25s
{color} | {color:red} hbase-server in branch-1 failed with JDK v1.7.0_80.
{color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
55s {color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
16s {color} | {color:green} branch-1 passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m 21s
{color} | {color:red} hbase-server in branch-1 failed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 25s
{color} | {color:green} branch-1 passed with JDK v1.8.0_121 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 33s
{color} | {color:green} branch-1 passed with JDK v1.7.0_80 {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m 26s
{color} | {color:red} hbase-server in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m 20s
{color} | {color:red} hbase-server in the patch failed with JDK v1.8.0_121.
{color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 20s {color}
| {color:red} hbase-server in the patch failed with JDK v1.8.0_121. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m 25s
{color} | {color:red} hbase-server in the patch failed with JDK v1.7.0_80.
{color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 25s {color}
| {color:red} hbase-server in the patch failed with JDK v1.7.0_80. {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
56s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
15s {color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m 0s
{color} | {color:red} The patch has 1 line(s) that end in whitespace. Use git
apply --whitespace=fix. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 0m 48s
{color} | {color:red} The patch causes 33 errors with Hadoop v2.4.0. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 1m 36s
{color} | {color:red} The patch causes 33 errors with Hadoop v2.4.1. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 2m 24s
{color} | {color:red} The patch causes 33 errors with Hadoop v2.5.0. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 3m 11s
{color} | {color:red} The patch causes 33 errors with Hadoop v2.5.1. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 3m 59s
{color} | {color:red} The patch causes 33 errors with Hadoop v2.5.2. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 4m 47s
{color} | {color:red} The patch causes 33 errors with Hadoop v2.6.1. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 5m 34s
{color} | {color:red} The patch causes 33 errors with Hadoop v2.6.2. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 6m 21s
{color} | {color:red} The patch causes 33 errors with Hadoop v2.6.3. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 7m 9s
{color} | {color:red} The patch causes 33 errors with Hadoop v2.7.1. {color} |
| {color:red}-1{color} | {color:red} hbaseprotoc {color} | {color:red} 0m 22s
{color} | {color:red} hbase-server in the patch failed. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m 21s
{color} | {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 24s
{color} | {color:green} the patch passed with JDK v1.8.0_121 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 33s
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 0m 26s {color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
9s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 16m 48s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:e01ee2f |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12855292/17710.branch-1.v1.txt
|
| JIRA Issue | HBASE-17710 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux 8207808b7a28 3.13.0-107-generic #154-Ubuntu SMP Tue Dec 20
09:57:27 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/hbase.sh |
| git revision | branch-1 / 779f9cb |
| Default Java | 1.7.0_80 |
| Multi-JDK versions | /usr/lib/jvm/java-8-oracle:1.8.0_121
/usr/lib/jvm/java-7-oracle:1.7.0_80 |
| mvninstall |
https://builds.apache.org/job/PreCommit-HBASE-Build/5894/artifact/patchprocess/branch-mvninstall-root.txt
|
| compile |
https://builds.apache.org/job/PreCommit-HBASE-Build/5894/artifact/patchprocess/branch-compile-hbase-server-jdk1.8.0_121.txt
|
| compile |
https://builds.apache.org/job/PreCommit-HBASE-Build/5894/artifact/patchprocess/branch-compile-hbase-server-jdk1.7.0_80.txt
|
| findbugs |
https://builds.apache.org/job/PreCommit-HBASE-Build/5894/artifact/patchprocess/branch-findbugs-hbase-server.txt
|
| mvninstall |
https://builds.apache.org/job/PreCommit-HBASE-Build/5894/artifact/patchprocess/patch-mvninstall-hbase-server.txt
|
| compile |
https://builds.apache.org/job/PreCommit-HBASE-Build/5894/artifact/patchprocess/patch-compile-hbase-server-jdk1.8.0_121.txt
|
| javac |
https://builds.apache.org/job/PreCommit-HBASE-Build/5894/artifact/patchprocess/patch-compile-hbase-server-jdk1.8.0_121.txt
|
| compile |
https://builds.apache.org/job/PreCommit-HBASE-Build/5894/artifact/patchprocess/patch-compile-hbase-server-jdk1.7.0_80.txt
|
| javac |
https://builds.apache.org/job/PreCommit-HBASE-Build/5894/artifact/patchprocess/patch-compile-hbase-server-jdk1.7.0_80.txt
|
| whitespace |
https://builds.apache.org/job/PreCommit-HBASE-Build/5894/artifact/patchprocess/whitespace-eol.txt
|
| hbaseprotoc |
https://builds.apache.org/job/PreCommit-HBASE-Build/5894/artifact/patchprocess/patch-hbaseprotoc-hbase-server.txt
|
| findbugs |
https://builds.apache.org/job/PreCommit-HBASE-Build/5894/artifact/patchprocess/patch-findbugs-hbase-server.txt
|
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/5894/artifact/patchprocess/patch-unit-hbase-server.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/5894/testReport/ |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/5894/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> HBase in standalone mode creates directories with 777 permission
> ----------------------------------------------------------------
>
> Key: HBASE-17710
> URL: https://issues.apache.org/jira/browse/HBASE-17710
> Project: HBase
> Issue Type: Bug
> Components: regionserver
> Affects Versions: 1.1.2
> Environment: HDP-2.5.3
> Reporter: Toshihiro Suzuki
> Assignee: Ted Yu
> Attachments: 17710.branch-1.v1.txt, 17710.v1.txt
>
>
> HBase in standalone mode creates directories with 777 permission in
> hbase.rootdir.
> Ambari metrics collector defaults to standalone mode.
> {code}
> # find /var/lib/ambari-metrics-collector/hbase -perm 777 -type d -exec ls -ld
> {} \;
> drwxrwxrwx. 2 ams hadoop 6 Mar 1 02:17
> /var/lib/ambari-metrics-collector/hbase/data/hbase/namespace/d0cca53847904f4b4add1caa0ce3a9af/info
> drwxrwxrwx. 2 ams hadoop 6 Mar 1 02:17
> /var/lib/ambari-metrics-collector/hbase/data/hbase/backup/cbceb8fccd968b4b4583365d4dc6e377/meta
> drwxrwxrwx. 2 ams hadoop 6 Mar 1 02:17
> /var/lib/ambari-metrics-collector/hbase/data/hbase/backup/cbceb8fccd968b4b4583365d4dc6e377/session
> drwxrwxrwx. 2 ams hadoop 6 Mar 1 02:17
> /var/lib/ambari-metrics-collector/hbase/data/default/SYSTEM.CATALOG/2f4ce2294cd21cecb58fd1aca5646144/0
> drwxrwxrwx. 2 ams hadoop 6 Mar 1 02:17
> /var/lib/ambari-metrics-collector/hbase/data/default/SYSTEM.SEQUENCE/0eb67274ece8a4a26cfeeef2c6d4cd37/0
> drwxrwxrwx. 2 ams hadoop 6 Mar 1 02:17
> /var/lib/ambari-metrics-collector/hbase/data/default/SYSTEM.SEQUENCE/aef86710a4005f98e2dc90675f2eb325/0
> drwxrwxrwx. 2 ams hadoop 6 Mar 1 02:17
> /var/lib/ambari-metrics-collector/hbase/data/default/SYSTEM.STATS/5b1d955e255e55979621214a7e4083b8/0
> drwxrwxrwx. 2 ams hadoop 6 Mar 1 02:17
> /var/lib/ambari-metrics-collector/hbase/data/default/SYSTEM.FUNCTION/32c033735cf144bac5637de23f7f7dd0/0
> drwxrwxrwx. 2 ams hadoop 6 Mar 1 02:17
> /var/lib/ambari-metrics-collector/hbase/data/default/METRICS_METADATA/e420dfa799742fe4516ad1e4deefb793/0
> drwxrwxrwx. 2 ams hadoop 6 Mar 1 02:17
> /var/lib/ambari-metrics-collector/hbase/data/default/HOSTED_APPS_METADATA/110be63e2a9994121fc5b48d663daf2c/0
> drwxrwxrwx. 2 ams hadoop 6 Mar 1 02:17
> /var/lib/ambari-metrics-collector/hbase/data/default/CONTAINER_METRICS/a103719f87e8430635abf51a7fe98637/0
> drwxrwxrwx. 2 ams hadoop 6 Mar 1 02:17
> /var/lib/ambari-metrics-collector/hbase/data/default/METRIC_RECORD/cdb1d032beb90e350ce309e5d383c78e/0
> drwxrwxrwx. 2 ams hadoop 6 Mar 1 02:17
> /var/lib/ambari-metrics-collector/hbase/data/default/METRIC_RECORD/294deab47187494e845a5199702b4d04/0
> drwxrwxrwx. 2 ams hadoop 6 Mar 1 02:17
> /var/lib/ambari-metrics-collector/hbase/data/default/METRIC_RECORD_MINUTE/1a263b4fe068ef2db5ba1c3e45553354/0
> drwxrwxrwx. 2 ams hadoop 6 Mar 1 02:17
> /var/lib/ambari-metrics-collector/hbase/data/default/METRIC_RECORD_MINUTE/48f94dfb0161d8a28f645d2e1a473235/0
> drwxrwxrwx. 2 ams hadoop 6 Mar 1 02:17
> /var/lib/ambari-metrics-collector/hbase/data/default/METRIC_RECORD_HOURLY/6d096ac3e70e54dd4a8612e17cfc4b11/0
> drwxrwxrwx. 2 ams hadoop 6 Mar 1 02:17
> /var/lib/ambari-metrics-collector/hbase/data/default/METRIC_RECORD_DAILY/e81850d62da64c8d1c67be309f136e23/0
> drwxrwxrwx. 2 ams hadoop 45 Mar 1 02:21
> /var/lib/ambari-metrics-collector/hbase/data/default/METRIC_AGGREGATE/b43ff796de887197834ad62fdb612b59/0
> drwxrwxrwx. 2 ams hadoop 6 Mar 1 02:21
> /var/lib/ambari-metrics-collector/hbase/data/default/METRIC_AGGREGATE/b43ff796de887197834ad62fdb612b59/.tmp
> drwxrwxrwx. 2 ams hadoop 45 Mar 1 02:21
> /var/lib/ambari-metrics-collector/hbase/data/default/METRIC_AGGREGATE/c8eadeb7dead8fda9729b8e9b10c4929/0
> drwxrwxrwx. 2 ams hadoop 6 Mar 1 02:21
> /var/lib/ambari-metrics-collector/hbase/data/default/METRIC_AGGREGATE/c8eadeb7dead8fda9729b8e9b10c4929/.tmp
> drwxrwxrwx. 2 ams hadoop 6 Mar 1 02:17
> /var/lib/ambari-metrics-collector/hbase/data/default/METRIC_AGGREGATE_MINUTE/ca9f9754ae9ae4cdc3e1b0523eecc390/0
> drwxrwxrwx. 2 ams hadoop 6 Mar 1 02:18
> /var/lib/ambari-metrics-collector/hbase/data/default/METRIC_AGGREGATE_HOURLY/8412e8a8aec5d6307943fac78ce14c7a/0
> drwxrwxrwx. 2 ams hadoop 6 Mar 1 02:18
> /var/lib/ambari-metrics-collector/hbase/data/default/METRIC_AGGREGATE_DAILY/7c3358aba91ea0d76ddd8bc3ceb2d578/0
> {code}
> My analysis is as follows:
> FileSystem.mkdirs(Path f) method creates a directory with permission 777.
> Because HFileSystem which inherits FileSystem doesn't override the method,
> when we call HFileSystem.mkdirs(Path f), it tries to create a directory with
> permission 777.
> I've found that HFileSystem.mkdirs(Path f) gets called in the following areas:
> https://github.com/hortonworks/hbase/blob/HDP-2.5.3.0-37-tag/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java#L686
> https://github.com/hortonworks/hbase/blob/HDP-2.5.3.0-37-tag/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionFileSystem.java#L968
> At these times, the call path is as follows ("fs" is an instance of the
> abstract FileSystem class, being a field / member variable of
> FilterFileSystem):
> HFileSystem.mkdirs(Path f) -> FileSystem.mkdirs(Path f) ->
> FilterFileSystem.mkdirs(Path f, FsPermission permission) -> fs.mkdirs(Path f,
> FsPermission permission)
> For example:
> * In HRegionFileSystem.createDir(Path dir), fs.mkdirs(dir) is called.
> * Here "fs" is an instance of HFileSystem.
> * HFileSystem who inherits from FileSystem does not have an overridden
> implementation of mkdirs(dir).
> * Therefore, the parent FileSystem.mkdirs(Path f) is called.
> * FileSystem.mkdirs(Path f) calls FileSystem.mkdirs(f,
> FsPermission.getDirDefault()).
> ** FsPermission.getDirDefault() returns a default permission of 777 via this
> line:
> *** return new FsPermission((short)00777);
> * FileSystem.mkdirs(f, FsPermission.getDirDefault()) is abstract, so
> FilterFileSystem.mkdirs(Path f, FsPermission permission) gets called.
> * FilterFileSystem.mkdirs(Path f, FsPermission permission) calls fs.mkdirs(f,
> permission).
> * "fs" at this point can be either RawLocalFileSystem or
> DistributedFileSystem depending on whether we are in standalone mode or
> distributed mode.
> RawLocalFileSystem.mkdirs creates directories with permission 777. On the
> other hand, DistributedFileSystem.mkdirs applies the umask to the 777
> permission
> (https://hadoop.apache.org/docs/r2.7.3/hadoop-project-dist/hadoop-hdfs/HdfsPermissionsGuide.html#Changes_to_the_File_System_API).
> By default, the umask (fs.permissions.umask-mode) is 0022, so the permission
> of the directory becomes 755 after application of the 0022 umask.
> In distributed mode directories with 777 are not created in HDFS, but in
> standalone mode directories with 777 are created.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
