[jira] [Commented] (HADOOP-14820) Wasb mkdirs security checks inconsistent with HDFS
[
https://issues.apache.org/jira/browse/HADOOP-14820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16450975#comment-16450975
]
Hudson commented on HADOOP-14820:
-
SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #14057 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/14057/])
HADOOP-14820 Wasb mkdirs security checks inconsistent with HDFS. (xyao: rev
a3e1a2dce2b03230ff412128897550e6373ace5d)
* (edit)
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/NativeAzureFileSystem.java
* (edit)
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azure/metrics/TestAzureFileSystemInstrumentation.java
* (edit)
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azure/TestNativeAzureFileSystemAuthorization.java
> Wasb mkdirs security checks inconsistent with HDFS
> --
>
> Key: HADOOP-14820
> URL: https://issues.apache.org/jira/browse/HADOOP-14820
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs/azure
>Affects Versions: 2.8.1
>Reporter: Sivaguru Sankaridurg
>Assignee: Sivaguru Sankaridurg
>Priority: Major
> Labels: azure, fs, secure, wasb
> Fix For: 2.9.0, 3.0.0-beta1
>
> Attachments: HADOOP-14820-006.patch, HADOOP-14820-007.patch,
> HADOOP-14820-branch-2-001.patch.txt, HADOOP-14820.001.patch,
> HADOOP-14820.002.patch, HADOOP-14820.003.patch, HADOOP-14820.004.patch,
> HADOOP-14820.005.patch
>
>
> No authorization checks should be made when a user tries to create (mkdirs
> -p) an existing folder hierarchy.
> For example, if we start with _/home/hdiuser/prefix_ pre-created, and do the
> following operations, the results should be as shown below.
> {noformat}
> hdiuser@hn0-0d2f67:~$ sudo chown root:root prefix
> hdiuser@hn0-0d2f67:~$ sudo chmod 555 prefix
> hdiuser@hn0-0d2f67:~$ ls -l
> dr-xr-xr-x 3 rootroot 4096 Aug 29 08:25 prefix
> hdiuser@hn0-0d2f67:~$ mkdir -p /home
> hdiuser@hn0-0d2f67:~$ mkdir -p /home/hdiuser
> hdiuser@hn0-0d2f67:~$ mkdir -p /home/hdiuser/prefix
> hdiuser@hn0-0d2f67:~$ mkdir -p /home/hdiuser/prefix/1
> mkdir: cannot create directory â/home/hdiuser/prefix/1â: Permission denied
> The first three mkdirs succeed, because the ancestor is already present. The
> fourth one fails because of a permission check against the (shorter) ancestor
> (as compared to the path being created).
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14820) Wasb mkdirs security checks inconsistent with HDFS
[
https://issues.apache.org/jira/browse/HADOOP-14820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16273707#comment-16273707
]
Thomas Marquardt commented on HADOOP-14820:
---
It was ported to branch-2. Who is using 2.8.x? I don't see any reason not to
backport.
> Wasb mkdirs security checks inconsistent with HDFS
> --
>
> Key: HADOOP-14820
> URL: https://issues.apache.org/jira/browse/HADOOP-14820
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs/azure
>Affects Versions: 2.8.1
>Reporter: Sivaguru Sankaridurg
>Assignee: Sivaguru Sankaridurg
> Labels: azure, fs, secure, wasb
> Fix For: 2.9.0, 3.0.0-beta1
>
> Attachments: HADOOP-14820-006.patch, HADOOP-14820-007.patch,
> HADOOP-14820-branch-2-001.patch.txt, HADOOP-14820.001.patch,
> HADOOP-14820.002.patch, HADOOP-14820.003.patch, HADOOP-14820.004.patch,
> HADOOP-14820.005.patch
>
>
> No authorization checks should be made when a user tries to create (mkdirs
> -p) an existing folder hierarchy.
> For example, if we start with _/home/hdiuser/prefix_ pre-created, and do the
> following operations, the results should be as shown below.
> {noformat}
> hdiuser@hn0-0d2f67:~$ sudo chown root:root prefix
> hdiuser@hn0-0d2f67:~$ sudo chmod 555 prefix
> hdiuser@hn0-0d2f67:~$ ls -l
> dr-xr-xr-x 3 rootroot 4096 Aug 29 08:25 prefix
> hdiuser@hn0-0d2f67:~$ mkdir -p /home
> hdiuser@hn0-0d2f67:~$ mkdir -p /home/hdiuser
> hdiuser@hn0-0d2f67:~$ mkdir -p /home/hdiuser/prefix
> hdiuser@hn0-0d2f67:~$ mkdir -p /home/hdiuser/prefix/1
> mkdir: cannot create directory â/home/hdiuser/prefix/1â: Permission denied
> The first three mkdirs succeed, because the ancestor is already present. The
> fourth one fails because of a permission check against the (shorter) ancestor
> (as compared to the path being created).
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14820) Wasb mkdirs security checks inconsistent with HDFS
[
https://issues.apache.org/jira/browse/HADOOP-14820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16273001#comment-16273001
]
Steve Loughran commented on HADOOP-14820:
-
I should add that this appears to fix the problem whereas mkdirs() can fail
near the root of a tree, because getParent() is being invoked before check for
null
Here's a stack trace of wasb client which *doesn't* have this patch in
{code}
java.lang.NullPointerException
at
org.apache.hadoop.fs.azure.NativeAzureFileSystem.getAncestor(NativeAzureFileSystem.java:2404)
at
org.apache.hadoop.fs.azure.NativeAzureFileSystem.mkdirs(NativeAzureFileSystem.java:2436)
at
org.apache.hadoop.fs.azure.NativeAzureFileSystem.mkdirs(NativeAzureFileSystem.java:2422)
at org.apache.hadoop.fs.FileSystem.mkdirs(FileSystem.java:1924)
...
{code}
directory being created is something like
{{wasb://contr...@stevel.blob.core.windows.net/out/}} in some spark code.
This makes me wonder whether this should be backported to 2.8.x. Thoughts?
> Wasb mkdirs security checks inconsistent with HDFS
> --
>
> Key: HADOOP-14820
> URL: https://issues.apache.org/jira/browse/HADOOP-14820
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs/azure
>Affects Versions: 2.8.1
>Reporter: Sivaguru Sankaridurg
>Assignee: Sivaguru Sankaridurg
> Labels: azure, fs, secure, wasb
> Fix For: 2.9.0, 3.0.0-beta1
>
> Attachments: HADOOP-14820-006.patch, HADOOP-14820-007.patch,
> HADOOP-14820-branch-2-001.patch.txt, HADOOP-14820.001.patch,
> HADOOP-14820.002.patch, HADOOP-14820.003.patch, HADOOP-14820.004.patch,
> HADOOP-14820.005.patch
>
>
> No authorization checks should be made when a user tries to create (mkdirs
> -p) an existing folder hierarchy.
> For example, if we start with _/home/hdiuser/prefix_ pre-created, and do the
> following operations, the results should be as shown below.
> {noformat}
> hdiuser@hn0-0d2f67:~$ sudo chown root:root prefix
> hdiuser@hn0-0d2f67:~$ sudo chmod 555 prefix
> hdiuser@hn0-0d2f67:~$ ls -l
> dr-xr-xr-x 3 rootroot 4096 Aug 29 08:25 prefix
> hdiuser@hn0-0d2f67:~$ mkdir -p /home
> hdiuser@hn0-0d2f67:~$ mkdir -p /home/hdiuser
> hdiuser@hn0-0d2f67:~$ mkdir -p /home/hdiuser/prefix
> hdiuser@hn0-0d2f67:~$ mkdir -p /home/hdiuser/prefix/1
> mkdir: cannot create directory â/home/hdiuser/prefix/1â: Permission denied
> The first three mkdirs succeed, because the ancestor is already present. The
> fourth one fails because of a permission check against the (shorter) ancestor
> (as compared to the path being created).
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14820) Wasb mkdirs security checks inconsistent with HDFS
[
https://issues.apache.org/jira/browse/HADOOP-14820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16157689#comment-16157689
]
Hudson commented on HADOOP-14820:
-
SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #12811 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/12811/])
HADOOP-14820 Wasb mkdirs security checks inconsistent with HDFS. (stevel: rev
792eff9ea70da2c6e0ff5a1b177a51e7b2fb96eb)
* (edit)
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azure/TestNativeAzureFileSystemAuthorization.java
* (edit)
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/NativeAzureFileSystem.java
* (edit)
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azure/metrics/TestAzureFileSystemInstrumentation.java
> Wasb mkdirs security checks inconsistent with HDFS
> --
>
> Key: HADOOP-14820
> URL: https://issues.apache.org/jira/browse/HADOOP-14820
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs/azure
>Affects Versions: 2.8.1
>Reporter: Sivaguru Sankaridurg
>Assignee: Sivaguru Sankaridurg
> Labels: azure, fs, secure, wasb
> Fix For: 2.9.0, 3.0.0-beta1
>
> Attachments: HADOOP-14820.001.patch, HADOOP-14820.002.patch,
> HADOOP-14820.003.patch, HADOOP-14820.004.patch, HADOOP-14820.005.patch,
> HADOOP-14820-006.patch, HADOOP-14820-007.patch,
> HADOOP-14820-branch-2-001.patch.txt
>
>
> No authorization checks should be made when a user tries to create (mkdirs
> -p) an existing folder hierarchy.
> For example, if we start with _/home/hdiuser/prefix_ pre-created, and do the
> following operations, the results should be as shown below.
> {noformat}
> hdiuser@hn0-0d2f67:~$ sudo chown root:root prefix
> hdiuser@hn0-0d2f67:~$ sudo chmod 555 prefix
> hdiuser@hn0-0d2f67:~$ ls -l
> dr-xr-xr-x 3 rootroot 4096 Aug 29 08:25 prefix
> hdiuser@hn0-0d2f67:~$ mkdir -p /home
> hdiuser@hn0-0d2f67:~$ mkdir -p /home/hdiuser
> hdiuser@hn0-0d2f67:~$ mkdir -p /home/hdiuser/prefix
> hdiuser@hn0-0d2f67:~$ mkdir -p /home/hdiuser/prefix/1
> mkdir: cannot create directory â/home/hdiuser/prefix/1â: Permission denied
> The first three mkdirs succeed, because the ancestor is already present. The
> fourth one fails because of a permission check against the (shorter) ancestor
> (as compared to the path being created).
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14820) Wasb mkdirs security checks inconsistent with HDFS
[
https://issues.apache.org/jira/browse/HADOOP-14820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16154332#comment-16154332
]
Andrew Wang commented on HADOOP-14820:
--
Cherry-picked this back to branch-3.0 for beta1 as well, thanks folks.
> Wasb mkdirs security checks inconsistent with HDFS
> --
>
> Key: HADOOP-14820
> URL: https://issues.apache.org/jira/browse/HADOOP-14820
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs/azure
>Affects Versions: 2.8.1
>Reporter: Sivaguru Sankaridurg
>Assignee: Sivaguru Sankaridurg
> Labels: azure, fs, secure, wasb
> Fix For: 2.9.0, 3.0.0-beta1
>
> Attachments: HADOOP-14820.001.patch, HADOOP-14820.002.patch,
> HADOOP-14820.003.patch, HADOOP-14820.004.patch, HADOOP-14820.005.patch,
> HADOOP-14820-006.patch, HADOOP-14820-007.patch,
> HADOOP-14820-branch-2-001.patch.txt
>
>
> No authorization checks should be made when a user tries to create (mkdirs
> -p) an existing folder hierarchy.
> For example, if we start with _/home/hdiuser/prefix_ pre-created, and do the
> following operations, the results should be as shown below.
> {noformat}
> hdiuser@hn0-0d2f67:~$ sudo chown root:root prefix
> hdiuser@hn0-0d2f67:~$ sudo chmod 555 prefix
> hdiuser@hn0-0d2f67:~$ ls -l
> dr-xr-xr-x 3 rootroot 4096 Aug 29 08:25 prefix
> hdiuser@hn0-0d2f67:~$ mkdir -p /home
> hdiuser@hn0-0d2f67:~$ mkdir -p /home/hdiuser
> hdiuser@hn0-0d2f67:~$ mkdir -p /home/hdiuser/prefix
> hdiuser@hn0-0d2f67:~$ mkdir -p /home/hdiuser/prefix/1
> mkdir: cannot create directory â/home/hdiuser/prefix/1â: Permission denied
> The first three mkdirs succeed, because the ancestor is already present. The
> fourth one fails because of a permission check against the (shorter) ancestor
> (as compared to the path being created).
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14820) Wasb mkdirs security checks inconsistent with HDFS
[
https://issues.apache.org/jira/browse/HADOOP-14820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16152568#comment-16152568
]
Hadoop QA commented on HADOOP-14820:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
24s{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} patch {color} | {color:blue} 0m
3s{color} | {color:blue} The patch file was not named according to hadoop's
naming conventions. Please see https://wiki.apache.org/hadoop/HowToContribute
for instructions. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} branch-2 Compile Tests {color} ||
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 4m
25s{color} | {color:red} root in branch-2 failed. {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
27s{color} | {color:green} branch-2 passed with JDK v1.8.0_144 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
19s{color} | {color:green} branch-2 passed with JDK v1.7.0_131 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
16s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
25s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
37s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
13s{color} | {color:green} branch-2 passed with JDK v1.8.0_144 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
15s{color} | {color:green} branch-2 passed with JDK v1.7.0_131 {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
15s{color} | {color:green} the patch passed with JDK v1.8.0_144 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
17s{color} | {color:green} the patch passed with JDK v1.7.0_131 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
17s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 13s{color} | {color:orange} hadoop-tools/hadoop-azure: The patch generated 1
new + 65 unchanged - 1 fixed = 66 total (was 66) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
12s{color} | {color:green} the patch passed with JDK v1.8.0_144 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
14s{color} | {color:green} the patch passed with JDK v1.7.0_131 {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
17s{color} | {color:green} hadoop-azure in the patch passed with JDK
v1.7.0_131. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
17s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 16m 27s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:5e40efe |
| JIRA Issue | HADOOP-14820 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12885254/HADOOP-14820-branch-2-001.patch.txt
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 736408dee01f 3.13.0-117-generic #164-Ubuntu SMP Fri Apr 7
11:05:26 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personalit
[jira] [Commented] (HADOOP-14820) Wasb mkdirs security checks inconsistent with HDFS
[
https://issues.apache.org/jira/browse/HADOOP-14820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16150959#comment-16150959
]
Hadoop QA commented on HADOOP-14820:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
25s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 15m
2s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
21s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
15s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
23s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
30s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
15s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
17s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 13s{color} | {color:orange} hadoop-tools/hadoop-azure: The patch generated 1
new + 80 unchanged - 1 fixed = 81 total (was 81) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
15s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
16s{color} | {color:green} hadoop-azure in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 23m 4s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:71bbb86 |
| JIRA Issue | HADOOP-14820 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12884944/HADOOP-14820-007.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 1e2c6997393d 3.13.0-119-generic #166-Ubuntu SMP Wed May 3
12:18:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 063b6d0 |
| Default Java | 1.8.0_144 |
| findbugs | v3.1.0-RC1 |
| checkstyle |
https://builds.apache.org/job/PreCommit-HADOOP-Build/13154/artifact/patchprocess/diff-checkstyle-hadoop-tools_hadoop-azure.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HADOOP-Build/13154/testReport/ |
| modules | C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure |
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/13154/console |
| Powered by | Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Wasb mkdirs security checks inconsistent with HDFS
> --
>
> Key: HADOOP-14820
> URL: https://issues.apache.org/jira/browse/HADOOP-14820
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs/azure
>Affects Versions: 2.8.1
>Reporter: Sivaguru Sankaridurg
>Assignee: Sivaguru Sankaridurg
> Lab
[jira] [Commented] (HADOOP-14820) Wasb mkdirs security checks inconsistent with HDFS
[
https://issues.apache.org/jira/browse/HADOOP-14820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16150349#comment-16150349
]
Sivaguru Sankaridurg commented on HADOOP-14820:
---
patch 006 :- {{testMkdirsWithExistingHierarchyCheckPositive2}} does not seem
right. childPath3 has already been created.
Later when childPath3 is created again, it won't go up two levels .. it will be
a no-op.
The modifications to the test, to do what you intended is below:
{code:java}
@Test
public void testMkdirsWithExistingHierarchyCheckPositive2() throws Throwable {
Path testPath = new Path("/testMkdirsWithExistingHierarchyCheckPositive2");
Path childPath1 = new Path(testPath, "1");
Path childPath2 = new Path(childPath1, "2");
Path childPath3 = new Path(childPath2, "3");
authorizer.addAuthRule("/",
WasbAuthorizationOperations.WRITE.toString(), true);
authorizer.addAuthRule(childPath1.toString(),
WasbAuthorizationOperations.WRITE.toString(), true);
fs.updateWasbAuthorizer(authorizer);
try {
fs.mkdirs(childPath1);
ContractTestUtils.assertIsDirectory(fs, childPath1);
// Path already exists => no-op.
fs.mkdirs(testPath);
ContractTestUtils.assertIsDirectory(fs, testPath);
// Path already exists => no-op.
fs.mkdirs(childPath1);
ContractTestUtils.assertIsDirectory(fs, childPath1);
// Check permissions against existing ancestor childPath1
fs.mkdirs(childPath3);
ContractTestUtils.assertIsDirectory(fs, childPath3);
}
finally {
allowRecursiveDelete(fs, testPath.toString());
fs.delete(testPath, true);
}
}
{code}
> Wasb mkdirs security checks inconsistent with HDFS
> --
>
> Key: HADOOP-14820
> URL: https://issues.apache.org/jira/browse/HADOOP-14820
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs/azure
>Affects Versions: 2.8.1
>Reporter: Sivaguru Sankaridurg
>Assignee: Sivaguru Sankaridurg
> Labels: azure, fs, secure, wasb
> Attachments: HADOOP-14820.001.patch, HADOOP-14820.002.patch,
> HADOOP-14820.003.patch, HADOOP-14820.004.patch, HADOOP-14820.005.patch,
> HADOOP-14820-006.patch
>
>
> No authorization checks should be made when a user tries to create (mkdirs
> -p) an existing folder hierarchy.
> For example, if we start with _/home/hdiuser/prefix_ pre-created, and do the
> following operations, the results should be as shown below.
> {noformat}
> hdiuser@hn0-0d2f67:~$ sudo chown root:root prefix
> hdiuser@hn0-0d2f67:~$ sudo chmod 555 prefix
> hdiuser@hn0-0d2f67:~$ ls -l
> dr-xr-xr-x 3 rootroot 4096 Aug 29 08:25 prefix
> hdiuser@hn0-0d2f67:~$ mkdir -p /home
> hdiuser@hn0-0d2f67:~$ mkdir -p /home/hdiuser
> hdiuser@hn0-0d2f67:~$ mkdir -p /home/hdiuser/prefix
> hdiuser@hn0-0d2f67:~$ mkdir -p /home/hdiuser/prefix/1
> mkdir: cannot create directory â/home/hdiuser/prefix/1â: Permission denied
> The first three mkdirs succeed, because the ancestor is already present. The
> fourth one fails because of a permission check against the (shorter) ancestor
> (as compared to the path being created).
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14820) Wasb mkdirs security checks inconsistent with HDFS
[
https://issues.apache.org/jira/browse/HADOOP-14820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16149071#comment-16149071
]
Hadoop QA commented on HADOOP-14820:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 24m
26s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 20m
57s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
30s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
25s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
36s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
43s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
24s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
28s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 16s{color} | {color:orange} hadoop-tools/hadoop-azure: The patch generated 1
new + 80 unchanged - 1 fixed = 81 total (was 81) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
16s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
41s{color} | {color:green} hadoop-azure in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
21s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 55m 32s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:71bbb86 |
| JIRA Issue | HADOOP-14820 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12884676/HADOOP-14820.005.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux f8a07f985d04 3.13.0-117-generic #164-Ubuntu SMP Fri Apr 7
11:05:26 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / f9e0cc8 |
| Default Java | 1.8.0_144 |
| findbugs | v3.1.0-RC1 |
| checkstyle |
https://builds.apache.org/job/PreCommit-HADOOP-Build/13145/artifact/patchprocess/diff-checkstyle-hadoop-tools_hadoop-azure.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HADOOP-Build/13145/testReport/ |
| modules | C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure |
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/13145/console |
| Powered by | Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Wasb mkdirs security checks inconsistent with HDFS
> --
>
> Key: HADOOP-14820
> URL: https://issues.apache.org/jira/browse/HADOOP-14820
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs/azure
>Affects Versions: 2.8.1
>Reporter: Sivaguru Sankaridurg
>Assignee: Sivaguru Sankaridurg
> Lab
[jira] [Commented] (HADOOP-14820) Wasb mkdirs security checks inconsistent with HDFS
[
https://issues.apache.org/jira/browse/HADOOP-14820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16148988#comment-16148988
]
Sivaguru Sankaridurg commented on HADOOP-14820:
---
I added another test to cover directories partway up the tree with 005.patch.
> Wasb mkdirs security checks inconsistent with HDFS
> --
>
> Key: HADOOP-14820
> URL: https://issues.apache.org/jira/browse/HADOOP-14820
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs/azure
>Affects Versions: 2.8.1
>Reporter: Sivaguru Sankaridurg
>Assignee: Sivaguru Sankaridurg
> Labels: azure, fs, secure, wasb
> Attachments: HADOOP-14820.001.patch, HADOOP-14820.002.patch,
> HADOOP-14820.003.patch, HADOOP-14820.004.patch, HADOOP-14820.005.patch
>
>
> No authorization checks should be made when a user tries to create (mkdirs
> -p) an existing folder hierarchy.
> For example, if we start with _/home/hdiuser/prefix_ pre-created, and do the
> following operations, the results should be as shown below.
> {noformat}
> hdiuser@hn0-0d2f67:~$ sudo chown root:root prefix
> hdiuser@hn0-0d2f67:~$ sudo chmod 555 prefix
> hdiuser@hn0-0d2f67:~$ ls -l
> dr-xr-xr-x 3 rootroot 4096 Aug 29 08:25 prefix
> hdiuser@hn0-0d2f67:~$ mkdir -p /home
> hdiuser@hn0-0d2f67:~$ mkdir -p /home/hdiuser
> hdiuser@hn0-0d2f67:~$ mkdir -p /home/hdiuser/prefix
> hdiuser@hn0-0d2f67:~$ mkdir -p /home/hdiuser/prefix/1
> mkdir: cannot create directory â/home/hdiuser/prefix/1â: Permission denied
> The first three mkdirs succeed, because the ancestor is already present. The
> fourth one fails because of a permission check against the (shorter) ancestor
> (as compared to the path being created).
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14820) Wasb mkdirs security checks inconsistent with HDFS
[
https://issues.apache.org/jira/browse/HADOOP-14820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16148921#comment-16148921
]
Sivaguru Sankaridurg commented on HADOOP-14820:
---
Fixed the checkstyle issue with the latest patch.
The new test creates {{/testMkdirsWithExistingHierarchyCheckPositive}} and then
removes permission on the root folder; and tries to create
{{/testMkdirsWithExistingHierarchyCheckPositive}} again. The test fails without
the patch. This test is expected to work, because the second create is a no-op
(w.r.t authorization.)
> Wasb mkdirs security checks inconsistent with HDFS
> --
>
> Key: HADOOP-14820
> URL: https://issues.apache.org/jira/browse/HADOOP-14820
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs/azure
>Affects Versions: 2.8.1
>Reporter: Sivaguru Sankaridurg
>Assignee: Sivaguru Sankaridurg
> Labels: azure, fs, secure, wasb
> Attachments: HADOOP-14820.001.patch, HADOOP-14820.002.patch,
> HADOOP-14820.003.patch, HADOOP-14820.004.patch
>
>
> No authorization checks should be made when a user tries to create (mkdirs
> -p) an existing folder hierarchy.
> For example, if we start with _/home/hdiuser/prefix_ pre-created, and do the
> following operations, the results should be as shown below.
> {noformat}
> hdiuser@hn0-0d2f67:~$ sudo chown root:root prefix
> hdiuser@hn0-0d2f67:~$ sudo chmod 555 prefix
> hdiuser@hn0-0d2f67:~$ ls -l
> dr-xr-xr-x 3 rootroot 4096 Aug 29 08:25 prefix
> hdiuser@hn0-0d2f67:~$ mkdir -p /home
> hdiuser@hn0-0d2f67:~$ mkdir -p /home/hdiuser
> hdiuser@hn0-0d2f67:~$ mkdir -p /home/hdiuser/prefix
> hdiuser@hn0-0d2f67:~$ mkdir -p /home/hdiuser/prefix/1
> mkdir: cannot create directory â/home/hdiuser/prefix/1â: Permission denied
> The first three mkdirs succeed, because the ancestor is already present. The
> fourth one fails because of a permission check against the (shorter) ancestor
> (as compared to the path being created).
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14820) Wasb mkdirs security checks inconsistent with HDFS
[
https://issues.apache.org/jira/browse/HADOOP-14820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16148890#comment-16148890
]
Hadoop QA commented on HADOOP-14820:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
16s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 14m
25s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
21s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
15s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
22s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
32s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
14s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
17s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 13s{color} | {color:orange} hadoop-tools/hadoop-azure: The patch generated 1
new + 80 unchanged - 1 fixed = 81 total (was 81) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
12s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
9s{color} | {color:green} hadoop-azure in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 22m 3s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:71bbb86 |
| JIRA Issue | HADOOP-14820 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12884659/HADOOP-14820.004.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 8733c9835e68 3.13.0-117-generic #164-Ubuntu SMP Fri Apr 7
11:05:26 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / ac12e15 |
| Default Java | 1.8.0_144 |
| findbugs | v3.1.0-RC1 |
| checkstyle |
https://builds.apache.org/job/PreCommit-HADOOP-Build/13144/artifact/patchprocess/diff-checkstyle-hadoop-tools_hadoop-azure.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HADOOP-Build/13144/testReport/ |
| modules | C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure |
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/13144/console |
| Powered by | Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Wasb mkdirs security checks inconsistent with HDFS
> --
>
> Key: HADOOP-14820
> URL: https://issues.apache.org/jira/browse/HADOOP-14820
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs/azure
>Affects Versions: 2.8.1
>Reporter: Sivaguru Sankaridurg
>Assignee: Sivaguru Sankaridurg
> Lab
