[jira] [Commented] (HADOOP-14845) Azure wasb: getFileStatus not making any auth checks
[
https://issues.apache.org/jira/browse/HADOOP-14845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16192976#comment-16192976
]
Hadoop QA commented on HADOOP-14845:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
17s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} branch-2 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
10s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
19s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
17s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
24s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
38s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
17s{color} | {color:green} branch-2 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
13s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
21s{color} | {color:green} hadoop-azure in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
18s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 16m 38s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:eaf5c66 |
| JIRA Issue | HADOOP-14845 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12890526/HADOOP-14845-branch-2-005.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit shadedclient findbugs checkstyle |
| uname | Linux 10523454f196 3.13.0-129-generic #178-Ubuntu SMP Fri Aug 11
12:48:20 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | branch-2 / 7fd4a99 |
| Default Java | 1.7.0_151 |
| findbugs | v3.0.0 |
| Test Results |
https://builds.apache.org/job/PreCommit-HADOOP-Build/13458/testReport/ |
| modules | C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure |
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/13458/console |
| Powered by | Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Azure wasb: getFileStatus not making any auth checks
>
>
> Key: HADOOP-14845
> URL: https://issues.apache.org/jira/browse/HADOOP-14845
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/azure, security
>Affects Versions: 2.8.0, 2.7.4
>Reporter: Sivaguru Sankaridurg
>Assignee: Sivaguru Sankaridurg
> Labels: azure, fs, secure, wasb
> Fix For: 2.9.0
>
> Attachments: HADOOP-14845.001.patch, HADOOP-14845.002.patch,
> HADOOP-14845.003.patch,
[jira] [Commented] (HADOOP-14845) Azure wasb: getFileStatus not making any auth checks
[
https://issues.apache.org/jira/browse/HADOOP-14845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16192941#comment-16192941
]
Hudson commented on HADOOP-14845:
-
SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #13032 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/13032/])
HADOOP-14845. Azure wasb: getFileStatus not making any auth check. (stevel: rev
9288206cb3c1a39044a8e106436987185ef43ddf)
* (edit)
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azure/ITestWasbRemoteCallHelper.java
* (edit)
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azure/metrics/ITestAzureFileSystemInstrumentation.java
* (edit)
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azure/TestNativeAzureFileSystemAuthorization.java
* (edit)
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/NativeAzureFileSystem.java
> Azure wasb: getFileStatus not making any auth checks
>
>
> Key: HADOOP-14845
> URL: https://issues.apache.org/jira/browse/HADOOP-14845
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/azure, security
>Affects Versions: 2.8.0, 2.7.4
>Reporter: Sivaguru Sankaridurg
>Assignee: Sivaguru Sankaridurg
> Labels: azure, fs, secure, wasb
> Fix For: 2.9.0
>
> Attachments: HADOOP-14845.001.patch, HADOOP-14845.002.patch,
> HADOOP-14845.003.patch, HADOOP-14845.004.patch,
> HADOOP-14845-branch-2-001.patch.txt, HADOOP-14845-branch-2-002.patch,
> HADOOP-14845-branch-2-003.patch
>
>
> The HDFS spec requires only traverse checks for any file accessed via
> getFileStatus ... and since WASB does not support traverse checks, removing
> this call effectively removed all protections for the getFileStatus call. The
> reasoning at that time was that doing a performAuthCheck was the wrong thing
> to do, since it was going against the specand that the correct fix to the
> getFileStatus issue was to implement traverse checks rather than go against
> the spec by calling performAuthCheck. The side-effects of such a change were
> not fully clear at that time, but the thinking was that it was safer to
> remain true to the spec, as far as possible.
> The reasoning remains correct even today. But in view of the security hole
> introduced by this change (that anyone can load up any other user's data in
> hive), and keeping in mind that WASB does not intend to implement traverse
> checks, we propose a compromise.
> We propose (re)introducing a read-access check to getFileStatus(), that would
> check the existing ancestor for read-access whenever invoked. Although not
> perfect (in that it is a departure from the spec), we believe that it is a
> good compromise between having no checks at all; and implementing full-blown
> traverse checks.
> For scenarios that deal with intermediate folders like mkdirs, the call would
> check for read access against an existing ancestor (when invoked from shell)
> for intermediate non-existent folders – {{ mkdirs /foo/bar, where only "/"
> exists, would result in read-checks against "/" for "/","/foo" and "/foo/bar"
> }}. This can be thought of, as being a close-enough substitute for the
> traverse checks that hdfs does.
> For other scenarios that don't deal with non-existent intermediate folders –
> like read, delete etc, the check will happen against the parent. Once again,
> we can think of the read-check against the parent as a substitute for the
> traverse check, which can be customized for various users with ranger
> policies.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14845) Azure wasb: getFileStatus not making any auth checks
[
https://issues.apache.org/jira/browse/HADOOP-14845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16192917#comment-16192917
]
Steve Loughran commented on HADOOP-14845:
-
OK, testing on trunk and all is well; I was caught out by the fact that these
tests are skipped in branch-2 unless you enable auth in your test
azure-auth-keys.xml file. It seems to me that the tests could actually turn on
auth rather than skip; they just need to make sure that a new FS instance is
created just for this test suite.
which means: the branch-2 tests are broken as the merge was incomplete.
Anyway, I've applied patch 004 to runk & rerunning {{ITestNativeAzureFSAuth*}}
as well as {{TestNativeAzureFileSystemAuthorization}}: all is well
+1 for trunk, committing as is, and about to build a patch for branch-2 which
fixes the test runs
> Azure wasb: getFileStatus not making any auth checks
>
>
> Key: HADOOP-14845
> URL: https://issues.apache.org/jira/browse/HADOOP-14845
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/azure, security
>Affects Versions: 2.8.0, 2.7.4
>Reporter: Sivaguru Sankaridurg
>Assignee: Sivaguru Sankaridurg
> Labels: azure, fs, secure, wasb
> Fix For: 2.9.0
>
> Attachments: HADOOP-14845.001.patch, HADOOP-14845.002.patch,
> HADOOP-14845.003.patch, HADOOP-14845.004.patch,
> HADOOP-14845-branch-2-001.patch.txt, HADOOP-14845-branch-2-002.patch,
> HADOOP-14845-branch-2-003.patch
>
>
> The HDFS spec requires only traverse checks for any file accessed via
> getFileStatus ... and since WASB does not support traverse checks, removing
> this call effectively removed all protections for the getFileStatus call. The
> reasoning at that time was that doing a performAuthCheck was the wrong thing
> to do, since it was going against the specand that the correct fix to the
> getFileStatus issue was to implement traverse checks rather than go against
> the spec by calling performAuthCheck. The side-effects of such a change were
> not fully clear at that time, but the thinking was that it was safer to
> remain true to the spec, as far as possible.
> The reasoning remains correct even today. But in view of the security hole
> introduced by this change (that anyone can load up any other user's data in
> hive), and keeping in mind that WASB does not intend to implement traverse
> checks, we propose a compromise.
> We propose (re)introducing a read-access check to getFileStatus(), that would
> check the existing ancestor for read-access whenever invoked. Although not
> perfect (in that it is a departure from the spec), we believe that it is a
> good compromise between having no checks at all; and implementing full-blown
> traverse checks.
> For scenarios that deal with intermediate folders like mkdirs, the call would
> check for read access against an existing ancestor (when invoked from shell)
> for intermediate non-existent folders – {{ mkdirs /foo/bar, where only "/"
> exists, would result in read-checks against "/" for "/","/foo" and "/foo/bar"
> }}. This can be thought of, as being a close-enough substitute for the
> traverse checks that hdfs does.
> For other scenarios that don't deal with non-existent intermediate folders –
> like read, delete etc, the check will happen against the parent. Once again,
> we can think of the read-check against the parent as a substitute for the
> traverse check, which can be customized for various users with ranger
> policies.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14845) Azure wasb: getFileStatus not making any auth checks
[
https://issues.apache.org/jira/browse/HADOOP-14845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16189687#comment-16189687
]
Hadoop QA commented on HADOOP-14845:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
17s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 3 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 13m
50s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
20s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
15s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
22s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
9m 20s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
30s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
14s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
10m 6s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
35s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m
13s{color} | {color:red} hadoop-tools_hadoop-azure generated 1 new + 0
unchanged - 0 fixed = 1 total (was 0) {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
58s{color} | {color:green} hadoop-azure in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
18s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 39m 3s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:71bbb86 |
| JIRA Issue | HADOOP-14845 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12890147/HADOOP-14845.004.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit shadedclient findbugs checkstyle |
| uname | Linux c644c18a3ca7 3.13.0-117-generic #164-Ubuntu SMP Fri Apr 7
11:05:26 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 453d48b |
| Default Java | 1.8.0_144 |
| findbugs | v3.1.0-RC1 |
| javadoc |
https://builds.apache.org/job/PreCommit-HADOOP-Build/13446/artifact/patchprocess/diff-javadoc-javadoc-hadoop-tools_hadoop-azure.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HADOOP-Build/13446/testReport/ |
| modules | C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure |
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/13446/console |
| Powered by | Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Azure wasb: getFileStatus not making any auth checks
>
[jira] [Commented] (HADOOP-14845) Azure wasb: getFileStatus not making any auth checks
[
https://issues.apache.org/jira/browse/HADOOP-14845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16189447#comment-16189447
]
Steve Loughran commented on HADOOP-14845:
-
thanks. I've been doing some merge work internally too...its combination of the
(welcome) cleanup work in the other patch & the move to parallel test runs &
needing to rename all tests against live endpoints ITest*.
> Azure wasb: getFileStatus not making any auth checks
>
>
> Key: HADOOP-14845
> URL: https://issues.apache.org/jira/browse/HADOOP-14845
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/azure, security
>Affects Versions: 2.8.0, 2.7.4
>Reporter: Sivaguru Sankaridurg
>Assignee: Sivaguru Sankaridurg
> Labels: azure, fs, secure, wasb
> Fix For: 2.9.0
>
> Attachments: HADOOP-14845.001.patch, HADOOP-14845.002.patch,
> HADOOP-14845.003.patch, HADOOP-14845-branch-2-001.patch.txt,
> HADOOP-14845-branch-2-002.patch, HADOOP-14845-branch-2-003.patch
>
>
> The HDFS spec requires only traverse checks for any file accessed via
> getFileStatus ... and since WASB does not support traverse checks, removing
> this call effectively removed all protections for the getFileStatus call. The
> reasoning at that time was that doing a performAuthCheck was the wrong thing
> to do, since it was going against the specand that the correct fix to the
> getFileStatus issue was to implement traverse checks rather than go against
> the spec by calling performAuthCheck. The side-effects of such a change were
> not fully clear at that time, but the thinking was that it was safer to
> remain true to the spec, as far as possible.
> The reasoning remains correct even today. But in view of the security hole
> introduced by this change (that anyone can load up any other user's data in
> hive), and keeping in mind that WASB does not intend to implement traverse
> checks, we propose a compromise.
> We propose (re)introducing a read-access check to getFileStatus(), that would
> check the existing ancestor for read-access whenever invoked. Although not
> perfect (in that it is a departure from the spec), we believe that it is a
> good compromise between having no checks at all; and implementing full-blown
> traverse checks.
> For scenarios that deal with intermediate folders like mkdirs, the call would
> check for read access against an existing ancestor (when invoked from shell)
> for intermediate non-existent folders – {{ mkdirs /foo/bar, where only "/"
> exists, would result in read-checks against "/" for "/","/foo" and "/foo/bar"
> }}. This can be thought of, as being a close-enough substitute for the
> traverse checks that hdfs does.
> For other scenarios that don't deal with non-existent intermediate folders –
> like read, delete etc, the check will happen against the parent. Once again,
> we can think of the read-check against the parent as a substitute for the
> traverse check, which can be customized for various users with ranger
> policies.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14845) Azure wasb: getFileStatus not making any auth checks
[
https://issues.apache.org/jira/browse/HADOOP-14845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16189299#comment-16189299
]
Sivaguru Sankaridurg commented on HADOOP-14845:
---
[~asuresh], [~steve_l]. I'll try fwd porting, fix the tests and submit another
patch for trunk.
> Azure wasb: getFileStatus not making any auth checks
>
>
> Key: HADOOP-14845
> URL: https://issues.apache.org/jira/browse/HADOOP-14845
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/azure, security
>Affects Versions: 2.8.0, 2.7.4
>Reporter: Sivaguru Sankaridurg
>Assignee: Sivaguru Sankaridurg
> Labels: azure, fs, secure, wasb
> Fix For: 2.9.0
>
> Attachments: HADOOP-14845.001.patch, HADOOP-14845.002.patch,
> HADOOP-14845.003.patch, HADOOP-14845-branch-2-001.patch.txt,
> HADOOP-14845-branch-2-002.patch, HADOOP-14845-branch-2-003.patch
>
>
> The HDFS spec requires only traverse checks for any file accessed via
> getFileStatus ... and since WASB does not support traverse checks, removing
> this call effectively removed all protections for the getFileStatus call. The
> reasoning at that time was that doing a performAuthCheck was the wrong thing
> to do, since it was going against the specand that the correct fix to the
> getFileStatus issue was to implement traverse checks rather than go against
> the spec by calling performAuthCheck. The side-effects of such a change were
> not fully clear at that time, but the thinking was that it was safer to
> remain true to the spec, as far as possible.
> The reasoning remains correct even today. But in view of the security hole
> introduced by this change (that anyone can load up any other user's data in
> hive), and keeping in mind that WASB does not intend to implement traverse
> checks, we propose a compromise.
> We propose (re)introducing a read-access check to getFileStatus(), that would
> check the existing ancestor for read-access whenever invoked. Although not
> perfect (in that it is a departure from the spec), we believe that it is a
> good compromise between having no checks at all; and implementing full-blown
> traverse checks.
> For scenarios that deal with intermediate folders like mkdirs, the call would
> check for read access against an existing ancestor (when invoked from shell)
> for intermediate non-existent folders – {{ mkdirs /foo/bar, where only "/"
> exists, would result in read-checks against "/" for "/","/foo" and "/foo/bar"
> }}. This can be thought of, as being a close-enough substitute for the
> traverse checks that hdfs does.
> For other scenarios that don't deal with non-existent intermediate folders –
> like read, delete etc, the check will happen against the parent. Once again,
> we can think of the read-check against the parent as a substitute for the
> traverse check, which can be customized for various users with ranger
> policies.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14845) Azure wasb: getFileStatus not making any auth checks
[
https://issues.apache.org/jira/browse/HADOOP-14845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16188056#comment-16188056
]
Steve Loughran commented on HADOOP-14845:
-
its in branch-2, it's the forward port to trunk that's problematic
> Azure wasb: getFileStatus not making any auth checks
>
>
> Key: HADOOP-14845
> URL: https://issues.apache.org/jira/browse/HADOOP-14845
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/azure, security
>Affects Versions: 2.8.0, 2.7.4
>Reporter: Sivaguru Sankaridurg
>Assignee: Sivaguru Sankaridurg
> Labels: azure, fs, secure, wasb
> Attachments: HADOOP-14845.001.patch, HADOOP-14845.002.patch,
> HADOOP-14845.003.patch, HADOOP-14845-branch-2-001.patch.txt,
> HADOOP-14845-branch-2-002.patch, HADOOP-14845-branch-2-003.patch
>
>
> The HDFS spec requires only traverse checks for any file accessed via
> getFileStatus ... and since WASB does not support traverse checks, removing
> this call effectively removed all protections for the getFileStatus call. The
> reasoning at that time was that doing a performAuthCheck was the wrong thing
> to do, since it was going against the specand that the correct fix to the
> getFileStatus issue was to implement traverse checks rather than go against
> the spec by calling performAuthCheck. The side-effects of such a change were
> not fully clear at that time, but the thinking was that it was safer to
> remain true to the spec, as far as possible.
> The reasoning remains correct even today. But in view of the security hole
> introduced by this change (that anyone can load up any other user's data in
> hive), and keeping in mind that WASB does not intend to implement traverse
> checks, we propose a compromise.
> We propose (re)introducing a read-access check to getFileStatus(), that would
> check the existing ancestor for read-access whenever invoked. Although not
> perfect (in that it is a departure from the spec), we believe that it is a
> good compromise between having no checks at all; and implementing full-blown
> traverse checks.
> For scenarios that deal with intermediate folders like mkdirs, the call would
> check for read access against an existing ancestor (when invoked from shell)
> for intermediate non-existent folders – {{ mkdirs /foo/bar, where only "/"
> exists, would result in read-checks against "/" for "/","/foo" and "/foo/bar"
> }}. This can be thought of, as being a close-enough substitute for the
> traverse checks that hdfs does.
> For other scenarios that don't deal with non-existent intermediate folders –
> like read, delete etc, the check will happen against the parent. Once again,
> we can think of the read-check against the parent as a substitute for the
> traverse check, which can be customized for various users with ranger
> policies.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14845) Azure wasb: getFileStatus not making any auth checks
[
https://issues.apache.org/jira/browse/HADOOP-14845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16185658#comment-16185658
]
Steve Loughran commented on HADOOP-14845:
-
+1
committed to branch-2, leaving trunk awaiting a patch, where the other change
to trunk HADOOP-14768 has stamped on the same tests.
if you want to look at it, check out trunk, cherry pick your patch in and go
with
{code}
git cherry-pick -x -n a641bcec013a72555b0f67c
{code}
& merge them. In the other patch we've
> Azure wasb: getFileStatus not making any auth checks
>
>
> Key: HADOOP-14845
> URL: https://issues.apache.org/jira/browse/HADOOP-14845
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/azure, security
>Affects Versions: 2.8.0, 2.7.4
>Reporter: Sivaguru Sankaridurg
>Assignee: Sivaguru Sankaridurg
> Labels: azure, fs, secure, wasb
> Attachments: HADOOP-14845.001.patch, HADOOP-14845.002.patch,
> HADOOP-14845.003.patch, HADOOP-14845-branch-2-001.patch.txt,
> HADOOP-14845-branch-2-002.patch, HADOOP-14845-branch-2-003.patch
>
>
> The HDFS spec requires only traverse checks for any file accessed via
> getFileStatus ... and since WASB does not support traverse checks, removing
> this call effectively removed all protections for the getFileStatus call. The
> reasoning at that time was that doing a performAuthCheck was the wrong thing
> to do, since it was going against the specand that the correct fix to the
> getFileStatus issue was to implement traverse checks rather than go against
> the spec by calling performAuthCheck. The side-effects of such a change were
> not fully clear at that time, but the thinking was that it was safer to
> remain true to the spec, as far as possible.
> The reasoning remains correct even today. But in view of the security hole
> introduced by this change (that anyone can load up any other user's data in
> hive), and keeping in mind that WASB does not intend to implement traverse
> checks, we propose a compromise.
> We propose (re)introducing a read-access check to getFileStatus(), that would
> check the existing ancestor for read-access whenever invoked. Although not
> perfect (in that it is a departure from the spec), we believe that it is a
> good compromise between having no checks at all; and implementing full-blown
> traverse checks.
> For scenarios that deal with intermediate folders like mkdirs, the call would
> check for read access against an existing ancestor (when invoked from shell)
> for intermediate non-existent folders – {{ mkdirs /foo/bar, where only "/"
> exists, would result in read-checks against "/" for "/","/foo" and "/foo/bar"
> }}. This can be thought of, as being a close-enough substitute for the
> traverse checks that hdfs does.
> For other scenarios that don't deal with non-existent intermediate folders –
> like read, delete etc, the check will happen against the parent. Once again,
> we can think of the read-check against the parent as a substitute for the
> traverse check, which can be customized for various users with ranger
> policies.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
