[jira] [Updated] (HADOOP-14845) Azure wasb: getFileStatus not making any auth checks

2017-10-05 Thread Steve Loughran (JIRA)

 [ 
https://issues.apache.org/jira/browse/HADOOP-14845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Steve Loughran updated HADOOP-14845:

   Resolution: Fixed
Fix Version/s: 3.1.0
   Status: Resolved  (was: Patch Available)

Cherry picked in final changes to {{TestNativeAzureFileSystemAuthorization}} 
from trunk to branch-2. The alternative: rollback, build a new patch, reapply 
wouldn't have worked. 

Closing as fixed for 2.9+. Thanks for your contrib!

> Azure wasb: getFileStatus not making any auth checks
> 
>
> Key: HADOOP-14845
> URL: https://issues.apache.org/jira/browse/HADOOP-14845
> Project: Hadoop Common
>  Issue Type: Sub-task
>  Components: fs/azure, security
>Affects Versions: 2.8.0, 2.7.4
>Reporter: Sivaguru Sankaridurg
>Assignee: Sivaguru Sankaridurg
>  Labels: azure, fs, secure, wasb
> Fix For: 2.9.0, 3.1.0
>
> Attachments: HADOOP-14845.001.patch, HADOOP-14845.002.patch, 
> HADOOP-14845.003.patch, HADOOP-14845.004.patch, 
> HADOOP-14845-branch-2-001.patch.txt, HADOOP-14845-branch-2-002.patch, 
> HADOOP-14845-branch-2-003.patch, HADOOP-14845-branch-2-005.patch
>
>
> The HDFS spec requires only traverse checks for any file accessed via 
> getFileStatus ... and since WASB does not support traverse checks, removing 
> this call effectively removed all protections for the getFileStatus call. The 
> reasoning at that time was that doing a performAuthCheck was the wrong thing 
> to do, since it was going against the specand that the correct fix to the 
> getFileStatus issue was to implement traverse checks rather than go against 
> the spec by calling performAuthCheck. The side-effects of such a change were 
> not fully clear at that time, but the thinking was that it was safer to 
> remain true to the spec, as far as possible.
> The reasoning remains correct even today. But in view of the security hole 
> introduced by this change (that anyone can load up any other user's data in 
> hive), and keeping in mind that WASB does not intend to implement traverse 
> checks, we propose a compromise.
> We propose (re)introducing a read-access check to getFileStatus(), that would 
> check the existing ancestor for read-access whenever invoked. Although not 
> perfect (in that it is a departure from the spec), we believe that it is a 
> good compromise between having no checks at all; and implementing full-blown 
> traverse checks.
> For scenarios that deal with intermediate folders like mkdirs, the call would 
> check for read access against an existing ancestor (when invoked from shell) 
> for intermediate non-existent folders – {{ mkdirs /foo/bar, where only "/" 
> exists, would result in read-checks against "/" for "/","/foo" and "/foo/bar" 
> }}. This can be thought of, as being a close-enough substitute for the 
> traverse checks that hdfs does.
> For other scenarios that don't deal with non-existent intermediate folders – 
> like read, delete etc, the check will happen against the parent. Once again, 
> we can think of the read-check against the parent as a substitute for the 
> traverse check, which can be customized for various users with ranger 
> policies.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Updated] (HADOOP-14845) Azure wasb: getFileStatus not making any auth checks

2017-10-05 Thread Steve Loughran (JIRA)

 [ 
https://issues.apache.org/jira/browse/HADOOP-14845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Steve Loughran updated HADOOP-14845:

Status: Patch Available  (was: Open)

> Azure wasb: getFileStatus not making any auth checks
> 
>
> Key: HADOOP-14845
> URL: https://issues.apache.org/jira/browse/HADOOP-14845
> Project: Hadoop Common
>  Issue Type: Sub-task
>  Components: fs/azure, security
>Affects Versions: 2.7.4, 2.8.0
>Reporter: Sivaguru Sankaridurg
>Assignee: Sivaguru Sankaridurg
>  Labels: azure, fs, secure, wasb
> Fix For: 2.9.0
>
> Attachments: HADOOP-14845.001.patch, HADOOP-14845.002.patch, 
> HADOOP-14845.003.patch, HADOOP-14845.004.patch, 
> HADOOP-14845-branch-2-001.patch.txt, HADOOP-14845-branch-2-002.patch, 
> HADOOP-14845-branch-2-003.patch, HADOOP-14845-branch-2-005.patch
>
>
> The HDFS spec requires only traverse checks for any file accessed via 
> getFileStatus ... and since WASB does not support traverse checks, removing 
> this call effectively removed all protections for the getFileStatus call. The 
> reasoning at that time was that doing a performAuthCheck was the wrong thing 
> to do, since it was going against the specand that the correct fix to the 
> getFileStatus issue was to implement traverse checks rather than go against 
> the spec by calling performAuthCheck. The side-effects of such a change were 
> not fully clear at that time, but the thinking was that it was safer to 
> remain true to the spec, as far as possible.
> The reasoning remains correct even today. But in view of the security hole 
> introduced by this change (that anyone can load up any other user's data in 
> hive), and keeping in mind that WASB does not intend to implement traverse 
> checks, we propose a compromise.
> We propose (re)introducing a read-access check to getFileStatus(), that would 
> check the existing ancestor for read-access whenever invoked. Although not 
> perfect (in that it is a departure from the spec), we believe that it is a 
> good compromise between having no checks at all; and implementing full-blown 
> traverse checks.
> For scenarios that deal with intermediate folders like mkdirs, the call would 
> check for read access against an existing ancestor (when invoked from shell) 
> for intermediate non-existent folders – {{ mkdirs /foo/bar, where only "/" 
> exists, would result in read-checks against "/" for "/","/foo" and "/foo/bar" 
> }}. This can be thought of, as being a close-enough substitute for the 
> traverse checks that hdfs does.
> For other scenarios that don't deal with non-existent intermediate folders – 
> like read, delete etc, the check will happen against the parent. Once again, 
> we can think of the read-check against the parent as a substitute for the 
> traverse check, which can be customized for various users with ranger 
> policies.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Updated] (HADOOP-14845) Azure wasb: getFileStatus not making any auth checks

2017-10-05 Thread Steve Loughran (JIRA)

 [ 
https://issues.apache.org/jira/browse/HADOOP-14845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Steve Loughran updated HADOOP-14845:

Attachment: HADOOP-14845-branch-2-005.patch

Patch branch-2-005; the changes to trunk's 
TestNativeAzureFileSystemAuthorization merged into branch 2 &, where needed, 
fixup for java 7.

This is just from a diff of branch-2 & trunk & picking in the relevant changes, 
as such it is the final bit of the branch-2 patch, addressing test conflict 
with HADOOP-14768. 

Tested
{code}
ava HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m; 
support was removed in 8.0
Running org.apache.hadoop.fs.azure.TestNativeAzureFileSystemAuthorization
Tests run: 38, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 80.182 sec - 
in org.apache.hadoop.fs.azure.TestNativeAzureFileSystemAuthorization
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m; 
support was removed in 8.0
Running org.apache.hadoop.fs.azure.TestNativeAzureFSAuthorizationCaching
Tests run: 39, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 78.257 sec - 
in org.apache.hadoop.fs.azure.TestNativeAzureFSAuthorizationCaching
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m; 
support was removed in 8.0
Running org.apache.hadoop.fs.azure.TestNativeAzureFSAuthWithBlobSpecificKeys
Tests run: 38, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 77.394 sec - 
in org.apache.hadoop.fs.azure.TestNativeAzureFSAuthWithBlobSpecificKeys
{code}


If yetus is happy, I'm going to pull this is in as the final bit of the patch

> Azure wasb: getFileStatus not making any auth checks
> 
>
> Key: HADOOP-14845
> URL: https://issues.apache.org/jira/browse/HADOOP-14845
> Project: Hadoop Common
>  Issue Type: Sub-task
>  Components: fs/azure, security
>Affects Versions: 2.8.0, 2.7.4
>Reporter: Sivaguru Sankaridurg
>Assignee: Sivaguru Sankaridurg
>  Labels: azure, fs, secure, wasb
> Fix For: 2.9.0
>
> Attachments: HADOOP-14845.001.patch, HADOOP-14845.002.patch, 
> HADOOP-14845.003.patch, HADOOP-14845.004.patch, 
> HADOOP-14845-branch-2-001.patch.txt, HADOOP-14845-branch-2-002.patch, 
> HADOOP-14845-branch-2-003.patch, HADOOP-14845-branch-2-005.patch
>
>
> The HDFS spec requires only traverse checks for any file accessed via 
> getFileStatus ... and since WASB does not support traverse checks, removing 
> this call effectively removed all protections for the getFileStatus call. The 
> reasoning at that time was that doing a performAuthCheck was the wrong thing 
> to do, since it was going against the specand that the correct fix to the 
> getFileStatus issue was to implement traverse checks rather than go against 
> the spec by calling performAuthCheck. The side-effects of such a change were 
> not fully clear at that time, but the thinking was that it was safer to 
> remain true to the spec, as far as possible.
> The reasoning remains correct even today. But in view of the security hole 
> introduced by this change (that anyone can load up any other user's data in 
> hive), and keeping in mind that WASB does not intend to implement traverse 
> checks, we propose a compromise.
> We propose (re)introducing a read-access check to getFileStatus(), that would 
> check the existing ancestor for read-access whenever invoked. Although not 
> perfect (in that it is a departure from the spec), we believe that it is a 
> good compromise between having no checks at all; and implementing full-blown 
> traverse checks.
> For scenarios that deal with intermediate folders like mkdirs, the call would 
> check for read access against an existing ancestor (when invoked from shell) 
> for intermediate non-existent folders – {{ mkdirs /foo/bar, where only "/" 
> exists, would result in read-checks against "/" for "/","/foo" and "/foo/bar" 
> }}. This can be thought of, as being a close-enough substitute for the 
> traverse checks that hdfs does.
> For other scenarios that don't deal with non-existent intermediate folders – 
> like read, delete etc, the check will happen against the parent. Once again, 
> we can think of the read-check against the parent as a substitute for the 
> traverse check, which can be customized for various users with ranger 
> policies.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Updated] (HADOOP-14845) Azure wasb: getFileStatus not making any auth checks

2017-10-05 Thread Steve Loughran (JIRA)

 [ 
https://issues.apache.org/jira/browse/HADOOP-14845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Steve Loughran updated HADOOP-14845:

Status: Open  (was: Patch Available)

> Azure wasb: getFileStatus not making any auth checks
> 
>
> Key: HADOOP-14845
> URL: https://issues.apache.org/jira/browse/HADOOP-14845
> Project: Hadoop Common
>  Issue Type: Sub-task
>  Components: fs/azure, security
>Affects Versions: 2.7.4, 2.8.0
>Reporter: Sivaguru Sankaridurg
>Assignee: Sivaguru Sankaridurg
>  Labels: azure, fs, secure, wasb
> Fix For: 2.9.0
>
> Attachments: HADOOP-14845.001.patch, HADOOP-14845.002.patch, 
> HADOOP-14845.003.patch, HADOOP-14845.004.patch, 
> HADOOP-14845-branch-2-001.patch.txt, HADOOP-14845-branch-2-002.patch, 
> HADOOP-14845-branch-2-003.patch
>
>
> The HDFS spec requires only traverse checks for any file accessed via 
> getFileStatus ... and since WASB does not support traverse checks, removing 
> this call effectively removed all protections for the getFileStatus call. The 
> reasoning at that time was that doing a performAuthCheck was the wrong thing 
> to do, since it was going against the specand that the correct fix to the 
> getFileStatus issue was to implement traverse checks rather than go against 
> the spec by calling performAuthCheck. The side-effects of such a change were 
> not fully clear at that time, but the thinking was that it was safer to 
> remain true to the spec, as far as possible.
> The reasoning remains correct even today. But in view of the security hole 
> introduced by this change (that anyone can load up any other user's data in 
> hive), and keeping in mind that WASB does not intend to implement traverse 
> checks, we propose a compromise.
> We propose (re)introducing a read-access check to getFileStatus(), that would 
> check the existing ancestor for read-access whenever invoked. Although not 
> perfect (in that it is a departure from the spec), we believe that it is a 
> good compromise between having no checks at all; and implementing full-blown 
> traverse checks.
> For scenarios that deal with intermediate folders like mkdirs, the call would 
> check for read access against an existing ancestor (when invoked from shell) 
> for intermediate non-existent folders – {{ mkdirs /foo/bar, where only "/" 
> exists, would result in read-checks against "/" for "/","/foo" and "/foo/bar" 
> }}. This can be thought of, as being a close-enough substitute for the 
> traverse checks that hdfs does.
> For other scenarios that don't deal with non-existent intermediate folders – 
> like read, delete etc, the check will happen against the parent. Once again, 
> we can think of the read-check against the parent as a substitute for the 
> traverse check, which can be customized for various users with ranger 
> policies.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Updated] (HADOOP-14845) Azure wasb: getFileStatus not making any auth checks

2017-10-03 Thread Sivaguru Sankaridurg (JIRA)

 [ 
https://issues.apache.org/jira/browse/HADOOP-14845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sivaguru Sankaridurg updated HADOOP-14845:
--
Attachment: HADOOP-14845.004.patch

> Azure wasb: getFileStatus not making any auth checks
> 
>
> Key: HADOOP-14845
> URL: https://issues.apache.org/jira/browse/HADOOP-14845
> Project: Hadoop Common
>  Issue Type: Sub-task
>  Components: fs/azure, security
>Affects Versions: 2.8.0, 2.7.4
>Reporter: Sivaguru Sankaridurg
>Assignee: Sivaguru Sankaridurg
>  Labels: azure, fs, secure, wasb
> Fix For: 2.9.0
>
> Attachments: HADOOP-14845.001.patch, HADOOP-14845.002.patch, 
> HADOOP-14845.003.patch, HADOOP-14845.004.patch, 
> HADOOP-14845-branch-2-001.patch.txt, HADOOP-14845-branch-2-002.patch, 
> HADOOP-14845-branch-2-003.patch
>
>
> The HDFS spec requires only traverse checks for any file accessed via 
> getFileStatus ... and since WASB does not support traverse checks, removing 
> this call effectively removed all protections for the getFileStatus call. The 
> reasoning at that time was that doing a performAuthCheck was the wrong thing 
> to do, since it was going against the specand that the correct fix to the 
> getFileStatus issue was to implement traverse checks rather than go against 
> the spec by calling performAuthCheck. The side-effects of such a change were 
> not fully clear at that time, but the thinking was that it was safer to 
> remain true to the spec, as far as possible.
> The reasoning remains correct even today. But in view of the security hole 
> introduced by this change (that anyone can load up any other user's data in 
> hive), and keeping in mind that WASB does not intend to implement traverse 
> checks, we propose a compromise.
> We propose (re)introducing a read-access check to getFileStatus(), that would 
> check the existing ancestor for read-access whenever invoked. Although not 
> perfect (in that it is a departure from the spec), we believe that it is a 
> good compromise between having no checks at all; and implementing full-blown 
> traverse checks.
> For scenarios that deal with intermediate folders like mkdirs, the call would 
> check for read access against an existing ancestor (when invoked from shell) 
> for intermediate non-existent folders – {{ mkdirs /foo/bar, where only "/" 
> exists, would result in read-checks against "/" for "/","/foo" and "/foo/bar" 
> }}. This can be thought of, as being a close-enough substitute for the 
> traverse checks that hdfs does.
> For other scenarios that don't deal with non-existent intermediate folders – 
> like read, delete etc, the check will happen against the parent. Once again, 
> we can think of the read-check against the parent as a substitute for the 
> traverse check, which can be customized for various users with ranger 
> policies.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Updated] (HADOOP-14845) Azure wasb: getFileStatus not making any auth checks

2017-10-02 Thread Steve Loughran (JIRA)

 [ 
https://issues.apache.org/jira/browse/HADOOP-14845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Steve Loughran updated HADOOP-14845:

Fix Version/s: 2.9.0

> Azure wasb: getFileStatus not making any auth checks
> 
>
> Key: HADOOP-14845
> URL: https://issues.apache.org/jira/browse/HADOOP-14845
> Project: Hadoop Common
>  Issue Type: Sub-task
>  Components: fs/azure, security
>Affects Versions: 2.8.0, 2.7.4
>Reporter: Sivaguru Sankaridurg
>Assignee: Sivaguru Sankaridurg
>  Labels: azure, fs, secure, wasb
> Fix For: 2.9.0
>
> Attachments: HADOOP-14845.001.patch, HADOOP-14845.002.patch, 
> HADOOP-14845.003.patch, HADOOP-14845-branch-2-001.patch.txt, 
> HADOOP-14845-branch-2-002.patch, HADOOP-14845-branch-2-003.patch
>
>
> The HDFS spec requires only traverse checks for any file accessed via 
> getFileStatus ... and since WASB does not support traverse checks, removing 
> this call effectively removed all protections for the getFileStatus call. The 
> reasoning at that time was that doing a performAuthCheck was the wrong thing 
> to do, since it was going against the specand that the correct fix to the 
> getFileStatus issue was to implement traverse checks rather than go against 
> the spec by calling performAuthCheck. The side-effects of such a change were 
> not fully clear at that time, but the thinking was that it was safer to 
> remain true to the spec, as far as possible.
> The reasoning remains correct even today. But in view of the security hole 
> introduced by this change (that anyone can load up any other user's data in 
> hive), and keeping in mind that WASB does not intend to implement traverse 
> checks, we propose a compromise.
> We propose (re)introducing a read-access check to getFileStatus(), that would 
> check the existing ancestor for read-access whenever invoked. Although not 
> perfect (in that it is a departure from the spec), we believe that it is a 
> good compromise between having no checks at all; and implementing full-blown 
> traverse checks.
> For scenarios that deal with intermediate folders like mkdirs, the call would 
> check for read access against an existing ancestor (when invoked from shell) 
> for intermediate non-existent folders – {{ mkdirs /foo/bar, where only "/" 
> exists, would result in read-checks against "/" for "/","/foo" and "/foo/bar" 
> }}. This can be thought of, as being a close-enough substitute for the 
> traverse checks that hdfs does.
> For other scenarios that don't deal with non-existent intermediate folders – 
> like read, delete etc, the check will happen against the parent. Once again, 
> we can think of the read-check against the parent as a substitute for the 
> traverse check, which can be customized for various users with ranger 
> policies.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Updated] (HADOOP-14845) Azure wasb: getFileStatus not making any auth checks

2017-09-29 Thread Arun Suresh (JIRA)

 [ 
https://issues.apache.org/jira/browse/HADOOP-14845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Arun Suresh updated HADOOP-14845:
-

Is this still on target for 2.9.0 ? If not, can we we push this out to the next 
major release ?

> Azure wasb: getFileStatus not making any auth checks
> 
>
> Key: HADOOP-14845
> URL: https://issues.apache.org/jira/browse/HADOOP-14845
> Project: Hadoop Common
>  Issue Type: Sub-task
>  Components: fs/azure, security
>Affects Versions: 2.8.0, 2.7.4
>Reporter: Sivaguru Sankaridurg
>Assignee: Sivaguru Sankaridurg
>  Labels: azure, fs, secure, wasb
> Attachments: HADOOP-14845.001.patch, HADOOP-14845.002.patch, 
> HADOOP-14845.003.patch, HADOOP-14845-branch-2-001.patch.txt, 
> HADOOP-14845-branch-2-002.patch, HADOOP-14845-branch-2-003.patch
>
>
> The HDFS spec requires only traverse checks for any file accessed via 
> getFileStatus ... and since WASB does not support traverse checks, removing 
> this call effectively removed all protections for the getFileStatus call. The 
> reasoning at that time was that doing a performAuthCheck was the wrong thing 
> to do, since it was going against the specand that the correct fix to the 
> getFileStatus issue was to implement traverse checks rather than go against 
> the spec by calling performAuthCheck. The side-effects of such a change were 
> not fully clear at that time, but the thinking was that it was safer to 
> remain true to the spec, as far as possible.
> The reasoning remains correct even today. But in view of the security hole 
> introduced by this change (that anyone can load up any other user's data in 
> hive), and keeping in mind that WASB does not intend to implement traverse 
> checks, we propose a compromise.
> We propose (re)introducing a read-access check to getFileStatus(), that would 
> check the existing ancestor for read-access whenever invoked. Although not 
> perfect (in that it is a departure from the spec), we believe that it is a 
> good compromise between having no checks at all; and implementing full-blown 
> traverse checks.
> For scenarios that deal with intermediate folders like mkdirs, the call would 
> check for read access against an existing ancestor (when invoked from shell) 
> for intermediate non-existent folders – {{ mkdirs /foo/bar, where only "/" 
> exists, would result in read-checks against "/" for "/","/foo" and "/foo/bar" 
> }}. This can be thought of, as being a close-enough substitute for the 
> traverse checks that hdfs does.
> For other scenarios that don't deal with non-existent intermediate folders – 
> like read, delete etc, the check will happen against the parent. Once again, 
> we can think of the read-check against the parent as a substitute for the 
> traverse check, which can be customized for various users with ranger 
> policies.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Updated] (HADOOP-14845) Azure wasb: getFileStatus not making any auth checks

2017-09-29 Thread Steve Loughran (JIRA)

 [ 
https://issues.apache.org/jira/browse/HADOOP-14845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Steve Loughran updated HADOOP-14845:

Summary: Azure wasb: getFileStatus not making any auth checks  (was: azure 
getFileStatus not making any auth checks)

> Azure wasb: getFileStatus not making any auth checks
> 
>
> Key: HADOOP-14845
> URL: https://issues.apache.org/jira/browse/HADOOP-14845
> Project: Hadoop Common
>  Issue Type: Sub-task
>  Components: fs/azure, security
>Affects Versions: 2.8.0, 2.7.4
>Reporter: Sivaguru Sankaridurg
>Assignee: Sivaguru Sankaridurg
>  Labels: azure, fs, secure, wasb
> Attachments: HADOOP-14845.001.patch, HADOOP-14845.002.patch, 
> HADOOP-14845.003.patch, HADOOP-14845-branch-2-001.patch.txt, 
> HADOOP-14845-branch-2-002.patch, HADOOP-14845-branch-2-003.patch
>
>
> The HDFS spec requires only traverse checks for any file accessed via 
> getFileStatus ... and since WASB does not support traverse checks, removing 
> this call effectively removed all protections for the getFileStatus call. The 
> reasoning at that time was that doing a performAuthCheck was the wrong thing 
> to do, since it was going against the specand that the correct fix to the 
> getFileStatus issue was to implement traverse checks rather than go against 
> the spec by calling performAuthCheck. The side-effects of such a change were 
> not fully clear at that time, but the thinking was that it was safer to 
> remain true to the spec, as far as possible.
> The reasoning remains correct even today. But in view of the security hole 
> introduced by this change (that anyone can load up any other user's data in 
> hive), and keeping in mind that WASB does not intend to implement traverse 
> checks, we propose a compromise.
> We propose (re)introducing a read-access check to getFileStatus(), that would 
> check the existing ancestor for read-access whenever invoked. Although not 
> perfect (in that it is a departure from the spec), we believe that it is a 
> good compromise between having no checks at all; and implementing full-blown 
> traverse checks.
> For scenarios that deal with intermediate folders like mkdirs, the call would 
> check for read access against an existing ancestor (when invoked from shell) 
> for intermediate non-existent folders – {{ mkdirs /foo/bar, where only "/" 
> exists, would result in read-checks against "/" for "/","/foo" and "/foo/bar" 
> }}. This can be thought of, as being a close-enough substitute for the 
> traverse checks that hdfs does.
> For other scenarios that don't deal with non-existent intermediate folders – 
> like read, delete etc, the check will happen against the parent. Once again, 
> we can think of the read-check against the parent as a substitute for the 
> traverse check, which can be customized for various users with ranger 
> policies.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org