[jira] [Commented] (HDFS-10276) Different results for exist call for file.ext/name
[ https://issues.apache.org/jira/browse/HDFS-10276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15281676#comment-15281676 ] Colin Patrick McCabe commented on HDFS-10276: - Thanks for this, [~yuanbo]. Sorry for the sometimes slow pace of reviews. If I understand correctly, the approach taken in this path is to change HDFS to throw an exception stating that the parent path is not a directory, rather than throwing an AccessControlException. So first of all, this sounds like an incompatible change. That's OK-- it just means this should probably go into branch-3 (trunk) rather than branch-2. Secondly, it seems like it would be better to make the modification inside {{FSPermissionChecker}}, rather than adding an external function. After all, this is a general problem, which affects more than just listDir. We also need to make sure that we are not giving away too much information about the filesystem. For example, if the user asks for {{/a/b/c}}, but does not have permission to list {{/a}}, we should not complain about {{/a/b}} not being a directory since that reveals privileged information. > Different results for exist call for file.ext/name > -- > > Key: HDFS-10276 > URL: https://issues.apache.org/jira/browse/HDFS-10276 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Kevin Cox >Assignee: Yuanbo Liu > Attachments: HDFS-10276.001.patch, HDFS-10276.002.patch, > HDFS-10276.003.patch, HDFS-10276.004.patch > > > Given you have a file {{/file}} an existence check for the path > {{/file/whatever}} will give different responses for different > implementations of FileSystem. > LocalFileSystem will return false while DistributedFileSystem will throw > {{org.apache.hadoop.security.AccessControlException: Permission denied: ..., > access=EXECUTE, ...}} -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-10276) Different results for exist call for file.ext/name
[ https://issues.apache.org/jira/browse/HDFS-10276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15275955#comment-15275955 ] Arpit Agarwal commented on HDFS-10276: -- Hi [~yuanbo], sorry not until later this week. > Different results for exist call for file.ext/name > -- > > Key: HDFS-10276 > URL: https://issues.apache.org/jira/browse/HDFS-10276 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Kevin Cox >Assignee: Yuanbo Liu > Attachments: HDFS-10276.001.patch, HDFS-10276.002.patch, > HDFS-10276.003.patch, HDFS-10276.004.patch > > > Given you have a file {{/file}} an existence check for the path > {{/file/whatever}} will give different responses for different > implementations of FileSystem. > LocalFileSystem will return false while DistributedFileSystem will throw > {{org.apache.hadoop.security.AccessControlException: Permission denied: ..., > access=EXECUTE, ...}} -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-10276) Different results for exist call for file.ext/name
[ https://issues.apache.org/jira/browse/HDFS-10276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15275950#comment-15275950 ] Yuanbo Liu commented on HDFS-10276: --- [~arpitagarwal] Could you please help me review this issue? > Different results for exist call for file.ext/name > -- > > Key: HDFS-10276 > URL: https://issues.apache.org/jira/browse/HDFS-10276 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Kevin Cox >Assignee: Yuanbo Liu > Attachments: HDFS-10276.001.patch, HDFS-10276.002.patch, > HDFS-10276.003.patch, HDFS-10276.004.patch > > > Given you have a file {{/file}} an existence check for the path > {{/file/whatever}} will give different responses for different > implementations of FileSystem. > LocalFileSystem will return false while DistributedFileSystem will throw > {{org.apache.hadoop.security.AccessControlException: Permission denied: ..., > access=EXECUTE, ...}} -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-10276) Different results for exist call for file.ext/name
[ https://issues.apache.org/jira/browse/HDFS-10276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15265661#comment-15265661 ] Yuanbo Liu commented on HDFS-10276: --- [~cmccabe] Would you please take a look at the last patch? {{exists}} depends on {{getFileInfo}}, so this patch will also change the behavior of {{exists}} method. I agree with you that this patch is branch-3 only. Please let me know if you have any thought about this patch. Thanks in advance. > Different results for exist call for file.ext/name > -- > > Key: HDFS-10276 > URL: https://issues.apache.org/jira/browse/HDFS-10276 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Kevin Cox >Assignee: Yuanbo Liu > Attachments: HDFS-10276.001.patch, HDFS-10276.002.patch, > HDFS-10276.003.patch, HDFS-10276.004.patch > > > Given you have a file {{/file}} an existence check for the path > {{/file/whatever}} will give different responses for different > implementations of FileSystem. > LocalFileSystem will return false while DistributedFileSystem will throw > {{org.apache.hadoop.security.AccessControlException: Permission denied: ..., > access=EXECUTE, ...}} -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-10276) Different results for exist call for file.ext/name
[ https://issues.apache.org/jira/browse/HDFS-10276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15265633#comment-15265633 ] Hadoop QA commented on HDFS-10276: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 11s {color} | {color:blue} Docker mode activated. {color} | | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s {color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s {color} | {color:green} The patch appears to include 6 new or modified test files. {color} | | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 2m 38s {color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m 35s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 5m 39s {color} | {color:green} trunk passed with JDK v1.8.0_91 {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m 38s {color} | {color:green} trunk passed with JDK v1.7.0_95 {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m 6s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m 23s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 41s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m 7s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m 20s {color} | {color:green} trunk passed with JDK v1.8.0_91 {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m 14s {color} | {color:green} trunk passed with JDK v1.7.0_95 {color} | | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 15s {color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 57s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 5m 36s {color} | {color:green} the patch passed with JDK v1.8.0_91 {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 5m 36s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m 39s {color} | {color:green} the patch passed with JDK v1.7.0_95 {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 6m 39s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m 5s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m 21s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 41s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s {color} | {color:green} Patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m 49s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m 17s {color} | {color:green} the patch passed with JDK v1.8.0_91 {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m 13s {color} | {color:green} the patch passed with JDK v1.7.0_95 {color} | | {color:red}-1{color} | {color:red} unit {color} | {color:red} 6m 48s {color} | {color:red} hadoop-common in the patch failed with JDK v1.8.0_91. {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 51s {color} | {color:green} hadoop-hdfs-client in the patch passed with JDK v1.8.0_91. {color} | | {color:red}-1{color} | {color:red} unit {color} | {color:red} 57m 11s {color} | {color:red} hadoop-hdfs in the patch failed with JDK v1.8.0_91. {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 7m 38s {color} | {color:green} hadoop-common in the patch passed with JDK v1.7.0_95. {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 58s {color} | {color:green} hadoop-hdfs-client in the patch passed with JDK v1.7.0_95. {color} | | {color:red}-1{color} | {color:red} unit {color} | {color:red} 54m 0s {color} | {color:red} hadoop-hdfs in the patch failed with JDK v1.7.0_95. {color} | | {
[jira] [Commented] (HDFS-10276) Different results for exist call for file.ext/name
[ https://issues.apache.org/jira/browse/HDFS-10276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15263898#comment-15263898 ] Hadoop QA commented on HDFS-10276: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 10s {color} | {color:blue} Docker mode activated. {color} | | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s {color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s {color} | {color:green} The patch appears to include 6 new or modified test files. {color} | | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 32s {color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m 46s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m 43s {color} | {color:green} trunk passed with JDK v1.8.0_91 {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m 7s {color} | {color:green} trunk passed with JDK v1.7.0_95 {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m 5s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m 25s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 41s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m 12s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m 24s {color} | {color:green} trunk passed with JDK v1.8.0_91 {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m 19s {color} | {color:green} trunk passed with JDK v1.7.0_95 {color} | | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 14s {color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m 1s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m 35s {color} | {color:green} the patch passed with JDK v1.8.0_91 {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 6m 35s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m 11s {color} | {color:green} the patch passed with JDK v1.7.0_95 {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 7m 11s {color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m 5s {color} | {color:red} root: patch generated 1 new + 114 unchanged - 0 fixed = 115 total (was 114) {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m 24s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 42s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s {color} | {color:green} Patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m 58s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m 23s {color} | {color:green} the patch passed with JDK v1.8.0_91 {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m 18s {color} | {color:green} the patch passed with JDK v1.7.0_95 {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 7m 53s {color} | {color:green} hadoop-common in the patch passed with JDK v1.8.0_91. {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 0s {color} | {color:green} hadoop-hdfs-client in the patch passed with JDK v1.8.0_91. {color} | | {color:red}-1{color} | {color:red} unit {color} | {color:red} 63m 36s {color} | {color:red} hadoop-hdfs in the patch failed with JDK v1.8.0_91. {color} | | {color:red}-1{color} | {color:red} unit {color} | {color:red} 7m 42s {color} | {color:red} hadoop-common in the patch failed with JDK v1.7.0_95. {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 0s {color} | {color:green} hadoop-hdfs-client in the patch passed with JDK v1.7.0_95. {color} | | {color:red}-1{color} | {color:red} unit {color} | {color:red} 57m 6s {color} | {color:red} hadoop-hdfs in t
[jira] [Commented] (HDFS-10276) Different results for exist call for file.ext/name
[ https://issues.apache.org/jira/browse/HDFS-10276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15262094#comment-15262094 ] Hadoop QA commented on HDFS-10276: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 10s {color} | {color:blue} Docker mode activated. {color} | | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s {color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s {color} | {color:green} The patch appears to include 7 new or modified test files. {color} | | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 30s {color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m 8s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m 21s {color} | {color:green} trunk passed with JDK v1.8.0_91 {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m 55s {color} | {color:green} trunk passed with JDK v1.7.0_95 {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m 6s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m 24s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 38s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m 27s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m 26s {color} | {color:green} trunk passed with JDK v1.8.0_91 {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m 20s {color} | {color:green} trunk passed with JDK v1.7.0_95 {color} | | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 14s {color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m 2s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m 13s {color} | {color:green} the patch passed with JDK v1.8.0_91 {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 6m 13s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m 59s {color} | {color:green} the patch passed with JDK v1.7.0_95 {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 6m 59s {color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m 8s {color} | {color:red} root: patch generated 2 new + 114 unchanged - 0 fixed = 116 total (was 114) {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m 29s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 41s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s {color} | {color:green} Patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 6m 15s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m 36s {color} | {color:green} the patch passed with JDK v1.8.0_91 {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m 33s {color} | {color:green} the patch passed with JDK v1.7.0_95 {color} | | {color:red}-1{color} | {color:red} unit {color} | {color:red} 19m 37s {color} | {color:red} hadoop-common in the patch failed with JDK v1.8.0_91. {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 58s {color} | {color:green} hadoop-hdfs-client in the patch passed with JDK v1.8.0_91. {color} | | {color:red}-1{color} | {color:red} unit {color} | {color:red} 64m 27s {color} | {color:red} hadoop-hdfs in the patch failed with JDK v1.8.0_91. {color} | | {color:red}-1{color} | {color:red} unit {color} | {color:red} 7m 47s {color} | {color:red} hadoop-common in the patch failed with JDK v1.7.0_95. {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 3s {color} | {color:green} hadoop-hdfs-client in the patch passed with JDK v1.7.0_95. {color} | | {color:red}-1{color} | {color:red} unit {color} | {color:red} 55m 50s {color} | {color:red} hadoop-hdfs in the pat
[jira] [Commented] (HDFS-10276) Different results for exist call for file.ext/name
[ https://issues.apache.org/jira/browse/HDFS-10276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15256260#comment-15256260 ] Hadoop QA commented on HDFS-10276: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 19s {color} | {color:blue} Docker mode activated. {color} | | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s {color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s {color} | {color:green} The patch appears to include 1 new or modified test files. {color} | | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 29s {color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m 49s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 24s {color} | {color:green} trunk passed with JDK v1.8.0_77 {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 0s {color} | {color:green} trunk passed with JDK v1.7.0_95 {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 34s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m 16s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 37s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m 1s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m 23s {color} | {color:green} trunk passed with JDK v1.8.0_77 {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m 15s {color} | {color:green} trunk passed with JDK v1.7.0_95 {color} | | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 13s {color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m 1s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 34s {color} | {color:green} the patch passed with JDK v1.8.0_77 {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m 34s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 1s {color} | {color:green} the patch passed with JDK v1.7.0_95 {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m 1s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 34s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 53s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 32s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s {color} | {color:green} Patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m 24s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m 26s {color} | {color:green} the patch passed with JDK v1.8.0_77 {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m 59s {color} | {color:green} the patch passed with JDK v1.7.0_95 {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 30s {color} | {color:green} hadoop-hdfs-client in the patch passed with JDK v1.8.0_77. {color} | | {color:red}-1{color} | {color:red} unit {color} | {color:red} 110m 28s {color} | {color:red} hadoop-hdfs in the patch failed with JDK v1.8.0_77. {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 46s {color} | {color:green} hadoop-hdfs-client in the patch passed with JDK v1.7.0_95. {color} | | {color:red}-1{color} | {color:red} unit {color} | {color:red} 94m 32s {color} | {color:red} hadoop-hdfs in the patch failed with JDK v1.7.0_95. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 24s {color} | {color:green} Patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 262m 24s {color} | {color:black} {color} | \\ \\ || Reason || Tests || | JDK v1.8.0_77 Failed juni
[jira] [Commented] (HDFS-10276) Different results for exist call for file.ext/name
[ https://issues.apache.org/jira/browse/HDFS-10276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15250459#comment-15250459 ] Colin Patrick McCabe commented on HDFS-10276: - bq. I think it's more reasonable to throw ParentNotDirectoryException rather than AccessControlException. I agree. This seems like it would be an incompatible change, though, so probably branch-3 only. > Different results for exist call for file.ext/name > -- > > Key: HDFS-10276 > URL: https://issues.apache.org/jira/browse/HDFS-10276 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Kevin Cox >Assignee: Yuanbo Liu > > Given you have a file {{/file}} an existence check for the path > {{/file/whatever}} will give different responses for different > implementations of FileSystem. > LocalFileSystem will return false while DistributedFileSystem will throw > {{org.apache.hadoop.security.AccessControlException: Permission denied: ..., > access=EXECUTE, ...}} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-10276) Different results for exist call for file.ext/name
[ https://issues.apache.org/jira/browse/HDFS-10276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15249555#comment-15249555 ] Yuanbo Liu commented on HDFS-10276: --- [~cmccabe] Thanks for your comment. Yes I agree with you that we should take unprivileged users as consider. But in this case, NameNode should prevent any further operation since it knows the "parent path" is a file, not a real path. I think it's more reasonable to throw {{ParentNotDirectoryException}} rather than {{AccessControlException}}. See the behavior of Linux as below: {code} [yuanbo@oc7702007844 ~]$ ls test/test1 ls: cannot access test/test1: Not a directory {code} One thing should be rectified is that {{getFileInfo}} in {{DFSClient}} causes this problem instead of {{exists}} method. > Different results for exist call for file.ext/name > -- > > Key: HDFS-10276 > URL: https://issues.apache.org/jira/browse/HDFS-10276 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Kevin Cox >Assignee: Yuanbo Liu > > Given you have a file {{/file}} an existence check for the path > {{/file/whatever}} will give different responses for different > implementations of FileSystem. > LocalFileSystem will return false while DistributedFileSystem will throw > {{org.apache.hadoop.security.AccessControlException: Permission denied: ..., > access=EXECUTE, ...}} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-10276) Different results for exist call for file.ext/name
[ https://issues.apache.org/jira/browse/HDFS-10276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15248402#comment-15248402 ] Colin Patrick McCabe commented on HDFS-10276: - bq. Thanks a lot for your comment. I've reproduced this error and it should be marked as a bug. I found that name node would check the access before checking whether the file existed. The NameNode needs to check access before checking whether the file exists. Otherwise, unprivileged users could get information about files and directories they should not have access to. In this specific case, though, we do want exists to return false, since the user apparently does have permissions to find out that the file doesn't exist. > Different results for exist call for file.ext/name > -- > > Key: HDFS-10276 > URL: https://issues.apache.org/jira/browse/HDFS-10276 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Kevin Cox >Assignee: Yuanbo Liu > > Given you have a file {{/file}} an existence check for the path > {{/file/whatever}} will give different responses for different > implementations of FileSystem. > LocalFileSystem will return false while DistributedFileSystem will throw > {{org.apache.hadoop.security.AccessControlException: Permission denied: ..., > access=EXECUTE, ...}} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-10276) Different results for exist call for file.ext/name
[ https://issues.apache.org/jira/browse/HDFS-10276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15247577#comment-15247577 ] Kevin Cox commented on HDFS-10276: -- Excellent. As I said we originally noticed this behaviour because of the difference between behaviour on HDFS and local. I haven't tested with any other implementations. > Different results for exist call for file.ext/name > -- > > Key: HDFS-10276 > URL: https://issues.apache.org/jira/browse/HDFS-10276 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Kevin Cox >Assignee: Yuanbo Liu > > Given you have a file {{/file}} an existence check for the path > {{/file/whatever}} will give different responses for different > implementations of FileSystem. > LocalFileSystem will return false while DistributedFileSystem will throw > {{org.apache.hadoop.security.AccessControlException: Permission denied: ..., > access=EXECUTE, ...}} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-10276) Different results for exist call for file.ext/name
[ https://issues.apache.org/jira/browse/HDFS-10276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15247553#comment-15247553 ] Yuanbo Liu commented on HDFS-10276: --- [~kevincox]Thanks a lot for your comment. I've reproduced this error and it should be marked as a bug. I found that name node would check the access before checking whether the file existed. It takes time to fix this bug since I have to find out whether other file systems work as expected and how to fix it appositely. > Different results for exist call for file.ext/name > -- > > Key: HDFS-10276 > URL: https://issues.apache.org/jira/browse/HDFS-10276 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Kevin Cox >Assignee: Yuanbo Liu > > Given you have a file {{/file}} an existence check for the path > {{/file/whatever}} will give different responses for different > implementations of FileSystem. > LocalFileSystem will return false while DistributedFileSystem will throw > {{org.apache.hadoop.security.AccessControlException: Permission denied: ..., > access=EXECUTE, ...}} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-10276) Different results for exist call for file.ext/name
[ https://issues.apache.org/jira/browse/HDFS-10276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15245590#comment-15245590 ] Kevin Cox commented on HDFS-10276: -- Oops, my mistake. That chmod should be 666 (non executable) which would cause the `cat` to throw the exception. The new log is below. {code} % hdfs --config starscream/hadoop dfs -put <(echo test) /test 2016-04-18 08:15:41,615 WARN [main] util.NativeCodeLoader (NativeCodeLoader.java:(62)) - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable hdfs --config starscream/hadoop dfs -put <(echo test) /test 2.72s user 0.18s system 128% cpu 2.269 total % hdfs --config starscream/hadoop dfs -chmod 666 /test 2016-04-18 08:16:52,903 WARN [main] util.NativeCodeLoader (NativeCodeLoader.java:(62)) - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable hdfs --config starscream/hadoop dfs -chmod 666 /test 2.37s user 0.16s system 182% cpu 1.390 total % hdfs --config starscream/hadoop dfs -cat /test/bar 2016-04-18 08:16:55,743 WARN [main] util.NativeCodeLoader (NativeCodeLoader.java:(62)) - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable cat: Permission denied: user=foo, access=EXECUTE, inode="/test/bar":kevincox:supergroup:-rw-rw-rw- HADOOP_USER_NAME=foo hdfs --config starscream/hadoop dfs -cat /test/bar 2.40s user 0.16s system 185% cpu 1.378 total {code} > Different results for exist call for file.ext/name > -- > > Key: HDFS-10276 > URL: https://issues.apache.org/jira/browse/HDFS-10276 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Kevin Cox >Assignee: Yuanbo Liu > > Given you have a file {{/file}} an existence check for the path > {{/file/whatever}} will give different responses for different > implementations of FileSystem. > LocalFileSystem will return false while DistributedFileSystem will throw > {{org.apache.hadoop.security.AccessControlException: Permission denied: ..., > access=EXECUTE, ...}} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-10276) Different results for exist call for file.ext/name
[ https://issues.apache.org/jira/browse/HDFS-10276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15245214#comment-15245214 ] Yuanbo Liu commented on HDFS-10276: --- Hi, Kevin Sorry for taking so long to reply you, I'm quite busy last week. Seems that you used the username {{kevincox}} to operate files of root directory on the distributed file system. I think the distributed file system works as expected since you may not have the access to operate the files. It's better to compare the two different file systems under the same condition, for instance, the username {{kevincox}} does not have the access to read/write/execute files on the both systems and then operate the files. I've tested different scenarios and {{exists}} method did not throw any error. So could you please provide the steps of throwing {{org.apache.hadoop.security.AccessControlException: Permission denied: ..., access=EXECUTE, ...}} ? > Different results for exist call for file.ext/name > -- > > Key: HDFS-10276 > URL: https://issues.apache.org/jira/browse/HDFS-10276 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Kevin Cox >Assignee: Yuanbo Liu > > Given you have a file {{/file}} an existence check for the path > {{/file/whatever}} will give different responses for different > implementations of FileSystem. > LocalFileSystem will return false while DistributedFileSystem will throw > {{org.apache.hadoop.security.AccessControlException: Permission denied: ..., > access=EXECUTE, ...}} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-10276) Different results for exist call for file.ext/name
[ https://issues.apache.org/jira/browse/HDFS-10276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15245215#comment-15245215 ] Yuanbo Liu commented on HDFS-10276: --- Hi, Kevin Sorry for taking so long to reply you, I'm quite busy last week. Seems that you used the username {{kevincox}} to operate files of root directory on the distributed file system. I think the distributed file system works as expected since you may not have the access to operate the files. It's better to compare the two different file systems under the same condition, for instance, the username {{kevincox}} does not have the access to read/write/execute files on the both systems and then operate the files. I've tested different scenarios and {{exists}} method did not throw any error. So could you please provide the steps of throwing {{org.apache.hadoop.security.AccessControlException: Permission denied: ..., access=EXECUTE, ...}} ? > Different results for exist call for file.ext/name > -- > > Key: HDFS-10276 > URL: https://issues.apache.org/jira/browse/HDFS-10276 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Kevin Cox >Assignee: Yuanbo Liu > > Given you have a file {{/file}} an existence check for the path > {{/file/whatever}} will give different responses for different > implementations of FileSystem. > LocalFileSystem will return false while DistributedFileSystem will throw > {{org.apache.hadoop.security.AccessControlException: Permission denied: ..., > access=EXECUTE, ...}} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-10276) Different results for exist call for file.ext/name
[ https://issues.apache.org/jira/browse/HDFS-10276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15239233#comment-15239233 ] Kevin Cox commented on HDFS-10276: -- I have tested on two moderately recent versions of hadoop and the following commands will reproduce the issue. The chmod is unnecessary in most cases but I have included it just to be sure. The following outputs are on a HDFS. When run on a local path {code} % hdfs version Hadoop 2.7.2 Subversion https://git-wip-us.apache.org/repos/asf/hadoop.git -r b165c4fe8a74265c792ce23f546c64604acf0e41 Compiled by jenkins on 2016-01-26T00:08Z Compiled with protoc 2.5.0 >From source with checksum d0fda26633fa762bff87ec759ebe689c This command was run using /usr/local/Cellar/hadoop/2.7.2/libexec/share/hadoop/common/hadoop-common-2.7.2.jar % hdfs --config hadoop dfs -put <(echo "test") /test 2016-04-13 09:24:58,851 WARN [main] util.NativeCodeLoader (NativeCodeLoader.java:(62)) - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable hdfs --config hadoop dfs -put <(echo "test") /test 2.74s user 0.21s system 180% cpu 1.638 total % hdfs --config hadoop dfs -chmod 777 /test 2016-04-13 09:25:11,737 WARN [main] util.NativeCodeLoader (NativeCodeLoader.java:(62)) - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable % hdfs --config hadoop dfs -cat /test/bar 2016-04-13 09:25:18,905 WARN [main] util.NativeCodeLoader (NativeCodeLoader.java:(62)) - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable cat: `/test/bar': No such file or directory % hdfs --config hadoop dfs -cat file:///home/kevincox/test/foo 2016-04-13 09:29:34,150 WARN [main] util.NativeCodeLoader (NativeCodeLoader.java:(62)) - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable cat: `file:///Users/kevincox/test/foo': No such file or directory {code} This isn't a huge issue for the command line client because it returns the same exit code but in particular for the {{FileSystem.exists()}} method this is the difference between returning false and throwing an error. > Different results for exist call for file.ext/name > -- > > Key: HDFS-10276 > URL: https://issues.apache.org/jira/browse/HDFS-10276 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Kevin Cox >Assignee: Yuanbo Liu > > Given you have a file {{/file}} an existence check for the path > {{/file/whatever}} will give different responses for different > implementations of FileSystem. > LocalFileSystem will return false while DistributedFileSystem will throw > {{org.apache.hadoop.security.AccessControlException: Permission denied: ..., > access=EXECUTE, ...}} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-10276) Different results for exist call for file.ext/name
[ https://issues.apache.org/jira/browse/HDFS-10276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15238913#comment-15238913 ] Yuanbo Liu commented on HDFS-10276: --- Hi, Kevin Can you provide more details about this issue? such as the steps of reproduction, the version of Hadoop. I have tested {{LocalFileSystem/DistributedFileSystem#getFileStatus}} and {{LocalFileSystem/DistributedFileSystem#exists}} on Hadoop trunk and they both worked as expected. > Different results for exist call for file.ext/name > -- > > Key: HDFS-10276 > URL: https://issues.apache.org/jira/browse/HDFS-10276 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Kevin Cox > > Given you have a file {{/file}} an existence check for the path > {{/file/whatever}} will give different responses for different > implementations of FileSystem. > LocalFileSystem will return false while DistributedFileSystem will throw > {{org.apache.hadoop.security.AccessControlException: Permission denied: ..., > access=EXECUTE, ...}} -- This message was sent by Atlassian JIRA (v6.3.4#6332)