[jira] [Commented] (HDFS-14210) RBF: ModifyACL should work over all the destinations
[ https://issues.apache.org/jira/browse/HDFS-14210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16829710#comment-16829710 ] Íñigo Goiri commented on HDFS-14210: I think the branch is having issues with TestRouterHttpDelegationToken. This should be fine as the code doesn't touch any of the DT part. +1 on [^HDFS-14210-HDFS-13891-05.patch]. Let's hold the commit until HDFs-13891 is fixed. > RBF: ModifyACL should work over all the destinations > > > Key: HDFS-14210 > URL: https://issues.apache.org/jira/browse/HDFS-14210 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Shubham Dewan >Assignee: Shubham Dewan >Priority: Major > Attachments: HDFS-14210-HDFS-13891-04.patch, > HDFS-14210-HDFS-13891-05.patch, HDFS-14210-HDFS-13891.002.patch, > HDFS-14210-HDFS-13891.003.patch, HDFS-14210.001.patch > > > 1) A mount point with multiple destinations. > 2) ./bin/hdfs dfs -setfacl -m user:abc:rwx /testacl > 3) where /testacl => /test1, /test2 > 4) command works for only one destination. > ACL should be set on both of the destinations. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14210) RBF: ModifyACL should work over all the destinations
[
https://issues.apache.org/jira/browse/HDFS-14210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16829607#comment-16829607
]
Hadoop QA commented on HDFS-14210:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 7m
35s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} HDFS-13891 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 19m
3s{color} | {color:green} HDFS-13891 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
29s{color} | {color:green} HDFS-13891 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
21s{color} | {color:green} HDFS-13891 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
33s{color} | {color:green} HDFS-13891 passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
12m 20s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
56s{color} | {color:green} HDFS-13891 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
35s{color} | {color:green} HDFS-13891 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
12m 47s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
31s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 21m 49s{color}
| {color:red} hadoop-hdfs-rbf in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
25s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 80m 44s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.hdfs.server.federation.security.TestRouterHttpDelegationToken |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:8f97d6f |
| JIRA Issue | HDFS-14210 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12967394/HDFS-14210-HDFS-13891-05.patch
|
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux f11b8d428a08 4.4.0-139-generic #165~14.04.1-Ubuntu SMP Wed Oct
31 10:55:11 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | HDFS-13891 / aeb3b61 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_191 |
| findbugs | v3.1.0-RC1 |
| unit |
https://builds.apache.org/job/PreCommit-HDFS-Build/26728/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs-rbf.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/26728/testReport/ |
| Max. process+thread count | 1041 (vs. ulimit of 1) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs-rbf U:
hadoop-hdfs-project/hadoop-hdfs-rbf |
| Console output |
[jira] [Commented] (HDFS-14210) RBF: ModifyACL should work over all the destinations
[
https://issues.apache.org/jira/browse/HDFS-14210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16829547#comment-16829547
]
Hadoop QA commented on HDFS-14210:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
44s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} HDFS-13891 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 23m
30s{color} | {color:green} HDFS-13891 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
34s{color} | {color:green} HDFS-13891 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
23s{color} | {color:green} HDFS-13891 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
34s{color} | {color:green} HDFS-13891 passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
14m 3s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
9s{color} | {color:green} HDFS-13891 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
43s{color} | {color:green} HDFS-13891 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
13m 23s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
34s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 23m 59s{color}
| {color:red} hadoop-hdfs-rbf in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
27s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 84m 15s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.hdfs.server.federation.router.TestRouterFaultTolerant |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:8f97d6f |
| JIRA Issue | HDFS-14210 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12967387/HDFS-14210-HDFS-13891-04.patch
|
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux f756cd65d841 3.13.0-153-generic #203-Ubuntu SMP Thu Jun 14
08:52:28 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | HDFS-13891 / aeb3b61 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_191 |
| findbugs | v3.1.0-RC1 |
| unit |
https://builds.apache.org/job/PreCommit-HDFS-Build/26725/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs-rbf.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/26725/testReport/ |
| Max. process+thread count | 1071 (vs. ulimit of 1) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs-rbf U:
hadoop-hdfs-project/hadoop-hdfs-rbf |
| Console output |
[jira] [Commented] (HDFS-14210) RBF: ModifyACL should work over all the destinations
[ https://issues.apache.org/jira/browse/HDFS-14210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16829457#comment-16829457 ] Ayush Saxena commented on HDFS-14210: - Thanx [~elgoiri] for the review. Have updated patch with the said change. Pls Review!!! > RBF: ModifyACL should work over all the destinations > > > Key: HDFS-14210 > URL: https://issues.apache.org/jira/browse/HDFS-14210 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Shubham Dewan >Assignee: Shubham Dewan >Priority: Major > Attachments: HDFS-14210-HDFS-13891-04.patch, > HDFS-14210-HDFS-13891-05.patch, HDFS-14210-HDFS-13891.002.patch, > HDFS-14210-HDFS-13891.003.patch, HDFS-14210.001.patch > > > 1) A mount point with multiple destinations. > 2) ./bin/hdfs dfs -setfacl -m user:abc:rwx /testacl > 3) where /testacl => /test1, /test2 > 4) command works for only one destination. > ACL should be set on both of the destinations. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14210) RBF: ModifyACL should work over all the destinations
[
https://issues.apache.org/jira/browse/HDFS-14210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16829420#comment-16829420
]
Íñigo Goiri commented on HDFS-14210:
Thanks [~ayushtkn] for the update.
Instead of using guava for the lists, what about just
{{Collecitons#singletonList()}}?
> RBF: ModifyACL should work over all the destinations
>
>
> Key: HDFS-14210
> URL: https://issues.apache.org/jira/browse/HDFS-14210
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Shubham Dewan
>Assignee: Shubham Dewan
>Priority: Major
> Attachments: HDFS-14210-HDFS-13891-04.patch,
> HDFS-14210-HDFS-13891.002.patch, HDFS-14210-HDFS-13891.003.patch,
> HDFS-14210.001.patch
>
>
> 1) A mount point with multiple destinations.
> 2) ./bin/hdfs dfs -setfacl -m user:abc:rwx /testacl
> 3) where /testacl => /test1, /test2
> 4) command works for only one destination.
> ACL should be set on both of the destinations.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14210) RBF: ModifyACL should work over all the destinations
[ https://issues.apache.org/jira/browse/HDFS-14210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16829380#comment-16829380 ] Ayush Saxena commented on HDFS-14210: - Thanx [~shubham.dewan] for the initial work. Have uploaded the patch with fix. Pls Review > RBF: ModifyACL should work over all the destinations > > > Key: HDFS-14210 > URL: https://issues.apache.org/jira/browse/HDFS-14210 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Shubham Dewan >Assignee: Shubham Dewan >Priority: Major > Attachments: HDFS-14210-HDFS-13891-04.patch, > HDFS-14210-HDFS-13891.002.patch, HDFS-14210-HDFS-13891.003.patch, > HDFS-14210.001.patch > > > 1) A mount point with multiple destinations. > 2) ./bin/hdfs dfs -setfacl -m user:abc:rwx /testacl > 3) where /testacl => /test1, /test2 > 4) command works for only one destination. > ACL should be set on both of the destinations. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14210) RBF: ModifyACL should work over all the destinations
[ https://issues.apache.org/jira/browse/HDFS-14210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16829374#comment-16829374 ] Shubham Dewan commented on HDFS-14210: -- I am not working on this anymore. Anyone would like to take it up? > RBF: ModifyACL should work over all the destinations > > > Key: HDFS-14210 > URL: https://issues.apache.org/jira/browse/HDFS-14210 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Shubham Dewan >Assignee: Shubham Dewan >Priority: Major > Attachments: HDFS-14210-HDFS-13891.002.patch, > HDFS-14210-HDFS-13891.003.patch, HDFS-14210.001.patch > > > 1) A mount point with multiple destinations. > 2) ./bin/hdfs dfs -setfacl -m user:abc:rwx /testacl > 3) where /testacl => /test1, /test2 > 4) command works for only one destination. > ACL should be set on both of the destinations. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14210) RBF: ModifyACL should work over all the destinations
[ https://issues.apache.org/jira/browse/HDFS-14210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16769531#comment-16769531 ] Íñigo Goiri commented on HDFS-14210: HDFS-14226 is already committed to HDFS-13891, do you mind rebasing? > RBF: ModifyACL should work over all the destinations > > > Key: HDFS-14210 > URL: https://issues.apache.org/jira/browse/HDFS-14210 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Shubham Dewan >Assignee: Shubham Dewan >Priority: Major > Attachments: HDFS-14210-HDFS-13891.002.patch, > HDFS-14210-HDFS-13891.003.patch, HDFS-14210.001.patch > > > 1) A mount point with multiple destinations. > 2) ./bin/hdfs dfs -setfacl -m user:abc:rwx /testacl > 3) where /testacl => /test1, /test2 > 4) command works for only one destination. > ACL should be set on both of the destinations. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14210) RBF: ModifyACL should work over all the destinations
[ https://issues.apache.org/jira/browse/HDFS-14210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16768972#comment-16768972 ] Shubham Dewan commented on HDFS-14210: -- [~elgoiri] , sure will have a look there once > RBF: ModifyACL should work over all the destinations > > > Key: HDFS-14210 > URL: https://issues.apache.org/jira/browse/HDFS-14210 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Shubham Dewan >Assignee: Shubham Dewan >Priority: Major > Attachments: HDFS-14210-HDFS-13891.002.patch, > HDFS-14210-HDFS-13891.003.patch, HDFS-14210.001.patch > > > 1) A mount point with multiple destinations. > 2) ./bin/hdfs dfs -setfacl -m user:abc:rwx /testacl > 3) where /testacl => /test1, /test2 > 4) command works for only one destination. > ACL should be set on both of the destinations. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14210) RBF: ModifyACL should work over all the destinations
[ https://issues.apache.org/jira/browse/HDFS-14210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16768596#comment-16768596 ] Íñigo Goiri commented on HDFS-14210: [~shubham.dewan], do you mind taking a look at HDFS-14226? Once we commit that one, we can leverage it here. > RBF: ModifyACL should work over all the destinations > > > Key: HDFS-14210 > URL: https://issues.apache.org/jira/browse/HDFS-14210 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Shubham Dewan >Assignee: Shubham Dewan >Priority: Major > Attachments: HDFS-14210-HDFS-13891.002.patch, > HDFS-14210-HDFS-13891.003.patch, HDFS-14210.001.patch > > > 1) A mount point with multiple destinations. > 2) ./bin/hdfs dfs -setfacl -m user:abc:rwx /testacl > 3) where /testacl => /test1, /test2 > 4) command works for only one destination. > ACL should be set on both of the destinations. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14210) RBF: ModifyACL should work over all the destinations
[
https://issues.apache.org/jira/browse/HDFS-14210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16764018#comment-16764018
]
Íñigo Goiri commented on HDFS-14210:
We should have a test for each mount point type.
Then we should have the folders and files setup as it should be for each type
and from there we can fix the issues with the paths.
Let's take this patch to improve the code for this part and give a better
semantic than just {{isPathAll()}}.
> RBF: ModifyACL should work over all the destinations
>
>
> Key: HDFS-14210
> URL: https://issues.apache.org/jira/browse/HDFS-14210
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Shubham Dewan
>Assignee: Shubham Dewan
>Priority: Major
> Attachments: HDFS-14210-HDFS-13891.002.patch,
> HDFS-14210-HDFS-13891.003.patch, HDFS-14210.001.patch
>
>
> 1) A mount point with multiple destinations.
> 2) ./bin/hdfs dfs -setfacl -m user:abc:rwx /testacl
> 3) where /testacl => /test1, /test2
> 4) command works for only one destination.
> ACL should be set on both of the destinations.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14210) RBF: ModifyACL should work over all the destinations
[
https://issues.apache.org/jira/browse/HDFS-14210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16763224#comment-16763224
]
Ayush Saxena commented on HDFS-14210:
-
LOCAL shall be having one only. I guess. That is not a part of pathAll() too.
{code:java}
public boolean isAll() {
DestinationOrder order = getDestOrder();
return order == DestinationOrder.HASH_ALL ||
order == DestinationOrder.RANDOM ||
order == DestinationOrder.SPACE;
{code}
LOCAL is like the local destination shall be tried first as it is written. So
since that isn't part of isPathAll(). It shall also be having in only one as
what I check in mkdir if not part of isPathAll() then call is single only.
So, To our implementation. if use the approach set for setOwner. Checking
isPathAll() shall be correct for all. But just not on HASH and LOCAL if we call
the operation on the mount only. Else that is correct. That is why we can check
if that is mount entry or not.
> RBF: ModifyACL should work over all the destinations
>
>
> Key: HDFS-14210
> URL: https://issues.apache.org/jira/browse/HDFS-14210
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Shubham Dewan
>Assignee: Shubham Dewan
>Priority: Major
> Attachments: HDFS-14210-HDFS-13891.002.patch,
> HDFS-14210-HDFS-13891.003.patch, HDFS-14210.001.patch
>
>
> 1) A mount point with multiple destinations.
> 2) ./bin/hdfs dfs -setfacl -m user:abc:rwx /testacl
> 3) where /testacl => /test1, /test2
> 4) command works for only one destination.
> ACL should be set on both of the destinations.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14210) RBF: ModifyACL should work over all the destinations
[ https://issues.apache.org/jira/browse/HDFS-14210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16763208#comment-16763208 ] Íñigo Goiri commented on HDFS-14210: Approaches that will have folders in all subclusters: HASH_ALL, LOCAL, RANDOM, SPACE (so HASH is the only one that doesn't). Approaches that will have files in all subclusters: RANDOM. > RBF: ModifyACL should work over all the destinations > > > Key: HDFS-14210 > URL: https://issues.apache.org/jira/browse/HDFS-14210 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Shubham Dewan >Assignee: Shubham Dewan >Priority: Major > Attachments: HDFS-14210-HDFS-13891.002.patch, > HDFS-14210-HDFS-13891.003.patch, HDFS-14210.001.patch > > > 1) A mount point with multiple destinations. > 2) ./bin/hdfs dfs -setfacl -m user:abc:rwx /testacl > 3) where /testacl => /test1, /test2 > 4) command works for only one destination. > ACL should be set on both of the destinations. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14210) RBF: ModifyACL should work over all the destinations
[ https://issues.apache.org/jira/browse/HDFS-14210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16763201#comment-16763201 ] Ayush Saxena commented on HDFS-14210: - Why we need to check for folder. A folder creation also follows order too? I checked mkdirs it also has the check isPathAll(). So folder also in case of HASH should be one. Am I missing something? > RBF: ModifyACL should work over all the destinations > > > Key: HDFS-14210 > URL: https://issues.apache.org/jira/browse/HDFS-14210 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Shubham Dewan >Assignee: Shubham Dewan >Priority: Major > Attachments: HDFS-14210-HDFS-13891.002.patch, > HDFS-14210-HDFS-13891.003.patch, HDFS-14210.001.patch > > > 1) A mount point with multiple destinations. > 2) ./bin/hdfs dfs -setfacl -m user:abc:rwx /testacl > 3) where /testacl => /test1, /test2 > 4) command works for only one destination. > ACL should be set on both of the destinations. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14210) RBF: ModifyACL should work over all the destinations
[ https://issues.apache.org/jira/browse/HDFS-14210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16763163#comment-16763163 ] Íñigo Goiri commented on HDFS-14210: I think instead of checking if it's a mount point, we need to check if it's a folder. > RBF: ModifyACL should work over all the destinations > > > Key: HDFS-14210 > URL: https://issues.apache.org/jira/browse/HDFS-14210 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Shubham Dewan >Assignee: Shubham Dewan >Priority: Major > Attachments: HDFS-14210-HDFS-13891.002.patch, > HDFS-14210-HDFS-13891.003.patch, HDFS-14210.001.patch > > > 1) A mount point with multiple destinations. > 2) ./bin/hdfs dfs -setfacl -m user:abc:rwx /testacl > 3) where /testacl => /test1, /test2 > 4) command works for only one destination. > ACL should be set on both of the destinations. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14210) RBF: ModifyACL should work over all the destinations
[
https://issues.apache.org/jira/browse/HDFS-14210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16763143#comment-16763143
]
Ayush Saxena commented on HDFS-14210:
-
{quote}Anyway, my proposal is to split MountTable#isPathAll() into
MountTable#isDirectoryAll() and MountTable#isFileAll().
{quote}
If I am catching up correct. I guess in the invokeAll scenario. We may just
need an extra check that src is a mount entry or not. With isPathAll() and if
src is mount entry than invokeConcurrent or else invokeSequential. I guess this
might work. Because for HASH only there would be multiple dest only for when we
call up on mount else it shall be always one. If we are working through router
only.
Might be something like this ::
{code:java}
/**
* Check if path is a mount entry.
* @param path
* @return true if path is a mount entry.
* @throws IOException
*/
private boolean isMountEntry(final String path) throws IOException {
if (subclusterResolver instanceof MountTableResolver) {
return ((MountTableResolver) subclusterResolver).getMountPoints("/")
.contains(path);
}
return false;
}
public void setOwner(String src, String username, String groupname)
throws IOException {
rpcServer.checkOperation(NameNode.OperationCategory.WRITE);
final List locations =
rpcServer.getLocationsForPath(src, true);
RemoteMethod method = new RemoteMethod("setOwner",
new Class[] { String.class, String.class, String.class },
new RemoteParam(), username, groupname);
if (isPathAll(src) || isMountEntry(src)) {
rpcClient.invokeConcurrent(locations, method);
} else {
rpcClient.invokeSequential(locations, method);
}
}
{code}
[~elgoiri] Pls give a check if that sounds good or still missing part. I think
the inconsistancy that I was catching up was only when we are calling the
setters on mount only.
> RBF: ModifyACL should work over all the destinations
>
>
> Key: HDFS-14210
> URL: https://issues.apache.org/jira/browse/HDFS-14210
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Shubham Dewan
>Assignee: Shubham Dewan
>Priority: Major
> Attachments: HDFS-14210-HDFS-13891.002.patch,
> HDFS-14210-HDFS-13891.003.patch, HDFS-14210.001.patch
>
>
> 1) A mount point with multiple destinations.
> 2) ./bin/hdfs dfs -setfacl -m user:abc:rwx /testacl
> 3) where /testacl => /test1, /test2
> 4) command works for only one destination.
> ACL should be set on both of the destinations.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14210) RBF: ModifyACL should work over all the destinations
[
https://issues.apache.org/jira/browse/HDFS-14210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16762943#comment-16762943
]
Íñigo Goiri commented on HDFS-14210:
Correct, my main concern is doing unnecessary RPC calls.
Another concern is having a misbehaving subcluster which may delay all the
requests because of this.
The other issues is that now we need to process the result of
{{invokeConcurrent()}} and check where it succeeded and where it did not.
{quote}
Problems with this approach what I am feeling is actually if we set on say one
and not on others then that is a bit of inconsistancy since for the end user
that mount is just a mere directory and he is unaware about that there is a
layer of router in between. And if I see someway it feels like half directory
is say erasure coded and half is not. Doesn't sound much logical to me.
{quote}
The main issue that this discussion brings up is that we should distinguish
files and directories.
Setting EC to a folder should happen in all folders and provide a consistent
view across subclusters.
We need to manage temporary unavailable subclusters though.
Anyway, my proposal is to split {{MountTable#isPathAll()}} into
{{MountTable#isDirectoryAll()}} and {{MountTable#isFileAll()}}.
Based on this we can setup the parameters using one approach or the other.
> RBF: ModifyACL should work over all the destinations
>
>
> Key: HDFS-14210
> URL: https://issues.apache.org/jira/browse/HDFS-14210
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Shubham Dewan
>Assignee: Shubham Dewan
>Priority: Major
> Attachments: HDFS-14210-HDFS-13891.002.patch,
> HDFS-14210-HDFS-13891.003.patch, HDFS-14210.001.patch
>
>
> 1) A mount point with multiple destinations.
> 2) ./bin/hdfs dfs -setfacl -m user:abc:rwx /testacl
> 3) where /testacl => /test1, /test2
> 4) command works for only one destination.
> ACL should be set on both of the destinations.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14210) RBF: ModifyACL should work over all the destinations
[ https://issues.apache.org/jira/browse/HDFS-14210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16762844#comment-16762844 ] Ayush Saxena commented on HDFS-14210: - Thanx Everyone for the discussion. What I am getting in, from the discussion. For the files or dirs if the order is HASH there should be only one destination and makes sense to perform operation on it only. Even if we make it invokeConcurrent the operation ideally will be successfull and give us the intended result. The problem I guess what is there by invokeConcurrent there would be unneccessary RPC calls in case of HASH for all others we would be doing concurrently only despite the fact of unneccesary RPC getting failing. The problem that we get up is if that setting operation is on mount point. There are two ways going around 1. invokeConcurrent : Set the policy owner or stuffs like that on all destinations. 2. invokeSequential : Call shall first go on the destination which comes up from the orderresolver for HASH. Problems with this approach what I am feeling is actually if we set on say one and not on others then that is a bit of inconsistancy since for the end user that mount is just a mere directory and he is unaware about that there is a layer of router in between. And if I see someway it feels like half directory is say erasure coded and half is not. Doesn't sound much logical to me. Other is that let say we set it on as per the order HASH on single destination. Somehow that namespace is not available or operation failed up there; the request shall go on other nameservice and if say for owner and permissions that operation shall fail and the user shall be confused only before he was having the permission he didn't change how does the operation is denying him permission. Other for say scenario like EC policy the operation shall be success but the other destination shall not be having the erasure coding. when the user shall esquire the EC policy for the file that shall give no ec policy. Still it would get messier. >From where I am seeing I think it should be concurrent only for HASH too . For >end user that mount is a single directory. so attributes shall be unified only >. All destination should be inline with such parameters. On a though Is there any functionality break kind of scenario if use invokeConcurrent? If anybody can think of. Just my thought. May be I am getting goofed up with facts. Pls Correct if went wrong . :) > RBF: ModifyACL should work over all the destinations > > > Key: HDFS-14210 > URL: https://issues.apache.org/jira/browse/HDFS-14210 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Shubham Dewan >Assignee: Shubham Dewan >Priority: Major > Attachments: HDFS-14210-HDFS-13891.002.patch, > HDFS-14210-HDFS-13891.003.patch, HDFS-14210.001.patch > > > 1) A mount point with multiple destinations. > 2) ./bin/hdfs dfs -setfacl -m user:abc:rwx /testacl > 3) where /testacl => /test1, /test2 > 4) command works for only one destination. > ACL should be set on both of the destinations. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14210) RBF: ModifyACL should work over all the destinations
[
https://issues.apache.org/jira/browse/HDFS-14210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16762020#comment-16762020
]
Íñigo Goiri commented on HDFS-14210:
Let's unify the discussion with [~ayushtkn] in HDFS-14226 here.
So the main question is what should happen with a mount point with multiple
subcluster destinations.
There are 4 types at this point:
* HASH: it will distribute the files across the subclusters and this should be
a mapping of one file (and folder) to one subcluster. If a file is under a
folder (not the root of the mount point) it will go to the subcluster of the
folder.
* HASH_ALL: It will distribute the files across the subclusters and this should
a mapping of one file to one subcluster and folders should be in all
subclusters.
* RANDOM: This is usually for load balancing and all files and folders should
be the same in all subclusters.
* LOCAL: It tries to write to the local subcluster of the writer. It should
create folders everywhere and the files in just one subcluster.
* SPACE: It tries to write to the subcluster based on available space. It
creates the folders everywhere but the files in just one subcluster
Currently, this follows {{RouterClientProtocol#isPathAll()}} which is
ultimately based on {{MountTable#isAll()}} which specifies that the folders
should be in all location (i.e., HASH_ALL, RANDOM, and SPACE).
>From this, it looks like LOCAL is wrong as it should create the folders
>everywhere.
There are some inconsistencies here:
* We should have a distinction between folders in all subclusters (HASH_ALL,
RANDOM, LOCAL, and SPACE) and file in all subclusters (only RANDOM).
* SPACE and LOCAL don't have an order that can be precomputed so we need to
check in all of them (HASH_ALL and HASH we know where the file should be in the
common case).
* For files in one subcluster (all except RANDOM), we should just execute
commands where the file actually is.
> RBF: ModifyACL should work over all the destinations
>
>
> Key: HDFS-14210
> URL: https://issues.apache.org/jira/browse/HDFS-14210
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Shubham Dewan
>Assignee: Shubham Dewan
>Priority: Major
> Attachments: HDFS-14210-HDFS-13891.002.patch,
> HDFS-14210-HDFS-13891.003.patch, HDFS-14210.001.patch
>
>
> 1) A mount point with multiple destinations.
> 2) ./bin/hdfs dfs -setfacl -m user:abc:rwx /testacl
> 3) where /testacl => /test1, /test2
> 4) command works for only one destination.
> ACL should be set on both of the destinations.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14210) RBF: ModifyACL should work over all the destinations
[ https://issues.apache.org/jira/browse/HDFS-14210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16761597#comment-16761597 ] Shubham Dewan commented on HDFS-14210: -- [~elgoiri] , setOwner according to me also should be Concurrent everywhere. There can be inconsistency if one destination have set-A(for reference only) permissions and other has different permissions. Also found same discussion going on HDFS-14226 regarding the sequential and concurrent. According to me, it should be invokeConcurrent here in all cases related to set operations irrespective of the order. [~surendrasingh], what is your opinion over the same? > RBF: ModifyACL should work over all the destinations > > > Key: HDFS-14210 > URL: https://issues.apache.org/jira/browse/HDFS-14210 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Shubham Dewan >Assignee: Shubham Dewan >Priority: Major > Attachments: HDFS-14210-HDFS-13891.002.patch, > HDFS-14210-HDFS-13891.003.patch, HDFS-14210.001.patch > > > 1) A mount point with multiple destinations. > 2) ./bin/hdfs dfs -setfacl -m user:abc:rwx /testacl > 3) where /testacl => /test1, /test2 > 4) command works for only one destination. > ACL should be set on both of the destinations. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14210) RBF: ModifyACL should work over all the destinations
[
https://issues.apache.org/jira/browse/HDFS-14210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16761100#comment-16761100
]
Íñigo Goiri commented on HDFS-14210:
[~shubham.dewan], I think this should follow the same approach as
{{RouterClientProtocol#setOwner()}}.
If the mount table is supposed to be invoked in all paths, it should be
concurrent, otherwise sequential.
Based on that, the tests should have mount points with both types: HASH_ALL and
HASH.
> RBF: ModifyACL should work over all the destinations
>
>
> Key: HDFS-14210
> URL: https://issues.apache.org/jira/browse/HDFS-14210
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Shubham Dewan
>Assignee: Shubham Dewan
>Priority: Major
> Attachments: HDFS-14210-HDFS-13891.002.patch,
> HDFS-14210-HDFS-13891.003.patch, HDFS-14210.001.patch
>
>
> 1) A mount point with multiple destinations.
> 2) ./bin/hdfs dfs -setfacl -m user:abc:rwx /testacl
> 3) where /testacl => /test1, /test2
> 4) command works for only one destination.
> ACL should be set on both of the destinations.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14210) RBF: ModifyACL should work over all the destinations
[
https://issues.apache.org/jira/browse/HDFS-14210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16760475#comment-16760475
]
Shubham Dewan commented on HDFS-14210:
--
Thanks [~elgoiri] for the reveiw
Changes suggested , I will do the required changes.
Regarding the *order* , some workout need to be done in case of multiple
destinations.
{code:java}
We should have tests for different mount points, right now all of them are for
a single destination which actually don't really test this.
{code}
I think all testCases are for multiple destinations not single destination.
*/setfacl* mount point destination is */testdir1* and */testdir2.* Or maybe I
am misunderstanding something.Pls correct me if the case.
> RBF: ModifyACL should work over all the destinations
>
>
> Key: HDFS-14210
> URL: https://issues.apache.org/jira/browse/HDFS-14210
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Shubham Dewan
>Assignee: Shubham Dewan
>Priority: Major
> Attachments: HDFS-14210-HDFS-13891.002.patch,
> HDFS-14210-HDFS-13891.003.patch, HDFS-14210.001.patch
>
>
> 1) A mount point with multiple destinations.
> 2) ./bin/hdfs dfs -setfacl -m user:abc:rwx /testacl
> 3) where /testacl => /test1, /test2
> 4) command works for only one destination.
> ACL should be set on both of the destinations.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14210) RBF: ModifyACL should work over all the destinations
[
https://issues.apache.org/jira/browse/HDFS-14210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16760088#comment-16760088
]
Íñigo Goiri commented on HDFS-14210:
I'm not sure we should do this for everybody all the time.
HASH_ALL and so on, yes they need it but HASH or others should be executed in
the ones that succeed.
We should have tests for different mount points, right now all of them are for
a single destination which actually don't really test this.
A few other minor comments:
* Fix the checkstyle.
* User {{assertNotEquals()}} instead of assert false for equals.
* Fix capitalization to {{testRemoveAclEntries()}}, {{testSetAcl()}} and
{{testModifyAclEntries()}}.
* Don't use javadoc comment in TestRouterACL#112,228.
* The test takes very long as we are setting up a cluster for every test; we
should make it @BeforeClass and @AfterClass.
* The mount points could be created in the beginning.
* You may want to set {{routerProtocol}} in the class itself instead of doing
it for every test. Actually, setting up the folders and deleting them could be
done in @Before and @After.
> RBF: ModifyACL should work over all the destinations
>
>
> Key: HDFS-14210
> URL: https://issues.apache.org/jira/browse/HDFS-14210
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Shubham Dewan
>Assignee: Shubham Dewan
>Priority: Major
> Attachments: HDFS-14210-HDFS-13891.002.patch,
> HDFS-14210-HDFS-13891.003.patch, HDFS-14210.001.patch
>
>
> 1) A mount point with multiple destinations.
> 2) ./bin/hdfs dfs -setfacl -m user:abc:rwx /testacl
> 3) where /testacl => /test1, /test2
> 4) command works for only one destination.
> ACL should be set on both of the destinations.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14210) RBF: ModifyACL should work over all the destinations
[
https://issues.apache.org/jira/browse/HDFS-14210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16759680#comment-16759680
]
Hadoop QA commented on HDFS-14210:
--
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
16s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} HDFS-13891 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 19m
4s{color} | {color:green} HDFS-13891 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
31s{color} | {color:green} HDFS-13891 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
21s{color} | {color:green} HDFS-13891 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
33s{color} | {color:green} HDFS-13891 passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
12m 37s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
54s{color} | {color:green} HDFS-13891 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
36s{color} | {color:green} HDFS-13891 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
25s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 17s{color} | {color:orange} hadoop-hdfs-project/hadoop-hdfs-rbf: The patch
generated 1 new + 4 unchanged - 0 fixed = 5 total (was 4) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
12m 41s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
35s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 22m
29s{color} | {color:green} hadoop-hdfs-rbf in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
27s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 74m 39s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:8f97d6f |
| JIRA Issue | HDFS-14210 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12957479/HDFS-14210-HDFS-13891.003.patch
|
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux a6c34265b777 4.4.0-139-generic #165-Ubuntu SMP Wed Oct 24
10:58:50 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | HDFS-13891 / d37590b |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_191 |
| findbugs | v3.1.0-RC1 |
| checkstyle |
https://builds.apache.org/job/PreCommit-HDFS-Build/26143/artifact/out/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs-rbf.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/26143/testReport/ |
| Max. process+thread count | 1453 (vs. ulimit of 1) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs-rbf U:
hadoop-hdfs-project/hadoop-hdfs-rbf |
| Console output |
[jira] [Commented] (HDFS-14210) RBF: ModifyACL should work over all the destinations
[ https://issues.apache.org/jira/browse/HDFS-14210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16759660#comment-16759660 ] Shubham Dewan commented on HDFS-14210: -- [~elgoiri] taken care of the review comments. Also, similar changes have been done for the Api's related to ACL's and added test cases for the same.Please have a look. > RBF: ModifyACL should work over all the destinations > > > Key: HDFS-14210 > URL: https://issues.apache.org/jira/browse/HDFS-14210 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Shubham Dewan >Assignee: Shubham Dewan >Priority: Major > Attachments: HDFS-14210-HDFS-13891.002.patch, > HDFS-14210-HDFS-13891.003.patch, HDFS-14210.001.patch > > > 1) A mount point with multiple destinations. > 2) ./bin/hdfs dfs -setfacl -m user:abc:rwx /testacl > 3) where /testacl => /test1, /test2 > 4) command works for only one destination. > ACL should be set on both of the destinations. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14210) RBF: ModifyACL should work over all the destinations
[
https://issues.apache.org/jira/browse/HDFS-14210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16752598#comment-16752598
]
Íñigo Goiri commented on HDFS-14210:
I'm still not sure you need the {{MultipleDestinationMountTableResolver}}.
{{TestRotuerRpcMultipleDestination}} should cover this part of the code with
the concurrent, etc.
Minor comments:
* {{routerConf.set(RBFConfigKeys.DFS_ROUTER_QUOTA_CACHE_UPATE_INTERVAL,
"2s");}} could use {{setTimeDuration()}}.
* Proper capitalization for {{testmodifyaclentries()}}.
> RBF: ModifyACL should work over all the destinations
>
>
> Key: HDFS-14210
> URL: https://issues.apache.org/jira/browse/HDFS-14210
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Shubham Dewan
>Assignee: Shubham Dewan
>Priority: Major
> Attachments: HDFS-14210-HDFS-13891.002.patch, HDFS-14210.001.patch
>
>
> 1) A mount point with multiple destinations.
> 2) ./bin/hdfs dfs -setfacl -m user:abc:rwx /testacl
> 3) where /testacl => /test1, /test2
> 4) command works for only one destination.
> ACL should be set on both of the destinations.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14210) RBF: ModifyACL should work over all the destinations
[ https://issues.apache.org/jira/browse/HDFS-14210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16751905#comment-16751905 ] Shubham Dewan commented on HDFS-14210: -- [~elgoiri], I checked ,under relative-path +_org/apache/hadoop/hdfs/server/federation/router/_+ only *TestRouterAllResolver* is using MultipleDestinationMountTableResolver. > RBF: ModifyACL should work over all the destinations > > > Key: HDFS-14210 > URL: https://issues.apache.org/jira/browse/HDFS-14210 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Shubham Dewan >Assignee: Shubham Dewan >Priority: Major > Attachments: HDFS-14210-HDFS-13891.002.patch, HDFS-14210.001.patch > > > 1) A mount point with multiple destinations. > 2) ./bin/hdfs dfs -setfacl -m user:abc:rwx /testacl > 3) where /testacl => /test1, /test2 > 4) command works for only one destination. > ACL should be set on both of the destinations. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14210) RBF: ModifyACL should work over all the destinations
[ https://issues.apache.org/jira/browse/HDFS-14210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16751332#comment-16751332 ] Íñigo Goiri commented on HDFS-14210: Don't we have any test using MultipleDestinationMountTableResolver? > RBF: ModifyACL should work over all the destinations > > > Key: HDFS-14210 > URL: https://issues.apache.org/jira/browse/HDFS-14210 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Shubham Dewan >Assignee: Shubham Dewan >Priority: Major > Attachments: HDFS-14210-HDFS-13891.002.patch, HDFS-14210.001.patch > > > 1) A mount point with multiple destinations. > 2) ./bin/hdfs dfs -setfacl -m user:abc:rwx /testacl > 3) where /testacl => /test1, /test2 > 4) command works for only one destination. > ACL should be set on both of the destinations. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14210) RBF: ModifyACL should work over all the destinations
[ https://issues.apache.org/jira/browse/HDFS-14210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16750711#comment-16750711 ] Shubham Dewan commented on HDFS-14210: -- Thanks [~elgoiri] for a quick review. I tried reuse TestRouterQuota class but faced issue due to MountTableResolver as in our Test case scenario we are using multiple destinations for which we need MultipleDestinationMountTableResolver .Hence added a new class for clear understanding. > RBF: ModifyACL should work over all the destinations > > > Key: HDFS-14210 > URL: https://issues.apache.org/jira/browse/HDFS-14210 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Shubham Dewan >Assignee: Shubham Dewan >Priority: Major > Attachments: HDFS-14210-HDFS-13891.002.patch, HDFS-14210.001.patch > > > 1) A mount point with multiple destinations. > 2) ./bin/hdfs dfs -setfacl -m user:abc:rwx /testacl > 3) where /testacl => /test1, /test2 > 4) command works for only one destination. > ACL should be set on both of the destinations. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14210) RBF: ModifyACL should work over all the destinations
[ https://issues.apache.org/jira/browse/HDFS-14210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16750631#comment-16750631 ] Íñigo Goiri commented on HDFS-14210: Thanks [~shubham.dewan] for the patch. Do we need a new test with a new MiniDFSCluster? Can we reuse TestRouterRPC or one of those? > RBF: ModifyACL should work over all the destinations > > > Key: HDFS-14210 > URL: https://issues.apache.org/jira/browse/HDFS-14210 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Shubham Dewan >Assignee: Shubham Dewan >Priority: Major > Attachments: HDFS-14210-HDFS-13891.002.patch, HDFS-14210.001.patch > > > 1) A mount point with multiple destinations. > 2) ./bin/hdfs dfs -setfacl -m user:abc:rwx /testacl > 3) where /testacl => /test1, /test2 > 4) command works for only one destination. > ACL should be set on both of the destinations. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14210) RBF: ModifyACL should work over all the destinations
[
https://issues.apache.org/jira/browse/HDFS-14210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16749957#comment-16749957
]
Hadoop QA commented on HDFS-14210:
--
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
17s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} HDFS-13891 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 20m
1s{color} | {color:green} HDFS-13891 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
29s{color} | {color:green} HDFS-13891 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
20s{color} | {color:green} HDFS-13891 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
32s{color} | {color:green} HDFS-13891 passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
13m 7s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
49s{color} | {color:green} HDFS-13891 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
33s{color} | {color:green} HDFS-13891 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
25s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 16s{color} | {color:orange} hadoop-hdfs-project/hadoop-hdfs-rbf: The patch
generated 1 new + 4 unchanged - 0 fixed = 5 total (was 4) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
13m 18s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
30s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 22m
37s{color} | {color:green} hadoop-hdfs-rbf in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
25s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 76m 24s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:8f97d6f |
| JIRA Issue | HDFS-14210 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12955955/HDFS-14210-HDFS-13891.002.patch
|
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux 5d41194ea2c3 3.13.0-153-generic #203-Ubuntu SMP Thu Jun 14
08:52:28 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | HDFS-13891 / 6ee477b |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_191 |
| findbugs | v3.1.0-RC1 |
| checkstyle |
https://builds.apache.org/job/PreCommit-HDFS-Build/26030/artifact/out/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs-rbf.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/26030/testReport/ |
| Max. process+thread count | 985 (vs. ulimit of 1) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs-rbf U:
hadoop-hdfs-project/hadoop-hdfs-rbf |
| Console output |
[jira] [Commented] (HDFS-14210) RBF: ModifyACL should work over all the destinations
[ https://issues.apache.org/jira/browse/HDFS-14210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16749857#comment-16749857 ] Shubham Dewan commented on HDFS-14210: -- [~elgoiri] ,pls review the patch [^HDFS-14210-HDFS-13891.002.patch] . > RBF: ModifyACL should work over all the destinations > > > Key: HDFS-14210 > URL: https://issues.apache.org/jira/browse/HDFS-14210 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Shubham Dewan >Assignee: Shubham Dewan >Priority: Major > Attachments: HDFS-14210-HDFS-13891.002.patch, HDFS-14210.001.patch > > > 1) A mount point with multiple destinations. > 2) ./bin/hdfs dfs -setfacl -m user:abc:rwx /testacl > 3) where /testacl => /test1, /test2 > 4) command works for only one destination. > ACL should be set on both of the destinations. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14210) RBF: ModifyACL should work over all the destinations
[ https://issues.apache.org/jira/browse/HDFS-14210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16744307#comment-16744307 ] Íñigo Goiri commented on HDFS-14210: Thanks [~shubham.dewan] for [^HDFS-14210.001.patch]. I think it makes sense to make it everywhere as you mention. Could you add a unit test? > RBF: ModifyACL should work over all the destinations > > > Key: HDFS-14210 > URL: https://issues.apache.org/jira/browse/HDFS-14210 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Shubham Dewan >Assignee: Shubham Dewan >Priority: Major > Attachments: HDFS-14210.001.patch > > > 1) A mount point with multiple destinations. > 2) ./bin/hdfs dfs -setfacl -m user:abc:rwx /testacl > 3) where /testacl => /test1, /test2 > 4) command works for only one destination. > ACL should be set on both of the destinations. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
