[jira] [Commented] (YARN-6727) Improve getQueueUserAcls API to query for specific queue and user
[ https://issues.apache.org/jira/browse/YARN-6727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16216741#comment-16216741 ] Bibin A Chundatt commented on YARN-6727: [~sunilg]/[~naganarasimha...@apache.org] Any more comments from you side. IMHO this would help is reducing lock on leaf queue for getAcl API > Improve getQueueUserAcls API to query for specific queue and user > -- > > Key: YARN-6727 > URL: https://issues.apache.org/jira/browse/YARN-6727 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Bibin A Chundatt >Assignee: Bibin A Chundatt > Attachments: YARN-6727.001.patch, YARN-6727.002.patch, > YARN-6727.003.patch, YARN-6727.004.patch, YARN-6727.005.patch, > YARN-6727.006.patch, YARN-6727.WIP.patch > > > Currently {{ApplicationClientProtocol#getQueueUserAcls}} return data for all > the queues available in scheduler for user. > User wants to know whether he has rights of a particular queue only. For > systems with 5K queues returning all queues list is not efficient. > Suggested change: support additional parameters *userName and queueName* as > optional. Admin user should be able to query other users ACL for a particular > queueName. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6727) Improve getQueueUserAcls API to query for specific queue and user
[ https://issues.apache.org/jira/browse/YARN-6727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16180357#comment-16180357 ] Bibin A Chundatt commented on YARN-6727: Testcase failures are not related to patch attached. [~sunilg]/[~naganarasimha...@apache.org] Please review patch attached > Improve getQueueUserAcls API to query for specific queue and user > -- > > Key: YARN-6727 > URL: https://issues.apache.org/jira/browse/YARN-6727 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Bibin A Chundatt >Assignee: Bibin A Chundatt > Attachments: YARN-6727.001.patch, YARN-6727.002.patch, > YARN-6727.003.patch, YARN-6727.004.patch, YARN-6727.005.patch, > YARN-6727.006.patch, YARN-6727.WIP.patch > > > Currently {{ApplicationClientProtocol#getQueueUserAcls}} return data for all > the queues available in scheduler for user. > User wants to know whether he has rights of a particular queue only. For > systems with 5K queues returning all queues list is not efficient. > Suggested change: support additional parameters *userName and queueName* as > optional. Admin user should be able to query other users ACL for a particular > queueName. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6727) Improve getQueueUserAcls API to query for specific queue and user
[ https://issues.apache.org/jira/browse/YARN-6727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16167496#comment-16167496 ] Hadoop QA commented on YARN-6727: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 17s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 4 new or modified test files. {color} | || || || || {color:brown} trunk Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 42s{color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 12m 54s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 9m 15s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 49s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 47s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m 15s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 25s{color} | {color:green} trunk passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 9s{color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 24s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m 0s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} cc {color} | {color:green} 6m 0s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 6m 0s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 48s{color} | {color:green} hadoop-yarn-project/hadoop-yarn: The patch generated 0 new + 75 unchanged - 1 fixed = 75 total (was 76) {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 52s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m 32s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 22s{color} | {color:green} hadoop-yarn-api in the patch passed. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 35s{color} | {color:green} hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common generated 0 new + 4458 unchanged - 1 fixed = 4458 total (was 4459) {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 22s{color} | {color:green} hadoop-yarn-server-resourcemanager in the patch passed. {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 32s{color} | {color:green} hadoop-yarn-api in the patch passed. {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m 38s{color} | {color:green} hadoop-yarn-common in the patch passed. {color} | | {color:red}-1{color} | {color:red} unit {color} | {color:red} 46m 51s{color} | {color:red} hadoop-yarn-server-resourcemanager in the patch failed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 31s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black}105m 10s{color} | {color:black} {color} | \\ \\ || Reason || Tests || | Failed junit tests | hadoop.yarn.server.resourcemanager.scheduler.capacity.TestContainerAllocation | \\ \\ || Subsystem || Report/Notes || | Docker | Image:yetus/hadoop:71bbb86 | | JIRA Issue | YARN-6727 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12887062/YARN-6727.006.patch | | Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc | | uname | Linux
[jira] [Commented] (YARN-6727) Improve getQueueUserAcls API to query for specific queue and user
[ https://issues.apache.org/jira/browse/YARN-6727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16154894#comment-16154894 ] Hadoop QA commented on YARN-6727: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 22s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 4 new or modified test files. {color} | || || || || {color:brown} trunk Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 9s{color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 14m 37s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 10m 19s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m 0s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m 1s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m 36s{color} | {color:green} trunk passed {color} | | {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m 47s{color} | {color:red} hadoop-yarn-common in trunk failed. {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 11s{color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 44s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m 34s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} cc {color} | {color:green} 6m 34s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 6m 34s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 56s{color} | {color:green} hadoop-yarn-project/hadoop-yarn: The patch generated 0 new + 75 unchanged - 1 fixed = 75 total (was 76) {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 56s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m 48s{color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m 43s{color} | {color:red} hadoop-yarn-common in the patch failed. {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 33s{color} | {color:green} hadoop-yarn-api in the patch passed. {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m 31s{color} | {color:green} hadoop-yarn-common in the patch passed. {color} | | {color:red}-1{color} | {color:red} unit {color} | {color:red} 46m 39s{color} | {color:red} hadoop-yarn-server-resourcemanager in the patch failed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 27s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black}109m 9s{color} | {color:black} {color} | \\ \\ || Reason || Tests || | Failed junit tests | hadoop.yarn.server.resourcemanager.scheduler.capacity.TestContainerAllocation | | Timed out junit tests | org.apache.hadoop.yarn.server.resourcemanager.TestSubmitApplicationWithRMHA | \\ \\ || Subsystem || Report/Notes || | Docker | Image:yetus/hadoop:71bbb86 | | JIRA Issue | YARN-6727 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12885380/YARN-6727.005.patch | | Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc | | uname | Linux 8c874360dd66 3.13.0-116-generic #163-Ubuntu SMP Fri Mar 31 14:13:22 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh | | git revision | trunk / d4035d4 | | Default Java | 1.8.0_144 | | findbugs |
[jira] [Commented] (YARN-6727) Improve getQueueUserAcls API to query for specific queue and user
[ https://issues.apache.org/jira/browse/YARN-6727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16154856#comment-16154856 ] Sunil G commented on YARN-6727: --- Sure. I think its make sense to add this annotation for new apis added. > Improve getQueueUserAcls API to query for specific queue and user > -- > > Key: YARN-6727 > URL: https://issues.apache.org/jira/browse/YARN-6727 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Bibin A Chundatt >Assignee: Bibin A Chundatt > Attachments: YARN-6727.001.patch, YARN-6727.002.patch, > YARN-6727.003.patch, YARN-6727.004.patch, YARN-6727.005.patch, > YARN-6727.WIP.patch > > > Currently {{ApplicationClientProtocol#getQueueUserAcls}} return data for all > the queues available in scheduler for user. > User wants to know whether he has rights of a particular queue only. For > systems with 5K queues returning all queues list is not efficient. > Suggested change: support additional parameters *userName and queueName* as > optional. Admin user should be able to query other users ACL for a particular > queueName. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6727) Improve getQueueUserAcls API to query for specific queue and user
[ https://issues.apache.org/jira/browse/YARN-6727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16154809#comment-16154809 ] Bibin A Chundatt commented on YARN-6727: {quote} GetQueueUserAclsInfoRequest apis are updated in this new patch. And its going 3.1.0. I think its very early to call out as Stable. {quote} How about only for methods we mark as unstable ? > Improve getQueueUserAcls API to query for specific queue and user > -- > > Key: YARN-6727 > URL: https://issues.apache.org/jira/browse/YARN-6727 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Bibin A Chundatt >Assignee: Bibin A Chundatt > Attachments: YARN-6727.001.patch, YARN-6727.002.patch, > YARN-6727.003.patch, YARN-6727.004.patch, YARN-6727.005.patch, > YARN-6727.WIP.patch > > > Currently {{ApplicationClientProtocol#getQueueUserAcls}} return data for all > the queues available in scheduler for user. > User wants to know whether he has rights of a particular queue only. For > systems with 5K queues returning all queues list is not efficient. > Suggested change: support additional parameters *userName and queueName* as > optional. Admin user should be able to query other users ACL for a particular > queueName. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6727) Improve getQueueUserAcls API to query for specific queue and user
[ https://issues.apache.org/jira/browse/YARN-6727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16153816#comment-16153816 ] Hadoop QA commented on YARN-6727: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 18s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 4 new or modified test files. {color} | || || || || {color:brown} trunk Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 10s{color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 13m 39s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 9m 30s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 57s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 59s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m 35s{color} | {color:green} trunk passed {color} | | {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m 46s{color} | {color:red} hadoop-yarn-common in trunk failed. {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 10s{color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 35s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 5m 40s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} cc {color} | {color:green} 5m 40s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 5m 40s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 56s{color} | {color:green} hadoop-yarn-project/hadoop-yarn: The patch generated 0 new + 75 unchanged - 1 fixed = 75 total (was 76) {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 54s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m 47s{color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m 42s{color} | {color:red} hadoop-yarn-common in the patch failed. {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 32s{color} | {color:green} hadoop-yarn-api in the patch passed. {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m 30s{color} | {color:green} hadoop-yarn-common in the patch passed. {color} | | {color:red}-1{color} | {color:red} unit {color} | {color:red} 44m 12s{color} | {color:red} hadoop-yarn-server-resourcemanager in the patch failed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 27s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black}103m 30s{color} | {color:black} {color} | \\ \\ || Reason || Tests || | Failed junit tests | hadoop.yarn.server.resourcemanager.scheduler.capacity.TestContainerResizing | | | hadoop.yarn.server.resourcemanager.scheduler.capacity.TestIncreaseAllocationExpirer | | | hadoop.yarn.server.resourcemanager.scheduler.capacity.TestContainerAllocation | \\ \\ || Subsystem || Report/Notes || | Docker | Image:yetus/hadoop:71bbb86 | | JIRA Issue | YARN-6727 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12885380/YARN-6727.005.patch | | Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc | | uname | Linux 7ca6a986c702 3.13.0-119-generic #166-Ubuntu SMP Wed May 3 12:18:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh | | git
[jira] [Commented] (YARN-6727) Improve getQueueUserAcls API to query for specific queue and user
[ https://issues.apache.org/jira/browse/YARN-6727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16153656#comment-16153656 ] Sunil G commented on YARN-6727: --- {{GetQueueUserAclsInfoRequest}} apis are updated in this new patch. And its going 3.1.0. I think its very early to call out as Stable. > Improve getQueueUserAcls API to query for specific queue and user > -- > > Key: YARN-6727 > URL: https://issues.apache.org/jira/browse/YARN-6727 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Bibin A Chundatt >Assignee: Bibin A Chundatt > Attachments: YARN-6727.001.patch, YARN-6727.002.patch, > YARN-6727.003.patch, YARN-6727.004.patch, YARN-6727.005.patch, > YARN-6727.WIP.patch > > > Currently {{ApplicationClientProtocol#getQueueUserAcls}} return data for all > the queues available in scheduler for user. > User wants to know whether he has rights of a particular queue only. For > systems with 5K queues returning all queues list is not efficient. > Suggested change: support additional parameters *userName and queueName* as > optional. Admin user should be able to query other users ACL for a particular > queueName. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6727) Improve getQueueUserAcls API to query for specific queue and user
[ https://issues.apache.org/jira/browse/YARN-6727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16153393#comment-16153393 ] Sunil G commented on YARN-6727: --- Sorry for the delay. Thanks [~bibinchundatt] # javadoc in GetQueueUserAclsInfoRequest is not very clear. Please help to update same {{getQueueUserAcls}} # I think we can keep apis in GetQueueUserAclsInfoRequest as *Unstable*. Or were there any other discussion on same? # Remove extra parenthesis in {{setUserName}}. ref {{GetQueueUserAclsInfoRequestPBImpl}} # {{getPrivilageEntity}} to {{getPrivilegeEntity}} # If we use an old client {{List queueNames = request.getQueueNames();}} could come as NULL. ref {{ClientRMService#getQueueUserAcls}} # In {{QueueACLsManager#getQueueUserAclInfo}}, ugi is passed and checking against that. This might cause some backward compatibility issue as we are now more stricter ? > Improve getQueueUserAcls API to query for specific queue and user > -- > > Key: YARN-6727 > URL: https://issues.apache.org/jira/browse/YARN-6727 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Bibin A Chundatt >Assignee: Bibin A Chundatt > Attachments: YARN-6727.001.patch, YARN-6727.002.patch, > YARN-6727.003.patch, YARN-6727.004.patch, YARN-6727.WIP.patch > > > Currently {{ApplicationClientProtocol#getQueueUserAcls}} return data for all > the queues available in scheduler for user. > User wants to know whether he has rights of a particular queue only. For > systems with 5K queues returning all queues list is not efficient. > Suggested change: support additional parameters *userName and queueName* as > optional. Admin user should be able to query other users ACL for a particular > queueName. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6727) Improve getQueueUserAcls API to query for specific queue and user
[ https://issues.apache.org/jira/browse/YARN-6727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16153189#comment-16153189 ] Hadoop QA commented on YARN-6727: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 21s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 4 new or modified test files. {color} | || || || || {color:brown} trunk Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 22s{color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 15m 0s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 9m 59s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m 3s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m 7s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m 51s{color} | {color:green} trunk passed {color} | | {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m 44s{color} | {color:red} hadoop-yarn-common in trunk failed. {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 10s{color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 38s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m 19s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} cc {color} | {color:green} 6m 19s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 6m 19s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m 0s{color} | {color:green} hadoop-yarn-project/hadoop-yarn: The patch generated 0 new + 75 unchanged - 1 fixed = 75 total (was 76) {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m 12s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m 27s{color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m 46s{color} | {color:red} hadoop-yarn-common in the patch failed. {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 38s{color} | {color:green} hadoop-yarn-api in the patch passed. {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m 39s{color} | {color:green} hadoop-yarn-common in the patch passed. {color} | | {color:red}-1{color} | {color:red} unit {color} | {color:red} 45m 41s{color} | {color:red} hadoop-yarn-server-resourcemanager in the patch failed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 31s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black}109m 35s{color} | {color:black} {color} | \\ \\ || Reason || Tests || | Failed junit tests | hadoop.yarn.server.resourcemanager.scheduler.capacity.TestContainerAllocation | \\ \\ || Subsystem || Report/Notes || | Docker | Image:yetus/hadoop:71bbb86 | | JIRA Issue | YARN-6727 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12885298/YARN-6727.004.patch | | Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc | | uname | Linux cf63fce47b4a 3.13.0-119-generic #166-Ubuntu SMP Wed May 3 12:18:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh | | git revision | trunk / ed162b7 | | Default Java | 1.8.0_144 | | findbugs | v3.1.0-RC1 | | javadoc |
[jira] [Commented] (YARN-6727) Improve getQueueUserAcls API to query for specific queue and user
[ https://issues.apache.org/jira/browse/YARN-6727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16145881#comment-16145881 ] Daniel Templeton commented on YARN-6727: Thanks for the patch. Here are my comments: # Minor nit: this _if_:{code} if (!applicationsACLsManager.isAdmin(currentUGI) && userName != null && !userName.equals(currentUGI.getShortUserName())) {{code} would be more efficient if you did the null check first. # I don't like ternary operators unless they actually make things more readable. This one:{code} UserGroupInformation ugi = userName == null ? currentUGI : UserGroupInformation.createRemoteUser(userName);{code} doesn't do it for me. It's much clearer to me to reuse {{currentUGI}}:{code} if (userName != null) { currentUGI = UserGroupInformation.createRemoteUser(userName); }{code} # You could apply the diamond operator in several places in {{QueueACLsManager}}. # I prefer {{x != null}} to {{null != x}}. # Any way we can avoid doing the string math in {{getShortQueueName()}}? # Please add failure messages to the {{assertTrue()}} calls in the test code. > Improve getQueueUserAcls API to query for specific queue and user > -- > > Key: YARN-6727 > URL: https://issues.apache.org/jira/browse/YARN-6727 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Bibin A Chundatt >Assignee: Bibin A Chundatt > Attachments: YARN-6727.001.patch, YARN-6727.002.patch, > YARN-6727.003.patch, YARN-6727.WIP.patch > > > Currently {{ApplicationClientProtocol#getQueueUserAcls}} return data for all > the queues available in scheduler for user. > User wants to know whether he has rights of a particular queue only. For > systems with 5K queues returning all queues list is not efficient. > Suggested change: support additional parameters *userName and queueName* as > optional. Admin user should be able to query other users ACL for a particular > queueName. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6727) Improve getQueueUserAcls API to query for specific queue and user
[ https://issues.apache.org/jira/browse/YARN-6727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16125221#comment-16125221 ] Bibin A Chundatt commented on YARN-6727: [~naganarasimha...@apache.org]/[~sunilg] Please review patch attached > Improve getQueueUserAcls API to query for specific queue and user > -- > > Key: YARN-6727 > URL: https://issues.apache.org/jira/browse/YARN-6727 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Bibin A Chundatt >Assignee: Bibin A Chundatt > Attachments: YARN-6727.001.patch, YARN-6727.002.patch, > YARN-6727.003.patch, YARN-6727.WIP.patch > > > Currently {{ApplicationClientProtocol#getQueueUserAcls}} return data for all > the queues available in scheduler for user. > User wants to know whether he has rights of a particular queue only. For > systems with 5K queues returning all queues list is not efficient. > Suggested change: support additional parameters *userName and queueName* as > optional. Admin user should be able to query other users ACL for a particular > queueName. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6727) Improve getQueueUserAcls API to query for specific queue and user
[ https://issues.apache.org/jira/browse/YARN-6727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16122964#comment-16122964 ] Hadoop QA commented on YARN-6727: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 21s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 4 new or modified test files. {color} | || || || || {color:brown} trunk Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 46s{color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 17m 44s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 11m 23s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m 8s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m 9s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m 16s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 51s{color} | {color:green} trunk passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 11s{color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 43s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m 52s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} cc {color} | {color:green} 6m 52s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 6m 52s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m 4s{color} | {color:green} hadoop-yarn-project/hadoop-yarn: The patch generated 0 new + 75 unchanged - 1 fixed = 75 total (was 76) {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m 11s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m 18s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 30s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 32s{color} | {color:green} hadoop-yarn-api in the patch passed. {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m 27s{color} | {color:green} hadoop-yarn-common in the patch passed. {color} | | {color:red}-1{color} | {color:red} unit {color} | {color:red} 42m 58s{color} | {color:red} hadoop-yarn-server-resourcemanager in the patch failed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 29s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black}112m 15s{color} | {color:black} {color} | \\ \\ || Reason || Tests || | Failed junit tests | hadoop.yarn.server.resourcemanager.security.TestDelegationTokenRenewer | \\ \\ || Subsystem || Report/Notes || | Docker | Image:yetus/hadoop:14b5c93 | | JIRA Issue | YARN-6727 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12881391/YARN-6727.003.patch | | Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc | | uname | Linux dc11a4653b9f 3.13.0-119-generic #166-Ubuntu SMP Wed May 3 12:18:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh | | git revision | trunk / a32e013 | | Default Java | 1.8.0_144 | | findbugs | v3.1.0-RC1 | | unit |
[jira] [Commented] (YARN-6727) Improve getQueueUserAcls API to query for specific queue and user
[ https://issues.apache.org/jira/browse/YARN-6727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16116297#comment-16116297 ] Hadoop QA commented on YARN-6727: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 18s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 4 new or modified test files. {color} | || || || || {color:brown} trunk Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 41s{color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 15m 19s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 9m 52s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 58s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m 17s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m 51s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 38s{color} | {color:green} trunk passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 10s{color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 33s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 5m 42s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} cc {color} | {color:green} 5m 42s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 5m 42s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 55s{color} | {color:green} hadoop-yarn-project/hadoop-yarn: The patch generated 0 new + 76 unchanged - 1 fixed = 76 total (was 77) {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 54s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m 21s{color} | {color:red} hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 33s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 40s{color} | {color:green} hadoop-yarn-api in the patch passed. {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m 52s{color} | {color:green} hadoop-yarn-common in the patch passed. {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 46m 29s{color} | {color:green} hadoop-yarn-server-resourcemanager in the patch passed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 33s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black}109m 32s{color} | {color:black} {color} | \\ \\ || Reason || Tests || | FindBugs | module:hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager | | | Comparison of String objects using == or != in org.apache.hadoop.yarn.server.resourcemanager.ClientRMService.getQueueUserAcls(GetQueueUserAclsInfoRequest) At ClientRMService.java:== or != in org.apache.hadoop.yarn.server.resourcemanager.ClientRMService.getQueueUserAcls(GetQueueUserAclsInfoRequest) At ClientRMService.java:[line 1015] | \\ \\ || Subsystem || Report/Notes || | Docker | Image:yetus/hadoop:14b5c93 | | JIRA Issue | YARN-6727 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12880601/YARN-6727.002.patch | | Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
[jira] [Commented] (YARN-6727) Improve getQueueUserAcls API to query for specific queue and user
[ https://issues.apache.org/jira/browse/YARN-6727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16114391#comment-16114391 ] Naganarasimha G R commented on YARN-6727: - Thanks [~bibinchundatt], At first glance, approach looks good to me but few nits i observed was : # YarnAuthorizationProvider.java ln 122: ??Get all permissions?? is invalid # ClientRMService.java ln no 1022: there is possibility that the user is not admin but set his name as {{userName}} in the request, hence we should not throw exception # ClientRMService.java ln no 1026 - 1039 : logic can be little optimized as we are calling twice {{queueACLsManager.getUserQueueACLInfo(queueNames, userUGI))}} # Some checkstyle issues seems to be valid Will review completely and update if any more present. > Improve getQueueUserAcls API to query for specific queue and user > -- > > Key: YARN-6727 > URL: https://issues.apache.org/jira/browse/YARN-6727 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Bibin A Chundatt >Assignee: Bibin A Chundatt > Attachments: YARN-6727.001.patch, YARN-6727.WIP.patch > > > Currently {{ApplicationClientProtocol#getQueueUserAcls}} return data for all > the queues available in scheduler for user. > User wants to know whether he has rights of a particular queue only. For > systems with 5K queues returning all queues list is not efficient. > Suggested change: support additional parameters *userName and queueName* as > optional. Admin user should be able to query other users ACL for a particular > queueName. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6727) Improve getQueueUserAcls API to query for specific queue and user
[ https://issues.apache.org/jira/browse/YARN-6727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16112420#comment-16112420 ] Hadoop QA commented on YARN-6727: - | (/) *{color:green}+1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 17s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 4 new or modified test files. {color} | || || || || {color:brown} trunk Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 45s{color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 13m 59s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 8m 43s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 56s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 55s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m 25s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 34s{color} | {color:green} trunk passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 9s{color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 27s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 5m 22s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} cc {color} | {color:green} 5m 22s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 5m 22s{color} | {color:green} the patch passed {color} | | {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange} 0m 59s{color} | {color:orange} hadoop-yarn-project/hadoop-yarn: The patch generated 4 new + 75 unchanged - 1 fixed = 79 total (was 76) {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 52s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m 53s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 32s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 32s{color} | {color:green} hadoop-yarn-api in the patch passed. {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m 27s{color} | {color:green} hadoop-yarn-common in the patch passed. {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 42m 47s{color} | {color:green} hadoop-yarn-server-resourcemanager in the patch passed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 28s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black}101m 20s{color} | {color:black} {color} | \\ \\ || Subsystem || Report/Notes || | Docker | Image:yetus/hadoop:14b5c93 | | JIRA Issue | YARN-6727 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12880159/YARN-6727.001.patch | | Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc | | uname | Linux 91fb4507bfd2 3.13.0-119-generic #166-Ubuntu SMP Wed May 3 12:18:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh | | git revision | trunk / 79df1e7 | | Default Java | 1.8.0_131 | | findbugs | v3.1.0-RC1 | | checkstyle | https://builds.apache.org/job/PreCommit-YARN-Build/16680/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt | | Test Results | https://builds.apache.org/job/PreCommit-YARN-Build/16680/testReport/ | |
[jira] [Commented] (YARN-6727) Improve getQueueUserAcls API to query for specific queue and user
[ https://issues.apache.org/jira/browse/YARN-6727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16112277#comment-16112277 ] Hadoop QA commented on YARN-6727: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 25s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 4 new or modified test files. {color} | || || || || {color:brown} trunk Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 10s{color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 13m 48s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 9m 20s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 57s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 56s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m 28s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 35s{color} | {color:green} trunk passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 11s{color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 29s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 5m 50s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} cc {color} | {color:green} 5m 50s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 5m 50s{color} | {color:green} the patch passed {color} | | {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange} 0m 55s{color} | {color:orange} hadoop-yarn-project/hadoop-yarn: The patch generated 4 new + 76 unchanged - 1 fixed = 80 total (was 77) {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 53s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m 49s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 34s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 32s{color} | {color:green} hadoop-yarn-api in the patch passed. {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m 29s{color} | {color:green} hadoop-yarn-common in the patch passed. {color} | | {color:red}-1{color} | {color:red} unit {color} | {color:red} 46m 31s{color} | {color:red} hadoop-yarn-server-resourcemanager in the patch failed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 27s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black}105m 44s{color} | {color:black} {color} | \\ \\ || Reason || Tests || | Failed junit tests | hadoop.yarn.server.resourcemanager.security.TestDelegationTokenRenewer | | Timed out junit tests | org.apache.hadoop.yarn.server.resourcemanager.TestSubmitApplicationWithRMHA | \\ \\ || Subsystem || Report/Notes || | Docker | Image:yetus/hadoop:14b5c93 | | JIRA Issue | YARN-6727 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12880159/YARN-6727.001.patch | | Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc | | uname | Linux cd195e0fbd4b 3.13.0-123-generic #172-Ubuntu SMP Mon Jun 26 18:04:35 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh | | git revision | trunk / 79df1e7 | | Default Java | 1.8.0_131 | | findbugs | v3.1.0-RC1 | | checkstyle |
[jira] [Commented] (YARN-6727) Improve getQueueUserAcls API to query for specific queue and user
[ https://issues.apache.org/jira/browse/YARN-6727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16076804#comment-16076804 ] Sunil G commented on YARN-6727: --- bq.+ queue level readlock {code:title=AbstractCSQueue#hasAccess} public boolean hasAccess(QueueACL acl, UserGroupInformation user) { return authorizer.checkPermission( new AccessRequest(queueEntity, user, SchedulerUtils.toAccessType(acl), null, null, Server.getRemoteAddress(), null)); } {code} We are currently invoking authorizer.checkPermission directly. So do we need queue readLock here? bq.Submission time QUEUE_SUBMIT right we could cache but we need all Sorry. I could not get you. could you please help to elaborate. bq.IIUC the refresh interval is about 5/10 min.We dont have direct update or notifier as of now. cache invalidate is needed in cases where user's acls are changed in system. Hence it makes sense. > Improve getQueueUserAcls API to query for specific queue and user > -- > > Key: YARN-6727 > URL: https://issues.apache.org/jira/browse/YARN-6727 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Bibin A Chundatt >Assignee: Bibin A Chundatt > Attachments: YARN-6727.WIP.patch > > > Currently {{ApplicationClientProtocol#getQueueUserAcls}} return data for all > the queues available in scheduler for user. > User wants to know whether he has rights of a particular queue only. For > systems with 5K queues returning all queues list is not efficient. > Suggested change: support additional parameters *userName and queueName* as > optional. Admin user should be able to query other users ACL for a particular > queueName. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6727) Improve getQueueUserAcls API to query for specific queue and user
[ https://issues.apache.org/jira/browse/YARN-6727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16074561#comment-16074561 ] Bibin A Chundatt commented on YARN-6727: Thank you [~sunilg] for explanation {quote} Is safer for CS as we lock with a readLock only in YarnAuthorizationProvider {quote} + queue level readlock {quote} Whichever user is passed with checkAccess at one point of time (from cli/api side or app submission time etc) could be cached. {quote} Submission time QUEUE_SUBMIT right we could cache but we need all .. am i missing something? {quote} Cache could be invalidated in cases here a config refresh happened for queues/acls or in similar conditions. {quote} The ACL mapping will depend on user to group mapping also which gets refreshed based in time interval. IIUC the refresh interval is about 5/10 min. I dont this we have direct update or notifier as of now. > Improve getQueueUserAcls API to query for specific queue and user > -- > > Key: YARN-6727 > URL: https://issues.apache.org/jira/browse/YARN-6727 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Bibin A Chundatt >Assignee: Bibin A Chundatt > Attachments: YARN-6727.WIP.patch > > > Currently {{ApplicationClientProtocol#getQueueUserAcls}} return data for all > the queues available in scheduler for user. > User wants to know whether he has rights of a particular queue only. For > systems with 5K queues returning all queues list is not efficient. > Suggested change: support additional parameters *userName and queueName* as > optional. Admin user should be able to query other users ACL for a particular > queueName. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6727) Improve getQueueUserAcls API to query for specific queue and user
[ https://issues.apache.org/jira/browse/YARN-6727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16074516#comment-16074516 ] Sunil G commented on YARN-6727: --- I guess I ll put my thoughts more clearer for better clarity in terms of understanding. # Invoking {{YarnScheduler#checkAccess}} is safer for CS as we lock with a readLock only in YarnAuthorizationProvider where as FS still has some locks in scheduler. So we can try limiting the checkAccess as possible. # Whichever user is passed with checkAccess at one point of time (from cli/api side or app submission time etc) could be cached. I prefer this to be stored outside scheduler. # So any further {{getQueueUserAcls}} could look into cache for first time. And given a cache miss, we can do {{YarnScheduler#checkAccess}} and update cache. # Cache could be invalidated in cases here a config refresh happened for queues/acls or in similar conditions. Given if changes are minimal, we can do in one ticket. But if its more, I am in favor of splitting to 2 jiras for api and scheduler. > Improve getQueueUserAcls API to query for specific queue and user > -- > > Key: YARN-6727 > URL: https://issues.apache.org/jira/browse/YARN-6727 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Bibin A Chundatt >Assignee: Bibin A Chundatt > Attachments: YARN-6727.WIP.patch > > > Currently {{ApplicationClientProtocol#getQueueUserAcls}} return data for all > the queues available in scheduler for user. > User wants to know whether he has rights of a particular queue only. For > systems with 5K queues returning all queues list is not efficient. > Suggested change: support additional parameters *userName and queueName* as > optional. Admin user should be able to query other users ACL for a particular > queueName. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6727) Improve getQueueUserAcls API to query for specific queue and user
[ https://issues.apache.org/jira/browse/YARN-6727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16074234#comment-16074234 ] Bibin A Chundatt commented on YARN-6727: Thank you [~sunilg] and [~naganarasimha...@apache.org] for looking into the same. I would like split into two # precomputed access checks for active/non-active users # Api Interface change lets focus of API interface change in this JIRA and will raise another for caching precomputed ACLS > Improve getQueueUserAcls API to query for specific queue and user > -- > > Key: YARN-6727 > URL: https://issues.apache.org/jira/browse/YARN-6727 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Bibin A Chundatt >Assignee: Bibin A Chundatt > Attachments: YARN-6727.WIP.patch > > > Currently {{ApplicationClientProtocol#getQueueUserAcls}} return data for all > the queues available in scheduler for user. > User wants to know whether he has rights of a particular queue only. For > systems with 5K queues returning all queues list is not efficient. > Suggested change: support additional parameters *userName and queueName* as > optional. Admin user should be able to query other users ACL for a particular > queueName. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6727) Improve getQueueUserAcls API to query for specific queue and user
[ https://issues.apache.org/jira/browse/YARN-6727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16064261#comment-16064261 ] Sunil G commented on YARN-6727: --- I second [~naganarasimha...@apache.org]'s opinion. We need not have to add more overhead such as invoking {{YarnAuthorizationProvider#checkPermission}} with a readLock (We already doing some what similar in existing public apis such as getQueueUserAclInfo), its better if we can avoid similar api's in future. Also a client api/cli getting a lock (read) to scheduler's internal structure could add more complexities later. Hence we can start thinking in having precomputed access checks for active/non-active users. New user's might need to do with some common checks from QueueACLManager itself. > Improve getQueueUserAcls API to query for specific queue and user > -- > > Key: YARN-6727 > URL: https://issues.apache.org/jira/browse/YARN-6727 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Bibin A Chundatt >Assignee: Bibin A Chundatt > Attachments: YARN-6727.WIP.patch > > > Currently {{ApplicationClientProtocol#getQueueUserAcls}} return data for all > the queues available in scheduler for user. > User wants to know whether he has rights of a particular queue only. For > systems with 5K queues returning all queues list is not efficient. > Suggested change: support additional parameters *userName and queueName* as > optional. Admin user should be able to query other users ACL for a particular > queueName. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6727) Improve getQueueUserAcls API to query for specific queue and user
[ https://issues.apache.org/jira/browse/YARN-6727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16062071#comment-16062071 ] Naganarasimha G R commented on YARN-6727: - Thanks [~bibinchundatt] for the WIP patch. As i understand, there are 2 issues currently for large queues # Amount of data which gets transferred # number of computations done in the server side as for each queue we are trying to match whether user or his group has rights. And also it would suffice to know in most case for the leaf queue only. So IIUC your patch is solving only the first part. But for the second one though its not too much of overhead as no locks are held but its unnecessary computation. So would it better to optimize it with additional scheduler API too, thoughts ? Further i was thinking it would be good to have list of queues rather than a single queue. > Improve getQueueUserAcls API to query for specific queue and user > -- > > Key: YARN-6727 > URL: https://issues.apache.org/jira/browse/YARN-6727 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Bibin A Chundatt >Assignee: Bibin A Chundatt > Attachments: YARN-6727.WIP.patch > > > Currently {{ApplicationClientProtocol#getQueueUserAcls}} return data for all > the queues available in scheduler for user. > User wants to know whether he has rights of a particular queue only. For > systems with 5K queues returning all queues list is not efficient. > Suggested change: support additional parameters *userName and queueName* as > optional. Admin user should be able to query other users ACL for a particular > queueName. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6727) Improve getQueueUserAcls API to query for specific queue and user
[ https://issues.apache.org/jira/browse/YARN-6727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16061789#comment-16061789 ] Hadoop QA commented on YARN-6727: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 16s{color} | {color:blue} Docker mode activated. {color} | | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s{color} | {color:red} The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. {color} | | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 41s{color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 14m 11s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 9m 15s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 51s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 56s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m 26s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 34s{color} | {color:green} trunk passed {color} | | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 10s{color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 30s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 5m 31s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} cc {color} | {color:green} 5m 31s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 5m 31s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 50s{color} | {color:green} hadoop-yarn-project/hadoop-yarn: The patch generated 0 new + 39 unchanged - 1 fixed = 39 total (was 40) {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 47s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m 45s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 24s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 32s{color} | {color:green} hadoop-yarn-api in the patch passed. {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m 26s{color} | {color:green} hadoop-yarn-common in the patch passed. {color} | | {color:red}-1{color} | {color:red} unit {color} | {color:red} 42m 39s{color} | {color:red} hadoop-yarn-server-resourcemanager in the patch failed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 32s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black}101m 10s{color} | {color:black} {color} | \\ \\ || Reason || Tests || | Failed junit tests | hadoop.yarn.server.resourcemanager.TestRMRestart | \\ \\ || Subsystem || Report/Notes || | Docker | Image:yetus/hadoop:14b5c93 | | JIRA Issue | YARN-6727 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12874295/YARN-6727.WIP.patch | | Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc | | uname | Linux 86f89542d43f 3.13.0-107-generic #154-Ubuntu SMP Tue Dec 20 09:57:27 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh | | git revision | trunk / 0111711 | | Default Java | 1.8.0_131 | | findbugs | v3.1.0-RC1 | | unit | https://builds.apache.org/job/PreCommit-YARN-Build/16242/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt | | Test Results |
[jira] [Commented] (YARN-6727) Improve getQueueUserAcls API to query for specific queue and user
[ https://issues.apache.org/jira/browse/YARN-6727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16058945#comment-16058945 ] Naganarasimha G R commented on YARN-6727: - Thanks [~bibinchundatt] for raising this issue. Agree that it can be simplified in the server side it gets simplified. > Improve getQueueUserAcls API to query for specific queue and user > -- > > Key: YARN-6727 > URL: https://issues.apache.org/jira/browse/YARN-6727 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Bibin A Chundatt >Assignee: Bibin A Chundatt > > Currently {{ApplicationClientProtocol#getQueueUserAcls}} return data for all > the queues available in scheduler for user. > User wants to know whether he has rights of a particular queue only. For > systems with 5K queues returning all queues list is not efficient. > Suggested change: support additional parameters *userName and queueName* as > optional. Admin user should be able to query other users ACL for a particular > queueName. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org