[jira] [Commented] (YARN-10726) Log the size of DelegationTokenRenewer event queue in case of too many pending events

2021-04-01 Thread Qi Zhu (Jira) 


[ 
https://issues.apache.org/jira/browse/YARN-10726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17313243#comment-17313243
 ] 

Qi Zhu commented on YARN-10726:
---

[~pbacsko]

I tested locally just now, it passed.

Thanks.

> Log the size of DelegationTokenRenewer event queue in case of too many 
> pending events
> -
>
> Key: YARN-10726
> URL: https://issues.apache.org/jira/browse/YARN-10726
> Project: Hadoop YARN
>  Issue Type: Improvement
>Reporter: Qi Zhu
>Assignee: Qi Zhu
>Priority: Major
> Attachments: YARN-10726.001.patch, YARN-10726.002.patch
>
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Commented] (YARN-10726) Log the size of DelegationTokenRenewer event queue in case of too many pending events

2021-04-01 Thread Peter Bacsko (Jira) 


[ 
https://issues.apache.org/jira/browse/YARN-10726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17313241#comment-17313241
 ] 

Peter Bacsko commented on YARN-10726:
-

Ok, I strongly believe that the failing tests are flaky.

[~zhuqi] could you verify it by running them locally a couple of times?

> Log the size of DelegationTokenRenewer event queue in case of too many 
> pending events
> -
>
> Key: YARN-10726
> URL: https://issues.apache.org/jira/browse/YARN-10726
> Project: Hadoop YARN
>  Issue Type: Improvement
>Reporter: Qi Zhu
>Assignee: Qi Zhu
>Priority: Major
> Attachments: YARN-10726.001.patch, YARN-10726.002.patch
>
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Commented] (YARN-10726) Log the size of DelegationTokenRenewer event queue in case of too many pending events

2021-04-01 Thread Hadoop QA (Jira) 


[ 
https://issues.apache.org/jira/browse/YARN-10726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17313240#comment-17313240
 ] 

Hadoop QA commented on YARN-10726:
--

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime ||  Logfile || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  1m 
18s{color} | {color:blue}{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} || ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green}  0m  
0s{color} | {color:green}{color} | {color:green} No case conflicting files 
found. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green}{color} | {color:green} The patch does not contain any 
@author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red}  0m  
0s{color} | {color:red}{color} | {color:red} The patch doesn't appear to 
include any new or modified tests. Please justify why no new tests are needed 
for this patch. Also please list what manual steps were performed to verify 
this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} || ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 22m 
 9s{color} | {color:green}{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m  
1s{color} | {color:green}{color} | {color:green} trunk passed with JDK 
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
51s{color} | {color:green}{color} | {color:green} trunk passed with JDK Private 
Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
44s{color} | {color:green}{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green}  0m 
53s{color} | {color:green}{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 
16m 40s{color} | {color:green}{color} | {color:green} branch has no errors when 
building and testing our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
40s{color} | {color:green}{color} | {color:green} trunk passed with JDK 
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
37s{color} | {color:green}{color} | {color:green} trunk passed with JDK Private 
Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 19m 
46s{color} | {color:blue}{color} | {color:blue} Both FindBugs and SpotBugs are 
enabled, using SpotBugs. {color} |
| {color:green}+1{color} | {color:green} spotbugs {color} | {color:green}  1m 
49s{color} | {color:green}{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} || ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  0m 
50s{color} | {color:green}{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
54s{color} | {color:green}{color} | {color:green} the patch passed with JDK 
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
54s{color} | {color:green}{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
44s{color} | {color:green}{color} | {color:green} the patch passed with JDK 
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
44s{color} | {color:green}{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
39s{color} | {color:green}{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green}  0m 
48s{color} | {color:green}{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green}{color} | {color:green} The patch has no whitespace 
issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 
15m  4s{color} | {color:green}{color} | {color:green} patch has no errors when 
building and testing our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
38s{color} | {color:green}{color} | {color:green} the patch passed with JDK 
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 {color} |
|

[jira] [Commented] (YARN-10726) Log the size of DelegationTokenRenewer event queue in case of too many pending events

2021-04-01 Thread Qi Zhu (Jira) 


[ 
https://issues.apache.org/jira/browse/YARN-10726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17313195#comment-17313195
 ] 

Qi Zhu commented on YARN-10726:
---

Thanks [~pbacsko] for commit.:D

> Log the size of DelegationTokenRenewer event queue in case of too many 
> pending events
> -
>
> Key: YARN-10726
> URL: https://issues.apache.org/jira/browse/YARN-10726
> Project: Hadoop YARN
>  Issue Type: Improvement
>Reporter: Qi Zhu
>Assignee: Qi Zhu
>Priority: Major
> Attachments: YARN-10726.001.patch, YARN-10726.002.patch
>
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Commented] (YARN-10726) Log the size of DelegationTokenRenewer event queue in case of too many pending events

2021-04-01 Thread Peter Bacsko (Jira) 


[ 
https://issues.apache.org/jira/browse/YARN-10726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17313192#comment-17313192
 ] 

Peter Bacsko commented on YARN-10726:
-

Ah, I already committed the change. Let's hope Jenkins comes back green :)

+1

> Log the size of DelegationTokenRenewer event queue in case of too many 
> pending events
> -
>
> Key: YARN-10726
> URL: https://issues.apache.org/jira/browse/YARN-10726
> Project: Hadoop YARN
>  Issue Type: Improvement
>Reporter: Qi Zhu
>Assignee: Qi Zhu
>Priority: Major
> Attachments: YARN-10726.001.patch, YARN-10726.002.patch
>
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Commented] (YARN-10726) Log the size of DelegationTokenRenewer event queue in case of too many pending events

2021-04-01 Thread Peter Bacsko (Jira) 


[ 
https://issues.apache.org/jira/browse/YARN-10726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17313189#comment-17313189
 ] 

Peter Bacsko commented on YARN-10726:
-

"hadoop.yarn.server.resourcemanager.security.TestDelegationTokenRenewer" - this 
is unrelated I believe. This test case has been failing for a long time.

> Log the size of DelegationTokenRenewer event queue in case of too many 
> pending events
> -
>
> Key: YARN-10726
> URL: https://issues.apache.org/jira/browse/YARN-10726
> Project: Hadoop YARN
>  Issue Type: Improvement
>Reporter: Qi Zhu
>Assignee: Qi Zhu
>Priority: Major
> Attachments: YARN-10726.001.patch, YARN-10726.002.patch
>
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Commented] (YARN-10726) Log the size of DelegationTokenRenewer event queue in case of too many pending events

2021-04-01 Thread Qi Zhu (Jira) 


[ 
https://issues.apache.org/jira/browse/YARN-10726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17313139#comment-17313139
 ] 

Qi Zhu commented on YARN-10726:
---

Thanks [~pbacsko] for your review and suggestion.

You concern is very right, i agree with you.

I have updated it in latest patch, just as we done in async dispatcher.:D

Thanks.

> Log the size of DelegationTokenRenewer event queue in case of too many 
> pending events
> -
>
> Key: YARN-10726
> URL: https://issues.apache.org/jira/browse/YARN-10726
> Project: Hadoop YARN
>  Issue Type: Improvement
>Reporter: Qi Zhu
>Assignee: Qi Zhu
>Priority: Major
> Attachments: YARN-10726.001.patch, YARN-10726.002.patch
>
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Commented] (YARN-10726) Log the size of DelegationTokenRenewer event queue in case of too many pending events

2021-04-01 Thread Peter Bacsko (Jira) 


[ 
https://issues.apache.org/jira/browse/YARN-10726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17313138#comment-17313138
 ] 

Peter Bacsko commented on YARN-10726:
-

This is from {{AsyncDispatcher}}:

{noformat}
 if (qSize != 0 && qSize % 1000 == 0
  && lastEventQueueSizeLogged != qSize) {
lastEventQueueSizeLogged = qSize;
LOG.info("Size of event-queue is " + qSize);
  }
{noformat}

Update the code with {{lastEventQueueSizeLogged}}.

> Log the size of DelegationTokenRenewer event queue in case of too many 
> pending events
> -
>
> Key: YARN-10726
> URL: https://issues.apache.org/jira/browse/YARN-10726
> Project: Hadoop YARN
>  Issue Type: Improvement
>Reporter: Qi Zhu
>Assignee: Qi Zhu
>Priority: Major
> Attachments: YARN-10726.001.patch
>
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Commented] (YARN-10726) Log the size of DelegationTokenRenewer event queue in case of too many pending events

2021-04-01 Thread Peter Bacsko (Jira) 


[ 
https://issues.apache.org/jira/browse/YARN-10726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17313123#comment-17313123
 ] 

Peter Bacsko commented on YARN-10726:
-

Thanks [~zhuqi]. I think it's a good idea. My only "concern" is that we have 
too many events, this code can possibly run too frequently. For example, if you 
go 998, 998, 999, 1000, 1001, 1002, then it prints at 1000, then it starts to 
consume events, size goes back from 1000 to 990, then it prints the size again.

I think we should limit how often we print this message. We should log it too 
often, I'm not sure how we do this in other parts of the code. I'll check what 
can be the best solution.

> Log the size of DelegationTokenRenewer event queue in case of too many 
> pending events
> -
>
> Key: YARN-10726
> URL: https://issues.apache.org/jira/browse/YARN-10726
> Project: Hadoop YARN
>  Issue Type: Improvement
>Reporter: Qi Zhu
>Assignee: Qi Zhu
>Priority: Major
> Attachments: YARN-10726.001.patch
>
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org