[jira] [Commented] (MAPREDUCE-7307) Potential thread leak in LocatedFileStatusFetcher

2020-11-18 Thread Zhihua Deng (Jira) 


[ 
https://issues.apache.org/jira/browse/MAPREDUCE-7307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17235088#comment-17235088
 ] 

Zhihua Deng commented on MAPREDUCE-7307:


[~ste...@apache.org], [~jlowe]  could you please take a look? thanks in 
advance!**

> Potential thread leak in LocatedFileStatusFetcher
> -
>
> Key: MAPREDUCE-7307
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-7307
> Project: Hadoop Map/Reduce
>  Issue Type: Bug
>  Components: job submission
>Reporter: Zhihua Deng
>Assignee: Zhihua Deng
>Priority: Major
>
> We see that when using LocatedFileStatusFetcher to get file infos In 
> parallel, if the listStatus thread is interrupted,  the  executor service in 
> LocatedFileStatusFetcher is left unclosed,  the thread stack will like this:
> {noformat}
> "GetFileInfo #63" #125 daemon prio=5 os_prio=0 tid=0x7f6198106800 
> nid=0x881 waiting on condition [0x7f60d9fde000]
> java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x82e810a8> (a 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
> at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
> at 
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748){noformat}
> This caused by if condition.await() throws InterruptedException,  the method 
> `shutDownNow` for the executor service would not be called as a result, 
> should move such resource releasing call into the finally block.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org

[jira] [Commented] (MAPREDUCE-7304) Enhance the map-reduce Job end notifier to be able to notify the given URL via a custom class

2020-11-18 Thread Hadoop QA (Jira) 


[ 
https://issues.apache.org/jira/browse/MAPREDUCE-7304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17234817#comment-17234817
 ] 

Hadoop QA commented on MAPREDUCE-7304:
--

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime ||  Logfile || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 10m 
34s{color} |  | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} || ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green}  0m  
0s{color} |  | {color:green} No case conflicting files found. {color} |
| {color:blue}0{color} | {color:blue} codespell {color} | {color:blue}  0m  
0s{color} |  | {color:blue} codespell was not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} |  | {color:green} The patch does not contain any @author tags. 
{color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m 
 0s{color} |  | {color:green} The patch appears to include 1 new or modified 
test files. {color} |
|| || || || {color:brown} branch-3.1 Compile Tests {color} || ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  3m 
49s{color} |  | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 23m 
27s{color} |  | {color:green} branch-3.1 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m 
58s{color} |  | {color:green} branch-3.1 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
42s{color} |  | {color:green} branch-3.1 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green}  1m 
19s{color} |  | {color:green} branch-3.1 passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 
14m  5s{color} |  | {color:green} branch has no errors when building and 
testing our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
59s{color} |  | {color:green} branch-3.1 passed {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue}  1m 
11s{color} |  | {color:blue} Used deprecated FindBugs config; considering 
switching to SpotBugs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  2m  
8s{color} |  | {color:green} branch-3.1 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} || ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
23s{color} |  | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  1m 
 6s{color} |  | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m 
50s{color} |  | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  1m 
50s{color} |  | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} blanks {color} | {color:green}  0m  
0s{color} |  | {color:green} The patch has no blanks issues. {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}  
0m 35s{color} | 
[/results-checkstyle-hadoop-mapreduce-project_hadoop-mapreduce-client.txt|https://ci-hadoop.apache.org/job/PreCommit-MAPREDUCE-Build/34/artifact/out/results-checkstyle-hadoop-mapreduce-project_hadoop-mapreduce-client.txt]
 | {color:orange} hadoop-mapreduce-project/hadoop-mapreduce-client: The patch 
generated 3 new + 672 unchanged - 0 fixed = 675 total (was 672) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green}  1m  
7s{color} |  | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green}  0m  
2s{color} |  | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 
13m  2s{color} |  | {color:green} patch has no errors when building and testing 
our client artifacts. {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red}  0m 
30s{color} | 
[/results-javadoc-javadoc-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core.txt|https://ci-hadoop.apache.org/job/PreCommit-MAPREDUCE-Build/34/artifact/out/results-javadoc-javadoc-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core.txt]
 | {color:red} 
hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core 
generated 1 new + 2540 unchanged - 0 fixed = 2541 total (was 2540) {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  2m 
16s{color} |  | {color:green} the patch passed {color} |
|| || || ||

[jira] [Updated] (MAPREDUCE-7304) Enhance the map-reduce Job end notifier to be able to notify the given URL via a custom class

2020-11-18 Thread Jira 


 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-7304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zoltán Erdmann updated MAPREDUCE-7304:
--
Attachment: MAPREDUCE-7304-branch-3.1-001.patch

> Enhance the map-reduce Job end notifier to be able to notify the given URL 
> via a custom class
> -
>
> Key: MAPREDUCE-7304
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-7304
> Project: Hadoop Map/Reduce
>  Issue Type: Improvement
>  Components: mrv2
>Reporter: Daniel Fritsi
>Assignee: Zoltán Erdmann
>Priority: Major
> Attachments: MAPREDUCE-7304-001.patch, MAPREDUCE-7304-002.patch, 
> MAPREDUCE-7304-003.patch, MAPREDUCE-7304-004.patch, 
> MAPREDUCE-7304-branch-3.1-001.patch, MAPREDUCE-7304-branch-3.2-001.patch, 
> MAPREDUCE-7304-branch-3.3-001.patch
>
>
> Currently 
> {color:#0747a6}{{*org.apache.hadoop.mapreduce.v2.app.JobEndNotifier*}}{color} 
> allows a very limited configuration on how the given Job end notification URL 
> should be notified. We should enhance this, but instead of adding more 
> *{color:#0747A6}{{mapreduce.job.end-notification.*}}{color}* properties to be 
> able to configure the underlying HttpURLConnection, we should add a new 
> property so users can use their own notifier class.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org

[jira] [Commented] (MAPREDUCE-7304) Enhance the map-reduce Job end notifier to be able to notify the given URL via a custom class

2020-11-18 Thread Hadoop QA (Jira) 


[ 
https://issues.apache.org/jira/browse/MAPREDUCE-7304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17234701#comment-17234701
 ] 

Hadoop QA commented on MAPREDUCE-7304:
--

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime ||  Logfile || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m  
0s{color} |  | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} docker {color} | {color:red}  0m 
37s{color} |  | {color:red} Docker failed to build yetus/hadoop:74c0764343b. 
{color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | MAPREDUCE-7304 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/13015558/MAPREDUCE-7304-branch-3.3-001.patch
 |
| Console output | 
https://ci-hadoop.apache.org/job/PreCommit-MAPREDUCE-Build/33/console |
| versions | git=2.17.1 |
| Powered by | Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org |


This message was automatically generated.



> Enhance the map-reduce Job end notifier to be able to notify the given URL 
> via a custom class
> -
>
> Key: MAPREDUCE-7304
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-7304
> Project: Hadoop Map/Reduce
>  Issue Type: Improvement
>  Components: mrv2
>Reporter: Daniel Fritsi
>Assignee: Zoltán Erdmann
>Priority: Major
> Attachments: MAPREDUCE-7304-001.patch, MAPREDUCE-7304-002.patch, 
> MAPREDUCE-7304-003.patch, MAPREDUCE-7304-004.patch, 
> MAPREDUCE-7304-branch-3.2-001.patch, MAPREDUCE-7304-branch-3.3-001.patch
>
>
> Currently 
> {color:#0747a6}{{*org.apache.hadoop.mapreduce.v2.app.JobEndNotifier*}}{color} 
> allows a very limited configuration on how the given Job end notification URL 
> should be notified. We should enhance this, but instead of adding more 
> *{color:#0747A6}{{mapreduce.job.end-notification.*}}{color}* properties to be 
> able to configure the underlying HttpURLConnection, we should add a new 
> property so users can use their own notifier class.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org

[jira] [Updated] (MAPREDUCE-7304) Enhance the map-reduce Job end notifier to be able to notify the given URL via a custom class

2020-11-18 Thread Jira 


 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-7304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zoltán Erdmann updated MAPREDUCE-7304:
--
Attachment: MAPREDUCE-7304-branch-3.3-001.patch
MAPREDUCE-7304-branch-3.2-001.patch

> Enhance the map-reduce Job end notifier to be able to notify the given URL 
> via a custom class
> -
>
> Key: MAPREDUCE-7304
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-7304
> Project: Hadoop Map/Reduce
>  Issue Type: Improvement
>  Components: mrv2
>Reporter: Daniel Fritsi
>Assignee: Zoltán Erdmann
>Priority: Major
> Attachments: MAPREDUCE-7304-001.patch, MAPREDUCE-7304-002.patch, 
> MAPREDUCE-7304-003.patch, MAPREDUCE-7304-004.patch, 
> MAPREDUCE-7304-branch-3.2-001.patch, MAPREDUCE-7304-branch-3.3-001.patch
>
>
> Currently 
> {color:#0747a6}{{*org.apache.hadoop.mapreduce.v2.app.JobEndNotifier*}}{color} 
> allows a very limited configuration on how the given Job end notification URL 
> should be notified. We should enhance this, but instead of adding more 
> *{color:#0747A6}{{mapreduce.job.end-notification.*}}{color}* properties to be 
> able to configure the underlying HttpURLConnection, we should add a new 
> property so users can use their own notifier class.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org

[jira] [Commented] (MAPREDUCE-7304) Enhance the map-reduce Job end notifier to be able to notify the given URL via a custom class

2020-11-18 Thread Peter Bacsko (Jira) 


[ 
https://issues.apache.org/jira/browse/MAPREDUCE-7304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17234669#comment-17234669
 ] 

Peter Bacsko commented on MAPREDUCE-7304:
-

Thanks [~zerdmann] +1. Remaining checkstyle can be ignored, just javadoc.

As we discussed in the chat, please create a patch for branch-3.3 also because 
that's relatively new.

> Enhance the map-reduce Job end notifier to be able to notify the given URL 
> via a custom class
> -
>
> Key: MAPREDUCE-7304
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-7304
> Project: Hadoop Map/Reduce
>  Issue Type: Improvement
>  Components: mrv2
>Reporter: Daniel Fritsi
>Assignee: Zoltán Erdmann
>Priority: Major
> Attachments: MAPREDUCE-7304-001.patch, MAPREDUCE-7304-002.patch, 
> MAPREDUCE-7304-003.patch, MAPREDUCE-7304-004.patch
>
>
> Currently 
> {color:#0747a6}{{*org.apache.hadoop.mapreduce.v2.app.JobEndNotifier*}}{color} 
> allows a very limited configuration on how the given Job end notification URL 
> should be notified. We should enhance this, but instead of adding more 
> *{color:#0747A6}{{mapreduce.job.end-notification.*}}{color}* properties to be 
> able to configure the underlying HttpURLConnection, we should add a new 
> property so users can use their own notifier class.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org

[jira] [Commented] (MAPREDUCE-7304) Enhance the map-reduce Job end notifier to be able to notify the given URL via a custom class

2020-11-18 Thread Hadoop QA (Jira) 


[ 
https://issues.apache.org/jira/browse/MAPREDUCE-7304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17234641#comment-17234641
 ] 

Hadoop QA commented on MAPREDUCE-7304:
--

| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime ||  Logfile || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
36s{color} |  | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} || ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green}  0m  
0s{color} |  | {color:green} No case conflicting files found. {color} |
| {color:blue}0{color} | {color:blue} codespell {color} | {color:blue}  0m  
1s{color} |  | {color:blue} codespell was not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} |  | {color:green} The patch does not contain any @author tags. 
{color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m 
 0s{color} |  | {color:green} The patch appears to include 1 new or modified 
test files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} || ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  1m 
46s{color} |  | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 20m 
 6s{color} |  | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  2m 
20s{color} |  | {color:green} trunk passed with JDK 
Ubuntu-11.0.9+11-Ubuntu-0ubuntu1.18.04.1 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m 
59s{color} |  | {color:green} trunk passed with JDK Private 
Build-1.8.0_272-8u272-b10-0ubuntu1~18.04-b10 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
45s{color} |  | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green}  1m 
17s{color} |  | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 
16m 57s{color} |  | {color:green} branch has no errors when building and 
testing our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
55s{color} |  | {color:green} trunk passed with JDK 
Ubuntu-11.0.9+11-Ubuntu-0ubuntu1.18.04.1 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
53s{color} |  | {color:green} trunk passed with JDK Private 
Build-1.8.0_272-8u272-b10-0ubuntu1~18.04-b10 {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue}  0m 
58s{color} |  | {color:blue} Used deprecated FindBugs config; considering 
switching to SpotBugs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  2m  
7s{color} |  | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} || ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
28s{color} |  | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  1m 
 4s{color} |  | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  2m 
11s{color} |  | {color:green} the patch passed with JDK 
Ubuntu-11.0.9+11-Ubuntu-0ubuntu1.18.04.1 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  2m 
11s{color} |  | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m 
51s{color} |  | {color:green} the patch passed with JDK Private 
Build-1.8.0_272-8u272-b10-0ubuntu1~18.04-b10 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  1m 
51s{color} |  | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} blanks {color} | {color:green}  0m  
0s{color} |  | {color:green} The patch has no blanks issues. {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}  
0m 39s{color} | 
[/results-checkstyle-hadoop-mapreduce-project_hadoop-mapreduce-client.txt|https://ci-hadoop.apache.org/job/PreCommit-MAPREDUCE-Build/32/artifact/out/results-checkstyle-hadoop-mapreduce-project_hadoop-mapreduce-client.txt]
 | {color:orange} hadoop-mapreduce-project/hadoop-mapreduce-client: The patch 
generated 3 new + 675 unchanged - 0 fixed = 678 total (was 675) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green}  1m  
2s{color} |  | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green}  0m  
2s{color} |  | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedclient

[jira] [Updated] (MAPREDUCE-7304) Enhance the map-reduce Job end notifier to be able to notify the given URL via a custom class

2020-11-18 Thread Jira 


 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-7304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zoltán Erdmann updated MAPREDUCE-7304:
--
Attachment: MAPREDUCE-7304-004.patch
Status: Patch Available  (was: Open)

Java doc lines are longer than 80 characters because of the long variable names.

> Enhance the map-reduce Job end notifier to be able to notify the given URL 
> via a custom class
> -
>
> Key: MAPREDUCE-7304
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-7304
> Project: Hadoop Map/Reduce
>  Issue Type: Improvement
>  Components: mrv2
>Reporter: Daniel Fritsi
>Assignee: Zoltán Erdmann
>Priority: Major
> Attachments: MAPREDUCE-7304-001.patch, MAPREDUCE-7304-002.patch, 
> MAPREDUCE-7304-003.patch, MAPREDUCE-7304-004.patch
>
>
> Currently 
> {color:#0747a6}{{*org.apache.hadoop.mapreduce.v2.app.JobEndNotifier*}}{color} 
> allows a very limited configuration on how the given Job end notification URL 
> should be notified. We should enhance this, but instead of adding more 
> *{color:#0747A6}{{mapreduce.job.end-notification.*}}{color}* properties to be 
> able to configure the underlying HttpURLConnection, we should add a new 
> property so users can use their own notifier class.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org

[jira] [Updated] (MAPREDUCE-7304) Enhance the map-reduce Job end notifier to be able to notify the given URL via a custom class

2020-11-18 Thread Jira 


 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-7304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zoltán Erdmann updated MAPREDUCE-7304:
--
Status: Open  (was: Patch Available)

> Enhance the map-reduce Job end notifier to be able to notify the given URL 
> via a custom class
> -
>
> Key: MAPREDUCE-7304
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-7304
> Project: Hadoop Map/Reduce
>  Issue Type: Improvement
>  Components: mrv2
>Reporter: Daniel Fritsi
>Assignee: Zoltán Erdmann
>Priority: Major
> Attachments: MAPREDUCE-7304-001.patch, MAPREDUCE-7304-002.patch, 
> MAPREDUCE-7304-003.patch
>
>
> Currently 
> {color:#0747a6}{{*org.apache.hadoop.mapreduce.v2.app.JobEndNotifier*}}{color} 
> allows a very limited configuration on how the given Job end notification URL 
> should be notified. We should enhance this, but instead of adding more 
> *{color:#0747A6}{{mapreduce.job.end-notification.*}}{color}* properties to be 
> able to configure the underlying HttpURLConnection, we should add a new 
> property so users can use their own notifier class.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org

[jira] [Created] (MAPREDUCE-7308) Containers never get reused as containersToReuse map gets cleared on makeRemoteRequest

2020-11-18 Thread Bilwa S T (Jira) 
Bilwa S T created MAPREDUCE-7308:


 Summary: Containers never get reused as containersToReuse map gets 
cleared on makeRemoteRequest
 Key: MAPREDUCE-7308
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-7308
 Project: Hadoop Map/Reduce
  Issue Type: Sub-task
Reporter: Bilwa S T
Assignee: Bilwa S T


In RMContainerReuseRequestor whenever containerAssigned is called it checks if 
allocated container can be reused. This always returns false as the map is 
getting cleared on makeRemoteRequest. I think there is no need to clear the map 
as container will be removed from containersToReuse map once its used.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org

[jira] [Commented] (MAPREDUCE-7304) Enhance the map-reduce Job end notifier to be able to notify the given URL via a custom class

2020-11-18 Thread Hadoop QA (Jira) 


[ 
https://issues.apache.org/jira/browse/MAPREDUCE-7304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17234443#comment-17234443
 ] 

Hadoop QA commented on MAPREDUCE-7304:
--

| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime ||  Logfile || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 26m  
8s{color} |  | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} || ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green}  0m  
1s{color} |  | {color:green} No case conflicting files found. {color} |
| {color:blue}0{color} | {color:blue} codespell {color} | {color:blue}  0m  
0s{color} |  | {color:blue} codespell was not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} |  | {color:green} The patch does not contain any @author tags. 
{color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m 
 0s{color} |  | {color:green} The patch appears to include 1 new or modified 
test files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} || ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  2m 
10s{color} |  | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 24m 
52s{color} |  | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  2m 
28s{color} |  | {color:green} trunk passed with JDK 
Ubuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  2m 
12s{color} |  | {color:green} trunk passed with JDK Private 
Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
46s{color} |  | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green}  1m 
17s{color} |  | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 
17m 53s{color} |  | {color:green} branch has no errors when building and 
testing our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
53s{color} |  | {color:green} trunk passed with JDK 
Ubuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
48s{color} |  | {color:green} trunk passed with JDK Private 
Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01 {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue}  1m  
2s{color} |  | {color:blue} Used deprecated FindBugs config; considering 
switching to SpotBugs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  2m 
25s{color} |  | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} || ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
27s{color} |  | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  1m 
 5s{color} |  | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  2m 
27s{color} |  | {color:green} the patch passed with JDK 
Ubuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  2m 
27s{color} |  | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  2m  
8s{color} |  | {color:green} the patch passed with JDK Private 
Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  2m  
8s{color} |  | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} blanks {color} | {color:green}  0m  
0s{color} |  | {color:green} The patch has no blanks issues. {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}  
0m 39s{color} | 
[/results-checkstyle-hadoop-mapreduce-project_hadoop-mapreduce-client.txt|https://ci-hadoop.apache.org/job/PreCommit-MAPREDUCE-Build/31/artifact/out/results-checkstyle-hadoop-mapreduce-project_hadoop-mapreduce-client.txt]
 | {color:orange} hadoop-mapreduce-project/hadoop-mapreduce-client: The patch 
generated 4 new + 675 unchanged - 0 fixed = 679 total (was 675) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green}  1m  
4s{color} |  | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green}  0m  
1s{color} |  | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedclient