[jira] [Commented] (MAPREDUCE-6305) AM/Task log page should be able to link back to the job
[ https://issues.apache.org/jira/browse/MAPREDUCE-6305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14660696#comment-14660696 ] Siqi Li commented on MAPREDUCE-6305: The latest patch can be applied to 2.6.0 branch cleanly AM/Task log page should be able to link back to the job --- Key: MAPREDUCE-6305 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6305 Project: Hadoop Map/Reduce Issue Type: Improvement Reporter: Siqi Li Assignee: Siqi Li Labels: BB2015-05-TBR Fix For: 2.8.0 Attachments: MAPREDUCE-6305.v1.patch, MAPREDUCE-6305.v2.patch, MAPREDUCE-6305.v3.patch, MAPREDUCE-6305.v4.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6240) Hadoop client displays confusing error message
[
https://issues.apache.org/jira/browse/MAPREDUCE-6240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14660806#comment-14660806
]
Gera Shegalov commented on MAPREDUCE-6240:
--
+1 for 1.7 addSuppressed for trunk, but we also need a patch backportable to
2.6.1.
Hadoop client displays confusing error message
--
Key: MAPREDUCE-6240
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6240
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: client
Affects Versions: 2.7.0
Reporter: Mohammad Kamrul Islam
Assignee: Gera Shegalov
Attachments: MAPREDUCE-6240-gera.001.patch,
MAPREDUCE-6240-gera.001.patch, MAPREDUCE-6240-gera.002.patch,
MAPREDUCE-6240.003.patch, MAPREDUCE-6240.1.patch
Hadoop client often throws exception with java.io.IOException: Cannot
initialize Cluster. Please check your configuration for
mapreduce.framework.name and the correspond server addresses.
This is a misleading and generic message for any cluster initialization
problem. It takes a lot of debugging hours to identify the root cause. The
correct error message could resolve this problem quickly.
In one such instance, Oozie log showed the following exception while the
root cause was CNF that Hadoop client didn't return in the exception.
{noformat}
JA009: Cannot initialize Cluster. Please check your configuration for
mapreduce.framework.name and the correspond server addresses.
at
org.apache.oozie.action.ActionExecutor.convertExceptionHelper(ActionExecutor.java:412)
at
org.apache.oozie.action.ActionExecutor.convertException(ActionExecutor.java:392)
at
org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:979)
at
org.apache.oozie.action.hadoop.JavaActionExecutor.start(JavaActionExecutor.java:1134)
at
org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:228)
at
org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:63)
at org.apache.oozie.command.XCommand.call(XCommand.java:281)
at
org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:323)
at
org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:252)
at
org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:174)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
Caused by: java.io.IOException: Cannot initialize Cluster. Please check your
configuration for mapreduce.framework.name and the correspond server
addresses.
at org.apache.hadoop.mapreduce.Cluster.initialize(Cluster.java:120)
at org.apache.hadoop.mapreduce.Cluster.init(Cluster.java:82)
at org.apache.hadoop.mapreduce.Cluster.init(Cluster.java:75)
at org.apache.hadoop.mapred.JobClient.init(JobClient.java:470)
at org.apache.hadoop.mapred.JobClient.init(JobClient.java:449)
at
org.apache.oozie.service.HadoopAccessorService$1.run(HadoopAccessorService.java:372)
at
org.apache.oozie.service.HadoopAccessorService$1.run(HadoopAccessorService.java:370)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
at
org.apache.oozie.service.HadoopAccessorService.createJobClient(HadoopAccessorService.java:379)
at
org.apache.oozie.action.hadoop.JavaActionExecutor.createJobClient(JavaActionExecutor.java:1185)
at
org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:927)
... 10 more
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6166) Reducers do not validate checksum of map outputs when fetching directly to disk
[ https://issues.apache.org/jira/browse/MAPREDUCE-6166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14660684#comment-14660684 ] Siqi Li commented on MAPREDUCE-6166: The latest patch can be applied to 2.6.0 branch cleanly Reducers do not validate checksum of map outputs when fetching directly to disk --- Key: MAPREDUCE-6166 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6166 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 2.6.0 Reporter: Eric Payne Assignee: Eric Payne Labels: 2.6.1-candidate Fix For: 2.7.0 Attachments: MAPREDUCE-6166.v1.201411221941.txt, MAPREDUCE-6166.v2.201411251627.txt, MAPREDUCE-6166.v3.txt, MAPREDUCE-6166.v4.txt, MAPREDUCE-6166.v5.txt In very large map/reduce jobs (5 maps, 2500 reducers), the intermediate map partition output gets corrupted on disk on the map side. If this corrupted map output is too large to shuffle in memory, the reducer streams it to disk without validating the checksum. In jobs this large, it could take hours before the reducer finally tries to read the corrupted file and fails. Since retries of the failed reduce attempt will also take hours, this delay in discovering the failure is multiplied greatly. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6257) Document encrypted spills
[ https://issues.apache.org/jira/browse/MAPREDUCE-6257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer updated MAPREDUCE-6257: Resolution: Fixed Fix Version/s: 3.0.0 Status: Resolved (was: Patch Available) +1 committed Document encrypted spills - Key: MAPREDUCE-6257 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6257 Project: Hadoop Map/Reduce Issue Type: Bug Components: security Reporter: Allen Wittenauer Assignee: Bibin A Chundatt Fix For: 3.0.0 Attachments: 0001-MAPREDUCE-6257.patch, 0002-MAPREDUCE-6257.patch, 0003-MAPREDUCE-6257.patch, EncryptedShuffle.html Encrypted spills appear to be completely undocumented. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6443) Add JvmPauseMonitor to Job History Server
[
https://issues.apache.org/jira/browse/MAPREDUCE-6443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14660437#comment-14660437
]
Robert Kanter commented on MAPREDUCE-6443:
--
Thanks for the review [~djp]
Add JvmPauseMonitor to Job History Server
-
Key: MAPREDUCE-6443
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6443
Project: Hadoop Map/Reduce
Issue Type: Improvement
Components: jobhistoryserver
Affects Versions: 2.8.0
Reporter: Robert Kanter
Assignee: Robert Kanter
Fix For: 2.8.0
Attachments: MAPREDUCE-6443.001.patch, MAPREDUCE-6443.002.patch
We should add the {{JvmPauseMonitor}} from HADOOP-9618 to the Job History
Server.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6240) Hadoop client displays confusing error message
[
https://issues.apache.org/jira/browse/MAPREDUCE-6240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14660599#comment-14660599
]
Chris Douglas commented on MAPREDUCE-6240:
--
bq. how about refactoring it and removing this class
org.apache.hadoop.io.MultipleIOException
We'd have to audit where it's used. If there could be systems that expect and
handle it, we'd have to deprecate it first, but I think it makes sense to
remove it in trunk. Separate issue, of course.
Hadoop client displays confusing error message
--
Key: MAPREDUCE-6240
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6240
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: client
Affects Versions: 2.7.0
Reporter: Mohammad Kamrul Islam
Assignee: Gera Shegalov
Attachments: MAPREDUCE-6240-gera.001.patch,
MAPREDUCE-6240-gera.001.patch, MAPREDUCE-6240-gera.002.patch,
MAPREDUCE-6240.003.patch, MAPREDUCE-6240.1.patch
Hadoop client often throws exception with java.io.IOException: Cannot
initialize Cluster. Please check your configuration for
mapreduce.framework.name and the correspond server addresses.
This is a misleading and generic message for any cluster initialization
problem. It takes a lot of debugging hours to identify the root cause. The
correct error message could resolve this problem quickly.
In one such instance, Oozie log showed the following exception while the
root cause was CNF that Hadoop client didn't return in the exception.
{noformat}
JA009: Cannot initialize Cluster. Please check your configuration for
mapreduce.framework.name and the correspond server addresses.
at
org.apache.oozie.action.ActionExecutor.convertExceptionHelper(ActionExecutor.java:412)
at
org.apache.oozie.action.ActionExecutor.convertException(ActionExecutor.java:392)
at
org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:979)
at
org.apache.oozie.action.hadoop.JavaActionExecutor.start(JavaActionExecutor.java:1134)
at
org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:228)
at
org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:63)
at org.apache.oozie.command.XCommand.call(XCommand.java:281)
at
org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:323)
at
org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:252)
at
org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:174)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
Caused by: java.io.IOException: Cannot initialize Cluster. Please check your
configuration for mapreduce.framework.name and the correspond server
addresses.
at org.apache.hadoop.mapreduce.Cluster.initialize(Cluster.java:120)
at org.apache.hadoop.mapreduce.Cluster.init(Cluster.java:82)
at org.apache.hadoop.mapreduce.Cluster.init(Cluster.java:75)
at org.apache.hadoop.mapred.JobClient.init(JobClient.java:470)
at org.apache.hadoop.mapred.JobClient.init(JobClient.java:449)
at
org.apache.oozie.service.HadoopAccessorService$1.run(HadoopAccessorService.java:372)
at
org.apache.oozie.service.HadoopAccessorService$1.run(HadoopAccessorService.java:370)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
at
org.apache.oozie.service.HadoopAccessorService.createJobClient(HadoopAccessorService.java:379)
at
org.apache.oozie.action.hadoop.JavaActionExecutor.createJobClient(JavaActionExecutor.java:1185)
at
org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:927)
... 10 more
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6257) Document encrypted spills
[ https://issues.apache.org/jira/browse/MAPREDUCE-6257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14660350#comment-14660350 ] Bibin A Chundatt commented on MAPREDUCE-6257: - [~aw] Please do review. Document encrypted spills - Key: MAPREDUCE-6257 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6257 Project: Hadoop Map/Reduce Issue Type: Bug Components: security Reporter: Allen Wittenauer Assignee: Bibin A Chundatt Attachments: 0001-MAPREDUCE-6257.patch, 0002-MAPREDUCE-6257.patch, 0003-MAPREDUCE-6257.patch, EncryptedShuffle.html Encrypted spills appear to be completely undocumented. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6257) Document encrypted spills
[ https://issues.apache.org/jira/browse/MAPREDUCE-6257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14660398#comment-14660398 ] Hudson commented on MAPREDUCE-6257: --- FAILURE: Integrated in Hadoop-trunk-Commit #8273 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/8273/]) MAPREDUCE-6257. Document encrypted spills (Bibin A Chundatt via aw) (aw: rev fb1be0b3100cdd69f6dc1987585fcadd4e7c8a2a) * hadoop-mapreduce-project/CHANGES.txt * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/site/markdown/EncryptedShuffle.md * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml Document encrypted spills - Key: MAPREDUCE-6257 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6257 Project: Hadoop Map/Reduce Issue Type: Bug Components: security Reporter: Allen Wittenauer Assignee: Bibin A Chundatt Fix For: 3.0.0 Attachments: 0001-MAPREDUCE-6257.patch, 0002-MAPREDUCE-6257.patch, 0003-MAPREDUCE-6257.patch, EncryptedShuffle.html Encrypted spills appear to be completely undocumented. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6443) Add JvmPauseMonitor to Job History Server
[
https://issues.apache.org/jira/browse/MAPREDUCE-6443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14660508#comment-14660508
]
Hudson commented on MAPREDUCE-6443:
---
SUCCESS: Integrated in Hadoop-Mapreduce-trunk-Java8 #276 (See
[https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/276/])
MAPREDUCE-6443. Add JvmPauseMonitor to JobHistoryServer. Contributed by Robert
Kanter. (junping_du: rev e73a928a6360f68aaee2ed58b3a8d180f4051407)
*
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/JobHistoryServer.java
* hadoop-mapreduce-project/CHANGES.txt
Add JvmPauseMonitor to Job History Server
-
Key: MAPREDUCE-6443
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6443
Project: Hadoop Map/Reduce
Issue Type: Improvement
Components: jobhistoryserver
Affects Versions: 2.8.0
Reporter: Robert Kanter
Assignee: Robert Kanter
Fix For: 2.8.0
Attachments: MAPREDUCE-6443.001.patch, MAPREDUCE-6443.002.patch
We should add the {{JvmPauseMonitor}} from HADOOP-9618 to the Job History
Server.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6443) Add JvmPauseMonitor to Job History Server
[
https://issues.apache.org/jira/browse/MAPREDUCE-6443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14660525#comment-14660525
]
Hudson commented on MAPREDUCE-6443:
---
FAILURE: Integrated in Hadoop-Mapreduce-trunk #2225 (See
[https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2225/])
MAPREDUCE-6443. Add JvmPauseMonitor to JobHistoryServer. Contributed by Robert
Kanter. (junping_du: rev e73a928a6360f68aaee2ed58b3a8d180f4051407)
*
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/JobHistoryServer.java
* hadoop-mapreduce-project/CHANGES.txt
Add JvmPauseMonitor to Job History Server
-
Key: MAPREDUCE-6443
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6443
Project: Hadoop Map/Reduce
Issue Type: Improvement
Components: jobhistoryserver
Affects Versions: 2.8.0
Reporter: Robert Kanter
Assignee: Robert Kanter
Fix For: 2.8.0
Attachments: MAPREDUCE-6443.001.patch, MAPREDUCE-6443.002.patch
We should add the {{JvmPauseMonitor}} from HADOOP-9618 to the Job History
Server.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6443) Add JvmPauseMonitor to Job History Server
[
https://issues.apache.org/jira/browse/MAPREDUCE-6443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14660017#comment-14660017
]
Junping Du commented on MAPREDUCE-6443:
---
+1. 002 patch LGTM. Will commit it shortly.
Add JvmPauseMonitor to Job History Server
-
Key: MAPREDUCE-6443
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6443
Project: Hadoop Map/Reduce
Issue Type: Improvement
Components: jobhistoryserver
Affects Versions: 2.8.0
Reporter: Robert Kanter
Assignee: Robert Kanter
Attachments: MAPREDUCE-6443.001.patch, MAPREDUCE-6443.002.patch
We should add the {{JvmPauseMonitor}} from HADOOP-9618 to the Job History
Server.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6443) Add JvmPauseMonitor to Job History Server
[
https://issues.apache.org/jira/browse/MAPREDUCE-6443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Junping Du updated MAPREDUCE-6443:
--
Resolution: Fixed
Hadoop Flags: Reviewed
Fix Version/s: 2.8.0
Status: Resolved (was: Patch Available)
Add JvmPauseMonitor to Job History Server
-
Key: MAPREDUCE-6443
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6443
Project: Hadoop Map/Reduce
Issue Type: Improvement
Components: jobhistoryserver
Affects Versions: 2.8.0
Reporter: Robert Kanter
Assignee: Robert Kanter
Fix For: 2.8.0
Attachments: MAPREDUCE-6443.001.patch, MAPREDUCE-6443.002.patch
We should add the {{JvmPauseMonitor}} from HADOOP-9618 to the Job History
Server.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (MAPREDUCE-6445) Shuffle hang
Peng Zhang created MAPREDUCE-6445:
-
Summary: Shuffle hang
Key: MAPREDUCE-6445
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6445
Project: Hadoop Map/Reduce
Issue Type: Bug
Affects Versions: 2.6.0
Reporter: Peng Zhang
Scale cluster has run for months with 2.6.0.
2 of 200 reduces hang on shuffle
instance 1 log seems like loop on 1 map output:
{noformat}
2015-08-06 21:54:14,649 INFO [fetcher#1]
org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 2 of 2
to node-132.bj:22408 to fetcher#1
2015-08-06 21:54:14,651 INFO [fetcher#1]
org.apache.hadoop.mapreduce.task.reduce.Fetcher: for
url=22408/mapOutput?job=job_1438689528746_10193reduce=20map=attempt_1438689528746_10193_m_13_0,attempt_1438689528746_10193_m_20_0
sent hash and received reply
2015-08-06 21:54:14,651 INFO [fetcher#1]
org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#1 - MergeManager
returned status WAIT ...
2015-08-06 21:54:14,651 INFO [fetcher#1]
org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: node-132.bj:22408
freed by fetcher#1 in 2ms
2015-08-06 21:54:14,651 INFO [fetcher#5]
org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning
node-132.bj:22408 with 2 to fetcher#5
2015-08-06 21:54:14,651 INFO [fetcher#5]
org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 2 of 2
to node-132.bj:22408 to fetcher#5
2015-08-06 21:54:14,656 INFO [fetcher#5]
org.apache.hadoop.mapreduce.task.reduce.Fetcher: for
url=22408/mapOutput?job=job_1438689528746_10193reduce=20map=attempt_1438689528746_10193_m_13_0,attempt_1438689528746_10193_m_20_0
sent hash and received reply
2015-08-06 21:54:14,656 INFO [fetcher#5]
org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#5 - MergeManager
returned status WAIT ...
2015-08-06 21:54:14,656 INFO [fetcher#5]
org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: node-132.bj:22408
freed by fetcher#5 in 4ms
2015-08-06 21:54:14,656 INFO [fetcher#5]
org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning
node-132.bj:22408 with 2 to fetcher#5
2015-08-06 21:54:14,656 INFO [fetcher#5]
org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 2 of 2
to node-132.bj:22408 to fetcher#5
2015-08-06 21:54:14,660 INFO [fetcher#5]
org.apache.hadoop.mapreduce.task.reduce.Fetcher: for
url=22408/mapOutput?job=job_1438689528746_10193reduce=20map=attempt_1438689528746_10193_m_13_0,attempt_1438689528746_10193_m_20_0
sent hash and received reply
2015-08-06 21:54:14,660 INFO [fetcher#5]
org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#5 - MergeManager
returned status WAIT ...
2015-08-06 21:54:14,660 INFO [fetcher#5]
org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: node-132.bj:22408
freed by fetcher#5 in 5ms
2015-08-06 21:54:14,660 INFO [fetcher#5]
org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning
node-132.bj:22408 with 2 to fetcher#5
2015-08-06 21:54:14,660 INFO [fetcher#5]
org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 2 of 2
to node-132.bj:22408 to fetcher#5
{noformat}
node 2 log seems like loop on 5 map output:
{noformat}
2015-08-06 21:43:33,626 INFO [fetcher#5]
org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning
node-172.bj:22408 with 1 to fetcher#5
2015-08-06 21:43:33,626 INFO [fetcher#5]
org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1
to node-172.bj:22408 to fetcher#5
2015-08-06 21:43:33,627 INFO [fetcher#3]
org.apache.hadoop.mapreduce.task.reduce.Fetcher: for
url=22408/mapOutput?job=job_1438689528746_10193reduce=85map=attempt_1438689528746_10193_m_13_0,attempt_1438689528746_10193_m_20_0
sent hash and received reply
2015-08-06 21:43:33,627 INFO [fetcher#3]
org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#3 - MergeManager
returned status WAIT ...
2015-08-06 21:43:33,627 INFO [fetcher#3]
org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: node-132.bj:22408
freed by fetcher#3 in 5ms
2015-08-06 21:43:33,627 INFO [fetcher#3]
org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning
node-179.bj:22408 with 1 to fetcher#3
2015-08-06 21:43:33,627 INFO [fetcher#3]
org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1
to node-179.bj:22408 to fetcher#3
2015-08-06 21:43:33,627 INFO [fetcher#4]
org.apache.hadoop.mapreduce.task.reduce.Fetcher: for
url=22408/mapOutput?job=job_1438689528746_10193reduce=85map=attempt_1438689528746_10193_m_84_0,attempt_1438689528746_10193_m_46_0
sent hash and received reply
2015-08-06 21:43:33,627 INFO [fetcher#4]
org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#4 - MergeManager
returned status WAIT ...
2015-08-06 21:43:33,627 INFO [fetcher#4]
org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: node-71.bj:22408
[jira] [Commented] (MAPREDUCE-6443) Add JvmPauseMonitor to Job History Server
[
https://issues.apache.org/jira/browse/MAPREDUCE-6443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14660053#comment-14660053
]
Junping Du commented on MAPREDUCE-6443:
---
I have commit the patch to trunk and branch-2. Thanks [~rkanter] for the
contribution!
Add JvmPauseMonitor to Job History Server
-
Key: MAPREDUCE-6443
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6443
Project: Hadoop Map/Reduce
Issue Type: Improvement
Components: jobhistoryserver
Affects Versions: 2.8.0
Reporter: Robert Kanter
Assignee: Robert Kanter
Attachments: MAPREDUCE-6443.001.patch, MAPREDUCE-6443.002.patch
We should add the {{JvmPauseMonitor}} from HADOOP-9618 to the Job History
Server.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6240) Hadoop client displays confusing error message
[
https://issues.apache.org/jira/browse/MAPREDUCE-6240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14659704#comment-14659704
]
Ajith S commented on MAPREDUCE-6240:
Hi [~chris.douglas]
+1 of avoiding composites. So, how about refactoring it and removing this class
org.apache.hadoop.io.MultipleIOException :) u think its a good idea.?
Hadoop client displays confusing error message
--
Key: MAPREDUCE-6240
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6240
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: client
Affects Versions: 2.7.0
Reporter: Mohammad Kamrul Islam
Assignee: Gera Shegalov
Attachments: MAPREDUCE-6240-gera.001.patch,
MAPREDUCE-6240-gera.001.patch, MAPREDUCE-6240-gera.002.patch,
MAPREDUCE-6240.003.patch, MAPREDUCE-6240.1.patch
Hadoop client often throws exception with java.io.IOException: Cannot
initialize Cluster. Please check your configuration for
mapreduce.framework.name and the correspond server addresses.
This is a misleading and generic message for any cluster initialization
problem. It takes a lot of debugging hours to identify the root cause. The
correct error message could resolve this problem quickly.
In one such instance, Oozie log showed the following exception while the
root cause was CNF that Hadoop client didn't return in the exception.
{noformat}
JA009: Cannot initialize Cluster. Please check your configuration for
mapreduce.framework.name and the correspond server addresses.
at
org.apache.oozie.action.ActionExecutor.convertExceptionHelper(ActionExecutor.java:412)
at
org.apache.oozie.action.ActionExecutor.convertException(ActionExecutor.java:392)
at
org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:979)
at
org.apache.oozie.action.hadoop.JavaActionExecutor.start(JavaActionExecutor.java:1134)
at
org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:228)
at
org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:63)
at org.apache.oozie.command.XCommand.call(XCommand.java:281)
at
org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:323)
at
org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:252)
at
org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:174)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
Caused by: java.io.IOException: Cannot initialize Cluster. Please check your
configuration for mapreduce.framework.name and the correspond server
addresses.
at org.apache.hadoop.mapreduce.Cluster.initialize(Cluster.java:120)
at org.apache.hadoop.mapreduce.Cluster.init(Cluster.java:82)
at org.apache.hadoop.mapreduce.Cluster.init(Cluster.java:75)
at org.apache.hadoop.mapred.JobClient.init(JobClient.java:470)
at org.apache.hadoop.mapred.JobClient.init(JobClient.java:449)
at
org.apache.oozie.service.HadoopAccessorService$1.run(HadoopAccessorService.java:372)
at
org.apache.oozie.service.HadoopAccessorService$1.run(HadoopAccessorService.java:370)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
at
org.apache.oozie.service.HadoopAccessorService.createJobClient(HadoopAccessorService.java:379)
at
org.apache.oozie.action.hadoop.JavaActionExecutor.createJobClient(JavaActionExecutor.java:1185)
at
org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:927)
... 10 more
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6443) Add JvmPauseMonitor to Job History Server
[
https://issues.apache.org/jira/browse/MAPREDUCE-6443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14660189#comment-14660189
]
Hudson commented on MAPREDUCE-6443:
---
FAILURE: Integrated in Hadoop-trunk-Commit #8272 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/8272/])
MAPREDUCE-6443. Add JvmPauseMonitor to JobHistoryServer. Contributed by Robert
Kanter. (junping_du: rev e73a928a6360f68aaee2ed58b3a8d180f4051407)
* hadoop-mapreduce-project/CHANGES.txt
*
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/JobHistoryServer.java
Add JvmPauseMonitor to Job History Server
-
Key: MAPREDUCE-6443
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6443
Project: Hadoop Map/Reduce
Issue Type: Improvement
Components: jobhistoryserver
Affects Versions: 2.8.0
Reporter: Robert Kanter
Assignee: Robert Kanter
Fix For: 2.8.0
Attachments: MAPREDUCE-6443.001.patch, MAPREDUCE-6443.002.patch
We should add the {{JvmPauseMonitor}} from HADOOP-9618 to the Job History
Server.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
