[jira] [Commented] (MAPREDUCE-6702) TestMiniMRChildTask.testTaskEnv and TestMiniMRChildTask.testTaskOldEnv are failing

2016-06-06 Thread Daniel Templeton (JIRA)

[ 
https://issues.apache.org/jira/browse/MAPREDUCE-6702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15317473#comment-15317473
 ] 

Daniel Templeton commented on MAPREDUCE-6702:
-

Looks good to me.

> TestMiniMRChildTask.testTaskEnv and TestMiniMRChildTask.testTaskOldEnv are 
> failing
> --
>
> Key: MAPREDUCE-6702
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6702
> Project: Hadoop Map/Reduce
>  Issue Type: Bug
>  Components: client
>Affects Versions: 3.0.0-alpha1
>Reporter: Daniel Templeton
>Assignee: Akira AJISAKA
> Attachments: MAPREDUCE-6702.01.patch
>
>
> ---
>  T E S T S
> ---
> Running org.apache.hadoop.mapred.TestMiniMRChildTask
> Tests run: 3, Failures: 2, Errors: 0, Skipped: 0, Time elapsed: 92.48 sec <<< 
> FAILURE! - in org.apache.hadoop.mapred.TestMiniMRChildTask
> testTaskEnv(org.apache.hadoop.mapred.TestMiniMRChildTask)  Time elapsed: 
> 21.906 sec  <<< FAILURE!
> java.lang.AssertionError: The environment checker job failed.
>   at org.junit.Assert.fail(Assert.java:88)
>   at org.junit.Assert.assertTrue(Assert.java:41)
>   at 
> org.apache.hadoop.mapred.TestMiniMRChildTask.runTestTaskEnv(TestMiniMRChildTask.java:550)
>   at 
> org.apache.hadoop.mapred.TestMiniMRChildTask.testTaskEnv(TestMiniMRChildTask.java:472)
> testTaskOldEnv(org.apache.hadoop.mapred.TestMiniMRChildTask)  Time elapsed: 
> 17.452 sec  <<< FAILURE!
> java.lang.AssertionError: The environment checker job failed.
>   at org.junit.Assert.fail(Assert.java:88)
>   at org.junit.Assert.assertTrue(Assert.java:41)
>   at 
> org.apache.hadoop.mapred.TestMiniMRChildTask.runTestTaskEnv(TestMiniMRChildTask.java:550)
>   at 
> org.apache.hadoop.mapred.TestMiniMRChildTask.testTaskOldEnv(TestMiniMRChildTask.java:496)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org



[jira] [Commented] (MAPREDUCE-6652) Add configuration property to prevent JHS from loading jobs with a task count greater than X

2016-06-06 Thread Robert Kanter (JIRA)

[ 
https://issues.apache.org/jira/browse/MAPREDUCE-6652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15317420#comment-15317420
 ] 

Robert Kanter commented on MAPREDUCE-6652:
--

Two last things
- {{UnparsedJob#getReport()}} needs to be synchronized now that we're storing 
the {{JobReport}}.
- It doesn't apply to branch-2.  Can you create a branch-2 version of the 
patch?  It looks like a trivial conflict though.


> Add configuration property to prevent JHS from loading jobs with a task count 
> greater than X
> 
>
> Key: MAPREDUCE-6652
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6652
> Project: Hadoop Map/Reduce
>  Issue Type: Improvement
>  Components: jobhistoryserver
>Reporter: Haibo Chen
>Assignee: Haibo Chen
> Attachments: mapreduce6652.001.patch, mapreduce6652.002.patch, 
> mapreduce6652.003.patch, mapreduce6652.004.patch, mapreduce6652.005.patch
>
>
> Jobs with large number of tasks can have job history files that are large in 
> size and resource-consuming(mainly memory) to parse in Job History Server. If 
> there are many such jobs, the job history server can very easily hang.
> It would be a good usability feature if we added a new config property that 
> could be set to X, where the JHS wouldn't load the details for a job with 
> more than X tasks. The job would still show up on the list of jobs page, but 
> clicking on it would give a warning message that the job is too big, instead 
> of actually loading the job. This way we can prevent users from loading a job 
> that's way too big for the JHS, which currently makes the JHS hang. The 
> default value can be -1 so that it's disabled.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org



[jira] [Commented] (MAPREDUCE-6652) Add configuration property to prevent JHS from loading jobs with a task count greater than X

2016-06-06 Thread Hadoop QA (JIRA)

[ 
https://issues.apache.org/jira/browse/MAPREDUCE-6652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15317401#comment-15317401
 ] 

Hadoop QA commented on MAPREDUCE-6652:
--

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s 
{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} docker {color} | {color:red} 0m 5s {color} 
| {color:red} Docker failed to build yetus/hadoop:2c91fd8. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12808485/mapreduce6652.005.patch
 |
| JIRA Issue | MAPREDUCE-6652 |
| Console output | 
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6538/console |
| Powered by | Apache Yetus 0.3.0   http://yetus.apache.org |


This message was automatically generated.



> Add configuration property to prevent JHS from loading jobs with a task count 
> greater than X
> 
>
> Key: MAPREDUCE-6652
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6652
> Project: Hadoop Map/Reduce
>  Issue Type: Improvement
>  Components: jobhistoryserver
>Reporter: Haibo Chen
>Assignee: Haibo Chen
> Attachments: mapreduce6652.001.patch, mapreduce6652.002.patch, 
> mapreduce6652.003.patch, mapreduce6652.004.patch, mapreduce6652.005.patch
>
>
> Jobs with large number of tasks can have job history files that are large in 
> size and resource-consuming(mainly memory) to parse in Job History Server. If 
> there are many such jobs, the job history server can very easily hang.
> It would be a good usability feature if we added a new config property that 
> could be set to X, where the JHS wouldn't load the details for a job with 
> more than X tasks. The job would still show up on the list of jobs page, but 
> clicking on it would give a warning message that the job is too big, instead 
> of actually loading the job. This way we can prevent users from loading a job 
> that's way too big for the JHS, which currently makes the JHS hang. The 
> default value can be -1 so that it's disabled.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org



[jira] [Commented] (MAPREDUCE-5044) Have AM trigger jstack on task attempts that timeout before killing them

2016-06-06 Thread Hudson (JIRA)

[ 
https://issues.apache.org/jira/browse/MAPREDUCE-5044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15317398#comment-15317398
 ] 

Hudson commented on MAPREDUCE-5044:
---

SUCCESS: Integrated in Hadoop-trunk-Commit #9915 (See 
[https://builds.apache.org/job/Hadoop-trunk-Commit/9915/])
MAPREDUCE-5044. Have AM trigger jstack on task attempts that timeout (mingma: 
rev 4a1cedc010d3fa1d8ef3f2773ca12acadfee5ba5)
* 
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestSignalContainer.java
* 
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/main/java/org/apache/hadoop/yarn/client/api/YarnClient.java
* 
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/applicationclient_protocol.proto
* 
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/TestRPC.java
* 
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/NodeManager.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ResourceMgrDelegate.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/launcher/ContainerLauncherImpl.java
* 
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/SignalContainerResponse.java
* 
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/main/java/org/apache/hadoop/yarn/client/cli/ApplicationCLI.java
* 
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/TestContainerResourceIncreaseRPC.java
* 
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/amrmproxy/MockResourceManagerFacade.java
* 
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/ContainerManagerImpl.java
* 
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ClientRMService.java
* 
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/impl/pb/service/ContainerManagementProtocolPBServiceImpl.java
* 
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestApplicationMasterLauncher.java
* 
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/ApplicationClientProtocol.java
* 
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/impl/pb/client/ApplicationClientProtocolPBClientImpl.java
* 
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/api/impl/TestYarnClient.java
* 
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/ContainerManagementProtocolPB.java
* 
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestAMAuthorization.java
* 
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockRM.java
* 
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/containermanagement_protocol.proto
* 
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/main/java/org/apache/hadoop/yarn/client/api/impl/YarnClientImpl.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/launcher/ContainerLauncherEvent.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TaskAttemptImpl.java
* 
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/ContainerManagementProtocol.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapred/LocalContainerLauncher.java
* 
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/TestContainerLaunchRPC.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/v2/TestMRJobs.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestClientRedirect.java
* 
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/impl/pb/service/ApplicationClientProtocolPBServiceImpl.java
* 

[jira] [Commented] (MAPREDUCE-6542) HistoryViewer use SimpleDateFormat,But SimpleDateFormat is not threadsafe

2016-06-06 Thread Daniel Templeton (JIRA)

[ 
https://issues.apache.org/jira/browse/MAPREDUCE-6542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15317368#comment-15317368
 ] 

Daniel Templeton commented on MAPREDUCE-6542:
-

Thanks for the updated patch, [~piaoyu zhang].  I see two little things to 
consider correcting.  First:

{code}
  String formattedTime1 = 
StringUtils.getFormattedTimeWithDiff(fastDateFormat, +start, end);
  String formattedTime2 = 
StringUtils.getFormattedTimeWithDiff(fastDateFormat, +start, end);
{code}

The unary plus operator is superfluous here.  (Is it ever not superfluous?)  
Since {{start}} is a long, the + has no effect.

Also:

{code}
  final private static FastDateFormat fastDateFormat =
  FastDateFormat.getInstance("d-MMM- HH:mm:ss");
{code}

should probably be:

{code}
  final private static FastDateFormat FAST_DATE_FORMAT =
  FastDateFormat.getInstance("d-MMM- HH:mm:ss");
{code}

Similar for:

{code}
  private final FastDateFormat dateFormat;
{code}

> HistoryViewer use SimpleDateFormat,But SimpleDateFormat is not threadsafe
> -
>
> Key: MAPREDUCE-6542
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6542
> Project: Hadoop Map/Reduce
>  Issue Type: Bug
>  Components: jobhistoryserver
>Affects Versions: 2.2.0, 2.7.1
> Environment: CentOS6.5 Hadoop  
>Reporter: zhangyubiao
>Assignee: zhangyubiao
> Attachments: MAPREDUCE-6542-v10.patch, MAPREDUCE-6542-v11.patch, 
> MAPREDUCE-6542-v12.patch, MAPREDUCE-6542-v2.patch, MAPREDUCE-6542-v3.patch, 
> MAPREDUCE-6542-v4.patch, MAPREDUCE-6542-v5.patch, MAPREDUCE-6542-v6.patch, 
> MAPREDUCE-6542-v7.patch, MAPREDUCE-6542-v8.patch, MAPREDUCE-6542-v9.patch, 
> MAPREDUCE-6542.patch
>
>
> I use SimpleDateFormat to Parse the JobHistory File before 
> {code}
> private static final SimpleDateFormat dateFormat =
> new SimpleDateFormat("-MM-dd HH:mm:ss");
>  public static String getJobDetail(JobInfo job) {
> StringBuffer jobDetails = new StringBuffer("");
> SummarizedJob ts = new SummarizedJob(job);
> jobDetails.append(job.getJobId().toString().trim()).append("\t");
> jobDetails.append(job.getUsername()).append("\t");
> jobDetails.append(job.getJobname().replaceAll("\\n", 
> "")).append("\t");
> jobDetails.append(job.getJobQueueName()).append("\t");
> jobDetails.append(job.getPriority()).append("\t");
> jobDetails.append(job.getJobConfPath()).append("\t");
> jobDetails.append(job.getUberized()).append("\t");
> 
> jobDetails.append(dateFormat.format(job.getSubmitTime())).append("\t");
> 
> jobDetails.append(dateFormat.format(job.getLaunchTime())).append("\t");
> 
> jobDetails.append(dateFormat.format(job.getFinishTime())).append("\t");
>return jobDetails.toString();
> }
> {code}
> But I find I query the SubmitTime and LaunchTime in hive and compare 
> JobHistory File time , I find that the submitTime  and launchTime was wrong.
> Finally,I change to use the FastDateFormat to parse the time format and the 
> time become right



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org



[jira] [Updated] (MAPREDUCE-5044) Have AM trigger jstack on task attempts that timeout before killing them

2016-06-06 Thread Ming Ma (JIRA)

 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-5044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ming Ma updated MAPREDUCE-5044:
---
   Resolution: Fixed
 Hadoop Flags: Reviewed
Fix Version/s: 2.8.0
   Status: Resolved  (was: Patch Available)

> Have AM trigger jstack on task attempts that timeout before killing them
> 
>
> Key: MAPREDUCE-5044
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5044
> Project: Hadoop Map/Reduce
>  Issue Type: Improvement
>  Components: mr-am
>Affects Versions: 2.1.0-beta
>Reporter: Jason Lowe
>Assignee: Eric Payne
> Fix For: 2.8.0
>
> Attachments: MAPREDUCE-5044.008.patch, MAPREDUCE-5044.009.patch, 
> MAPREDUCE-5044.010.patch, MAPREDUCE-5044.011.patch, MAPREDUCE-5044.012.patch, 
> MAPREDUCE-5044.013.patch, MAPREDUCE-5044.v01.patch, MAPREDUCE-5044.v02.patch, 
> MAPREDUCE-5044.v03.patch, MAPREDUCE-5044.v04.patch, MAPREDUCE-5044.v05.patch, 
> MAPREDUCE-5044.v06.patch, MAPREDUCE-5044.v07.local.patch, Screen Shot 
> 2013-11-12 at 1.05.32 PM.png, Screen Shot 2013-11-12 at 1.06.04 PM.png
>
>
> When an AM expires a task attempt it would be nice if it triggered a jstack 
> output via SIGQUIT before killing the task attempt.  This would be invaluable 
> for helping users debug their hung tasks, especially if they do not have 
> shell access to the nodes.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org



[jira] [Commented] (MAPREDUCE-5044) Have AM trigger jstack on task attempts that timeout before killing them

2016-06-06 Thread Ming Ma (JIRA)

[ 
https://issues.apache.org/jira/browse/MAPREDUCE-5044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15317343#comment-15317343
 ] 

Ming Ma commented on MAPREDUCE-5044:


I have committed the patch to trunk, branch-2 and branch-2.8. Thank you 
[~eepayne] and [~jira.shegalov] for the contribution and [~vinodkv] [~jlowe] 
and [~aw] for the review.

> Have AM trigger jstack on task attempts that timeout before killing them
> 
>
> Key: MAPREDUCE-5044
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5044
> Project: Hadoop Map/Reduce
>  Issue Type: Improvement
>  Components: mr-am
>Affects Versions: 2.1.0-beta
>Reporter: Jason Lowe
>Assignee: Eric Payne
> Attachments: MAPREDUCE-5044.008.patch, MAPREDUCE-5044.009.patch, 
> MAPREDUCE-5044.010.patch, MAPREDUCE-5044.011.patch, MAPREDUCE-5044.012.patch, 
> MAPREDUCE-5044.013.patch, MAPREDUCE-5044.v01.patch, MAPREDUCE-5044.v02.patch, 
> MAPREDUCE-5044.v03.patch, MAPREDUCE-5044.v04.patch, MAPREDUCE-5044.v05.patch, 
> MAPREDUCE-5044.v06.patch, MAPREDUCE-5044.v07.local.patch, Screen Shot 
> 2013-11-12 at 1.05.32 PM.png, Screen Shot 2013-11-12 at 1.06.04 PM.png
>
>
> When an AM expires a task attempt it would be nice if it triggered a jstack 
> output via SIGQUIT before killing the task attempt.  This would be invaluable 
> for helping users debug their hung tasks, especially if they do not have 
> shell access to the nodes.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org



[jira] [Commented] (MAPREDUCE-6652) Add configuration property to prevent JHS from loading jobs with a task count greater than X

2016-06-06 Thread Haibo Chen (JIRA)

[ 
https://issues.apache.org/jira/browse/MAPREDUCE-6652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15317324#comment-15317324
 ] 

Haibo Chen commented on MAPREDUCE-6652:
---

Thanks [~rkanter] for your reviews. I have updated the patch according to your 
comments, and also added a new test case as you suggested. 

> Add configuration property to prevent JHS from loading jobs with a task count 
> greater than X
> 
>
> Key: MAPREDUCE-6652
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6652
> Project: Hadoop Map/Reduce
>  Issue Type: Improvement
>  Components: jobhistoryserver
>Reporter: Haibo Chen
>Assignee: Haibo Chen
> Attachments: mapreduce6652.001.patch, mapreduce6652.002.patch, 
> mapreduce6652.003.patch, mapreduce6652.004.patch, mapreduce6652.005.patch
>
>
> Jobs with large number of tasks can have job history files that are large in 
> size and resource-consuming(mainly memory) to parse in Job History Server. If 
> there are many such jobs, the job history server can very easily hang.
> It would be a good usability feature if we added a new config property that 
> could be set to X, where the JHS wouldn't load the details for a job with 
> more than X tasks. The job would still show up on the list of jobs page, but 
> clicking on it would give a warning message that the job is too big, instead 
> of actually loading the job. This way we can prevent users from loading a job 
> that's way too big for the JHS, which currently makes the JHS hang. The 
> default value can be -1 so that it's disabled.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org



[jira] [Updated] (MAPREDUCE-6652) Add configuration property to prevent JHS from loading jobs with a task count greater than X

2016-06-06 Thread Haibo Chen (JIRA)

 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-6652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Haibo Chen updated MAPREDUCE-6652:
--
Attachment: mapreduce6652.005.patch

> Add configuration property to prevent JHS from loading jobs with a task count 
> greater than X
> 
>
> Key: MAPREDUCE-6652
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6652
> Project: Hadoop Map/Reduce
>  Issue Type: Improvement
>  Components: jobhistoryserver
>Reporter: Haibo Chen
>Assignee: Haibo Chen
> Attachments: mapreduce6652.001.patch, mapreduce6652.002.patch, 
> mapreduce6652.003.patch, mapreduce6652.004.patch, mapreduce6652.005.patch
>
>
> Jobs with large number of tasks can have job history files that are large in 
> size and resource-consuming(mainly memory) to parse in Job History Server. If 
> there are many such jobs, the job history server can very easily hang.
> It would be a good usability feature if we added a new config property that 
> could be set to X, where the JHS wouldn't load the details for a job with 
> more than X tasks. The job would still show up on the list of jobs page, but 
> clicking on it would give a warning message that the job is too big, instead 
> of actually loading the job. This way we can prevent users from loading a job 
> that's way too big for the JHS, which currently makes the JHS hang. The 
> default value can be -1 so that it's disabled.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org



[jira] [Commented] (MAPREDUCE-5044) Have AM trigger jstack on task attempts that timeout before killing them

2016-06-06 Thread Eric Payne (JIRA)

[ 
https://issues.apache.org/jira/browse/MAPREDUCE-5044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15317290#comment-15317290
 ] 

Eric Payne commented on MAPREDUCE-5044:
---

bq. The patch doesn't resolve automatically for branch-2 and 2.8. It is 
straightforward and I will resolve it for those two branches.
[~mingma],
I did see that, but I was hoping it was straightforward enough that it didn't 
need a separate patch. Thanks for doing the extra work for the cherry-pick.

> Have AM trigger jstack on task attempts that timeout before killing them
> 
>
> Key: MAPREDUCE-5044
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5044
> Project: Hadoop Map/Reduce
>  Issue Type: Improvement
>  Components: mr-am
>Affects Versions: 2.1.0-beta
>Reporter: Jason Lowe
>Assignee: Eric Payne
> Attachments: MAPREDUCE-5044.008.patch, MAPREDUCE-5044.009.patch, 
> MAPREDUCE-5044.010.patch, MAPREDUCE-5044.011.patch, MAPREDUCE-5044.012.patch, 
> MAPREDUCE-5044.013.patch, MAPREDUCE-5044.v01.patch, MAPREDUCE-5044.v02.patch, 
> MAPREDUCE-5044.v03.patch, MAPREDUCE-5044.v04.patch, MAPREDUCE-5044.v05.patch, 
> MAPREDUCE-5044.v06.patch, MAPREDUCE-5044.v07.local.patch, Screen Shot 
> 2013-11-12 at 1.05.32 PM.png, Screen Shot 2013-11-12 at 1.06.04 PM.png
>
>
> When an AM expires a task attempt it would be nice if it triggered a jstack 
> output via SIGQUIT before killing the task attempt.  This would be invaluable 
> for helping users debug their hung tasks, especially if they do not have 
> shell access to the nodes.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org



[jira] [Commented] (MAPREDUCE-5044) Have AM trigger jstack on task attempts that timeout before killing them

2016-06-06 Thread Ming Ma (JIRA)

[ 
https://issues.apache.org/jira/browse/MAPREDUCE-5044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15317271#comment-15317271
 ] 

Ming Ma commented on MAPREDUCE-5044:


+1 on the latest patch. Thanks [~eepayne]. The patch doesn't resolve 
automatically for branch-2 and 2.8. It is straightforward and I will resolve it 
for those two branches.

> Have AM trigger jstack on task attempts that timeout before killing them
> 
>
> Key: MAPREDUCE-5044
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5044
> Project: Hadoop Map/Reduce
>  Issue Type: Improvement
>  Components: mr-am
>Affects Versions: 2.1.0-beta
>Reporter: Jason Lowe
>Assignee: Eric Payne
> Attachments: MAPREDUCE-5044.008.patch, MAPREDUCE-5044.009.patch, 
> MAPREDUCE-5044.010.patch, MAPREDUCE-5044.011.patch, MAPREDUCE-5044.012.patch, 
> MAPREDUCE-5044.013.patch, MAPREDUCE-5044.v01.patch, MAPREDUCE-5044.v02.patch, 
> MAPREDUCE-5044.v03.patch, MAPREDUCE-5044.v04.patch, MAPREDUCE-5044.v05.patch, 
> MAPREDUCE-5044.v06.patch, MAPREDUCE-5044.v07.local.patch, Screen Shot 
> 2013-11-12 at 1.05.32 PM.png, Screen Shot 2013-11-12 at 1.06.04 PM.png
>
>
> When an AM expires a task attempt it would be nice if it triggered a jstack 
> output via SIGQUIT before killing the task attempt.  This would be invaluable 
> for helping users debug their hung tasks, especially if they do not have 
> shell access to the nodes.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org



[jira] [Commented] (MAPREDUCE-6702) TestMiniMRChildTask.testTaskEnv and TestMiniMRChildTask.testTaskOldEnv are failing

2016-06-06 Thread Hadoop QA (JIRA)

[ 
https://issues.apache.org/jira/browse/MAPREDUCE-6702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15316768#comment-15316768
 ] 

Hadoop QA commented on MAPREDUCE-6702:
--

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 27s 
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s 
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 
0s {color} | {color:green} The patch appears to include 1 new or modified test 
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 13s 
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m 
42s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m 33s 
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m 
21s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 43s 
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 
35s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 
33s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 30s 
{color} | {color:green} trunk passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 11s 
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 
18s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m 32s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 6m 32s 
{color} | {color:green} root generated 0 new + 689 unchanged - 8 fixed = 689 
total (was 697) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m 23s 
{color} | {color:red} root: The patch generated 2 new + 129 unchanged - 32 
fixed = 131 total (was 161) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 47s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 
36s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 
56s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 30s 
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 21m 23s {color} 
| {color:red} hadoop-common in the patch failed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 58s 
{color} | {color:green} hadoop-mapreduce-client-core in the patch passed. 
{color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 102m 48s 
{color} | {color:red} hadoop-mapreduce-client-jobclient in the patch failed. 
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 
28s {color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 165m 25s {color} 
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.mapred.TestMRCJCFileOutputCommitter |
| Timed out junit tests | org.apache.hadoop.http.TestHttpServerLifecycle |
\\
\\
|| Subsystem || Report/Notes ||
| Docker |  Image:yetus/hadoop:2c91fd8 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12808390/MAPREDUCE-6702.01.patch
 |
| JIRA Issue | MAPREDUCE-6702 |
| Optional Tests |  asflicense  mvnsite  compile  javac  javadoc  mvninstall  
unit  findbugs  checkstyle  |
| uname | Linux e81903d4ac23 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed 
Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh 
|
| git revision | trunk / 35f255b |
| Default Java | 1.8.0_91 |
| findbugs | v3.0.0 |
| checkstyle | 

[jira] [Commented] (MAPREDUCE-5044) Have AM trigger jstack on task attempts that timeout before killing them

2016-06-06 Thread Eric Payne (JIRA)

[ 
https://issues.apache.org/jira/browse/MAPREDUCE-5044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15316754#comment-15316754
 ] 

Eric Payne commented on MAPREDUCE-5044:
---

- FindBugs warning is not related. It pertains to 
{{org.apache.hadoop.yarn.api.records.ResourceRequest}} / 
{{ResourceRequest.java:[line 361]}}, which was not changed by this patch.
- Checkstyle warnings are as I expected (see my comment, above).
- Unit test failures all pass in my local environment for {{TestLogsCLI}}, 
which intermittently fails both with and without this patch, and 
{{TestYarnClient}}, which fails consistently both with and without the patch.

> Have AM trigger jstack on task attempts that timeout before killing them
> 
>
> Key: MAPREDUCE-5044
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5044
> Project: Hadoop Map/Reduce
>  Issue Type: Improvement
>  Components: mr-am
>Affects Versions: 2.1.0-beta
>Reporter: Jason Lowe
>Assignee: Eric Payne
> Attachments: MAPREDUCE-5044.008.patch, MAPREDUCE-5044.009.patch, 
> MAPREDUCE-5044.010.patch, MAPREDUCE-5044.011.patch, MAPREDUCE-5044.012.patch, 
> MAPREDUCE-5044.013.patch, MAPREDUCE-5044.v01.patch, MAPREDUCE-5044.v02.patch, 
> MAPREDUCE-5044.v03.patch, MAPREDUCE-5044.v04.patch, MAPREDUCE-5044.v05.patch, 
> MAPREDUCE-5044.v06.patch, MAPREDUCE-5044.v07.local.patch, Screen Shot 
> 2013-11-12 at 1.05.32 PM.png, Screen Shot 2013-11-12 at 1.06.04 PM.png
>
>
> When an AM expires a task attempt it would be nice if it triggered a jstack 
> output via SIGQUIT before killing the task attempt.  This would be invaluable 
> for helping users debug their hung tasks, especially if they do not have 
> shell access to the nodes.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org



[jira] [Updated] (MAPREDUCE-6702) TestMiniMRChildTask.testTaskEnv and TestMiniMRChildTask.testTaskOldEnv are failing

2016-06-06 Thread Akira AJISAKA (JIRA)

 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-6702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Akira AJISAKA updated MAPREDUCE-6702:
-
Issue Type: Bug  (was: Test)

> TestMiniMRChildTask.testTaskEnv and TestMiniMRChildTask.testTaskOldEnv are 
> failing
> --
>
> Key: MAPREDUCE-6702
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6702
> Project: Hadoop Map/Reduce
>  Issue Type: Bug
>  Components: client
>Affects Versions: 3.0.0-alpha1
>Reporter: Daniel Templeton
>Assignee: Akira AJISAKA
> Attachments: MAPREDUCE-6702.01.patch
>
>
> ---
>  T E S T S
> ---
> Running org.apache.hadoop.mapred.TestMiniMRChildTask
> Tests run: 3, Failures: 2, Errors: 0, Skipped: 0, Time elapsed: 92.48 sec <<< 
> FAILURE! - in org.apache.hadoop.mapred.TestMiniMRChildTask
> testTaskEnv(org.apache.hadoop.mapred.TestMiniMRChildTask)  Time elapsed: 
> 21.906 sec  <<< FAILURE!
> java.lang.AssertionError: The environment checker job failed.
>   at org.junit.Assert.fail(Assert.java:88)
>   at org.junit.Assert.assertTrue(Assert.java:41)
>   at 
> org.apache.hadoop.mapred.TestMiniMRChildTask.runTestTaskEnv(TestMiniMRChildTask.java:550)
>   at 
> org.apache.hadoop.mapred.TestMiniMRChildTask.testTaskEnv(TestMiniMRChildTask.java:472)
> testTaskOldEnv(org.apache.hadoop.mapred.TestMiniMRChildTask)  Time elapsed: 
> 17.452 sec  <<< FAILURE!
> java.lang.AssertionError: The environment checker job failed.
>   at org.junit.Assert.fail(Assert.java:88)
>   at org.junit.Assert.assertTrue(Assert.java:41)
>   at 
> org.apache.hadoop.mapred.TestMiniMRChildTask.runTestTaskEnv(TestMiniMRChildTask.java:550)
>   at 
> org.apache.hadoop.mapred.TestMiniMRChildTask.testTaskOldEnv(TestMiniMRChildTask.java:496)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org



[jira] [Updated] (MAPREDUCE-6702) TestMiniMRChildTask.testTaskEnv and TestMiniMRChildTask.testTaskOldEnv are failing

2016-06-06 Thread Akira AJISAKA (JIRA)

 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-6702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Akira AJISAKA updated MAPREDUCE-6702:
-
Hadoop Flags: Incompatible change
Release Note: 

In Hadoop 3, YARN containers do not inherit the NodeManagers' environment 
variables. Therefore, if you want to inherit NodeManager's environment 
variables (e.g. `HADOOP_MAPRED_HOME`), you need to set additional parameters 
(e.g. `mapreduce.admin.user.env` and `yarn.app.mapreduce.am.env`).

> TestMiniMRChildTask.testTaskEnv and TestMiniMRChildTask.testTaskOldEnv are 
> failing
> --
>
> Key: MAPREDUCE-6702
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6702
> Project: Hadoop Map/Reduce
>  Issue Type: Test
>  Components: client
>Affects Versions: 3.0.0-alpha1
>Reporter: Daniel Templeton
>Assignee: Akira AJISAKA
> Attachments: MAPREDUCE-6702.01.patch
>
>
> ---
>  T E S T S
> ---
> Running org.apache.hadoop.mapred.TestMiniMRChildTask
> Tests run: 3, Failures: 2, Errors: 0, Skipped: 0, Time elapsed: 92.48 sec <<< 
> FAILURE! - in org.apache.hadoop.mapred.TestMiniMRChildTask
> testTaskEnv(org.apache.hadoop.mapred.TestMiniMRChildTask)  Time elapsed: 
> 21.906 sec  <<< FAILURE!
> java.lang.AssertionError: The environment checker job failed.
>   at org.junit.Assert.fail(Assert.java:88)
>   at org.junit.Assert.assertTrue(Assert.java:41)
>   at 
> org.apache.hadoop.mapred.TestMiniMRChildTask.runTestTaskEnv(TestMiniMRChildTask.java:550)
>   at 
> org.apache.hadoop.mapred.TestMiniMRChildTask.testTaskEnv(TestMiniMRChildTask.java:472)
> testTaskOldEnv(org.apache.hadoop.mapred.TestMiniMRChildTask)  Time elapsed: 
> 17.452 sec  <<< FAILURE!
> java.lang.AssertionError: The environment checker job failed.
>   at org.junit.Assert.fail(Assert.java:88)
>   at org.junit.Assert.assertTrue(Assert.java:41)
>   at 
> org.apache.hadoop.mapred.TestMiniMRChildTask.runTestTaskEnv(TestMiniMRChildTask.java:550)
>   at 
> org.apache.hadoop.mapred.TestMiniMRChildTask.testTaskOldEnv(TestMiniMRChildTask.java:496)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org



[jira] [Updated] (MAPREDUCE-6702) TestMiniMRChildTask.testTaskEnv and TestMiniMRChildTask.testTaskOldEnv are failing

2016-06-06 Thread Akira AJISAKA (JIRA)

 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-6702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Akira AJISAKA updated MAPREDUCE-6702:
-
Target Version/s: 3.0.0-alpha1
  Status: Patch Available  (was: Open)

> TestMiniMRChildTask.testTaskEnv and TestMiniMRChildTask.testTaskOldEnv are 
> failing
> --
>
> Key: MAPREDUCE-6702
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6702
> Project: Hadoop Map/Reduce
>  Issue Type: Test
>  Components: client
>Affects Versions: 3.0.0-alpha1
>Reporter: Daniel Templeton
>Assignee: Akira AJISAKA
> Attachments: MAPREDUCE-6702.01.patch
>
>
> ---
>  T E S T S
> ---
> Running org.apache.hadoop.mapred.TestMiniMRChildTask
> Tests run: 3, Failures: 2, Errors: 0, Skipped: 0, Time elapsed: 92.48 sec <<< 
> FAILURE! - in org.apache.hadoop.mapred.TestMiniMRChildTask
> testTaskEnv(org.apache.hadoop.mapred.TestMiniMRChildTask)  Time elapsed: 
> 21.906 sec  <<< FAILURE!
> java.lang.AssertionError: The environment checker job failed.
>   at org.junit.Assert.fail(Assert.java:88)
>   at org.junit.Assert.assertTrue(Assert.java:41)
>   at 
> org.apache.hadoop.mapred.TestMiniMRChildTask.runTestTaskEnv(TestMiniMRChildTask.java:550)
>   at 
> org.apache.hadoop.mapred.TestMiniMRChildTask.testTaskEnv(TestMiniMRChildTask.java:472)
> testTaskOldEnv(org.apache.hadoop.mapred.TestMiniMRChildTask)  Time elapsed: 
> 17.452 sec  <<< FAILURE!
> java.lang.AssertionError: The environment checker job failed.
>   at org.junit.Assert.fail(Assert.java:88)
>   at org.junit.Assert.assertTrue(Assert.java:41)
>   at 
> org.apache.hadoop.mapred.TestMiniMRChildTask.runTestTaskEnv(TestMiniMRChildTask.java:550)
>   at 
> org.apache.hadoop.mapred.TestMiniMRChildTask.testTaskOldEnv(TestMiniMRChildTask.java:496)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org



[jira] [Updated] (MAPREDUCE-6702) TestMiniMRChildTask.testTaskEnv and TestMiniMRChildTask.testTaskOldEnv are failing

2016-06-06 Thread Akira AJISAKA (JIRA)

 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-6702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Akira AJISAKA updated MAPREDUCE-6702:
-
Attachment: MAPREDUCE-6702.01.patch

Attaching a patch to fix test failures and documents.

> TestMiniMRChildTask.testTaskEnv and TestMiniMRChildTask.testTaskOldEnv are 
> failing
> --
>
> Key: MAPREDUCE-6702
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6702
> Project: Hadoop Map/Reduce
>  Issue Type: Test
>  Components: client
>Affects Versions: 3.0.0-alpha1
>Reporter: Daniel Templeton
>Assignee: Akira AJISAKA
> Attachments: MAPREDUCE-6702.01.patch
>
>
> ---
>  T E S T S
> ---
> Running org.apache.hadoop.mapred.TestMiniMRChildTask
> Tests run: 3, Failures: 2, Errors: 0, Skipped: 0, Time elapsed: 92.48 sec <<< 
> FAILURE! - in org.apache.hadoop.mapred.TestMiniMRChildTask
> testTaskEnv(org.apache.hadoop.mapred.TestMiniMRChildTask)  Time elapsed: 
> 21.906 sec  <<< FAILURE!
> java.lang.AssertionError: The environment checker job failed.
>   at org.junit.Assert.fail(Assert.java:88)
>   at org.junit.Assert.assertTrue(Assert.java:41)
>   at 
> org.apache.hadoop.mapred.TestMiniMRChildTask.runTestTaskEnv(TestMiniMRChildTask.java:550)
>   at 
> org.apache.hadoop.mapred.TestMiniMRChildTask.testTaskEnv(TestMiniMRChildTask.java:472)
> testTaskOldEnv(org.apache.hadoop.mapred.TestMiniMRChildTask)  Time elapsed: 
> 17.452 sec  <<< FAILURE!
> java.lang.AssertionError: The environment checker job failed.
>   at org.junit.Assert.fail(Assert.java:88)
>   at org.junit.Assert.assertTrue(Assert.java:41)
>   at 
> org.apache.hadoop.mapred.TestMiniMRChildTask.runTestTaskEnv(TestMiniMRChildTask.java:550)
>   at 
> org.apache.hadoop.mapred.TestMiniMRChildTask.testTaskOldEnv(TestMiniMRChildTask.java:496)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org



[jira] [Commented] (MAPREDUCE-6705) Task failing continuously on trunk

2016-06-06 Thread Bibin A Chundatt (JIRA)

[ 
https://issues.apache.org/jira/browse/MAPREDUCE-6705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15316402#comment-15316402
 ] 

Bibin A Chundatt commented on MAPREDUCE-6705:
-

[~drankye]
Since MAPREDUCE-6706 is available to track the {{ProtocolRPCEngine}} change, we 
can close this JIRA. Thank you for handling the revert.


> Task failing continuously on trunk
> --
>
> Key: MAPREDUCE-6705
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6705
> Project: Hadoop Map/Reduce
>  Issue Type: Bug
>Reporter: Bibin A Chundatt
>Assignee: Kai Zheng
>Priority: Blocker
>
> Task attempt failing continuously. Submit any mapreduce application
> Run the job as below
> {code}
> ./yarn jar ../share/hadoop/mapreduce/hadoop-mapreduce-examples*.jar pi 
> -Dyarn.app.mapreduce.am.env="HADOOP_MAPRED_HOME={{HADOOP_COMMON_HOME}}" 
> -Dmapreduce.admin.user.env="HADOOP_MAPRED_HOME={{HADOOP_COMMON_HOME}}"  1 1
> {code}
> {noformat}
> 2016-05-27 11:28:27,148 DEBUG [main] org.apache.hadoop.ipc.Client: getting 
> client out of cache: org.apache.hadoop.ipc.Client@291ae
> 2016-05-27 11:28:27,160 DEBUG [main] org.apache.hadoop.mapred.YarnChild: PID: 
> 22305
> 2016-05-27 11:28:27,160 INFO [main] org.apache.hadoop.mapred.YarnChild: 
> Sleeping for 0ms before retrying again. Got null now.
> 2016-05-27 11:28:27,161 WARN [main] org.apache.hadoop.mapred.YarnChild: 
> Exception running child : java.lang.reflect.UndeclaredThrowableException
>   at com.sun.proxy.$Proxy10.getTask(Unknown Source)
>   at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:136)
> Caused by: com.google.protobuf.ServiceException: Too many or few parameters 
> for request. Method: [getTask], Expected: 2, Actual: 1
>   at 
> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:206)
>   ... 2 more
> 2016-05-27 11:28:27,161 DEBUG [main] org.apache.hadoop.ipc.Client: stopping 
> client from cache: org.apache.hadoop.ipc.Client@291ae
> 2016-05-27 11:28:27,161 DEBUG [main] org.apache.hadoop.ipc.Client: removing 
> client from cache: org.apache.hadoop.ipc.Client@291ae
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org