date:20160517

[jira] [Assigned] (MAPREDUCE-6703) Add flag to allow MapReduce AM to request for OPPORTUNISTIC containers

2016-05-17 Thread Arun Suresh (JIRA)


 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-6703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Arun Suresh reassigned MAPREDUCE-6703:
--

Assignee: Arun Suresh

> Add flag to allow MapReduce AM to request for OPPORTUNISTIC containers
> --
>
> Key: MAPREDUCE-6703
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6703
> Project: Hadoop Map/Reduce
>  Issue Type: Bug
>Reporter: Arun Suresh
>Assignee: Arun Suresh
>
> YARN-2882 and YARN-4335 introduces the concept of container ExecutionTypes 
> and specifically OPPORTUNISTIC containers.
> The default ExecutionType is GUARANTEED. This JIRA proposes to allow users to 
> provide hints via config to the MR framework as to the number of containers 
> it would like to schedule as OPPORTUNISTIC.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org

[jira] [Comment Edited] (MAPREDUCE-6696) Add a configuration to limit the number of map tasks allowed per job.

2016-05-17 Thread zhihai xu (JIRA)


[ 
https://issues.apache.org/jira/browse/MAPREDUCE-6696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15288083#comment-15288083
 ] 

zhihai xu edited comment on MAPREDUCE-6696 at 5/18/16 2:23 AM:
---

Thanks for the review [~jianhe]! good finding yes, JobImpl#checkTaskLimits was 
the very initial code for the task limit, but this will happen at AM so it 
still will waste some resource (AM container). Yes, MRJobConfig.NUM_MAPS is 
giving a hint about but my patch is based on InputFormat.getSplits, which will 
exactly match the number of mappers of the MapReduce Job:
{code}
   LOG.debug("Creating splits at " + jtFs.makeQualified(submitJobDir));
  int maps = writeSplits(job, submitJobDir);
  conf.setInt(MRJobConfig.NUM_MAPS, maps);
  LOG.info("number of splits:" + maps);
{code} 
writeSplits will call InputFormat.getSplits and writeJobSplitMetaInfo to create 
file "job.splitmetainfo". JobImpl called by AM will read  file 
"job.splitmetainfo" by calling createSplits and readSplitMetaInfo to get input 
split info.
{code}
   /** 
   * Logically split the set of input files for the job.  
   * 
   * Each {@link InputSplit} is then assigned to an individual {@link Mapper}
   * for processing.
   *
   * Note: The split is a logical split of the inputs and the
   * input files are not physically split into chunks. For e.g. a split could
   * be input-file-path, start, offset tuple.
   * 
   * @param job job configuration.
   * @param numSplits the desired number of splits, a hint.
   * @return an array of {@link InputSplit}s for the job.
   */
  InputSplit[] getSplits(JobConf job, int numSplits) throws IOException;
{code}
My patch will reject the job during submission, which can save AM container 
resource.


was (Author: zxu):
Thanks for the review [~jianhe]! good finding yes, JobImpl#checkTaskLimits was 
the very initial code for the task limit, but this will happen at AM so it 
still will waste some resource (AM container). Yes, MRJobConfig.NUM_MAPS is 
giving a hint about but my patch is based on InputFormat.getSplits, which will 
exactly match the number of mappers of the MapReduce Job:
{code}
   LOG.debug("Creating splits at " + jtFs.makeQualified(submitJobDir));
  int maps = writeSplits(job, submitJobDir);
  conf.setInt(MRJobConfig.NUM_MAPS, maps);
  LOG.info("number of splits:" + maps);
{code} 
writeSplits will call InputFormat.getSplits. 
{code}
   /** 
   * Logically split the set of input files for the job.  
   * 
   * Each {@link InputSplit} is then assigned to an individual {@link Mapper}
   * for processing.
   *
   * Note: The split is a logical split of the inputs and the
   * input files are not physically split into chunks. For e.g. a split could
   * be input-file-path, start, offset tuple.
   * 
   * @param job job configuration.
   * @param numSplits the desired number of splits, a hint.
   * @return an array of {@link InputSplit}s for the job.
   */
  InputSplit[] getSplits(JobConf job, int numSplits) throws IOException;
{code}
My patch will reject the job during submission, which can save AM container 
resource.

> Add a configuration to limit the number of map tasks allowed per job.
> -
>
> Key: MAPREDUCE-6696
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6696
> Project: Hadoop Map/Reduce
>  Issue Type: Improvement
>  Components: job submission
>Affects Versions: 2.8.0
>Reporter: zhihai xu
>Assignee: zhihai xu
> Attachments: MAPREDUCE-6696.000.patch, MAPREDUCE-6696.001.patch, 
> MAPREDUCE-6696.002.patch
>
>
> Add a configuration "mapreduce.job.max.map" to limit the number of map tasks 
> allowed per job. It will be useful for Hadoop admin to save Hadoop cluster 
> resource by preventing users from submitting big mapreduce jobs. A mapredeuce 
> job with too many mappers may fail with OOM after running for long time. It 
> will be a big waste.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org

[jira] [Commented] (MAPREDUCE-6696) Add a configuration to limit the number of map tasks allowed per job.

2016-05-17 Thread zhihai xu (JIRA)


[ 
https://issues.apache.org/jira/browse/MAPREDUCE-6696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15288083#comment-15288083
 ] 

zhihai xu commented on MAPREDUCE-6696:
--

Thanks for the review [~jianhe]! good finding yes, JobImpl#checkTaskLimits was 
the very initial code for the task limit, but this will happen at AM so it 
still will waste some resource (AM container). Yes, MRJobConfig.NUM_MAPS is 
giving a hint about but my patch is based on InputFormat.getSplits, which will 
exactly match the number of mappers of the MapReduce Job:
{code}
   LOG.debug("Creating splits at " + jtFs.makeQualified(submitJobDir));
  int maps = writeSplits(job, submitJobDir);
  conf.setInt(MRJobConfig.NUM_MAPS, maps);
  LOG.info("number of splits:" + maps);
{code} 
writeSplits will call InputFormat.getSplits. 
{code}
   /** 
   * Logically split the set of input files for the job.  
   * 
   * Each {@link InputSplit} is then assigned to an individual {@link Mapper}
   * for processing.
   *
   * Note: The split is a logical split of the inputs and the
   * input files are not physically split into chunks. For e.g. a split could
   * be input-file-path, start, offset tuple.
   * 
   * @param job job configuration.
   * @param numSplits the desired number of splits, a hint.
   * @return an array of {@link InputSplit}s for the job.
   */
  InputSplit[] getSplits(JobConf job, int numSplits) throws IOException;
{code}
My patch will reject the job during submission, which can save AM container 
resource.

> Add a configuration to limit the number of map tasks allowed per job.
> -
>
> Key: MAPREDUCE-6696
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6696
> Project: Hadoop Map/Reduce
>  Issue Type: Improvement
>  Components: job submission
>Affects Versions: 2.8.0
>Reporter: zhihai xu
>Assignee: zhihai xu
> Attachments: MAPREDUCE-6696.000.patch, MAPREDUCE-6696.001.patch, 
> MAPREDUCE-6696.002.patch
>
>
> Add a configuration "mapreduce.job.max.map" to limit the number of map tasks 
> allowed per job. It will be useful for Hadoop admin to save Hadoop cluster 
> resource by preventing users from submitting big mapreduce jobs. A mapredeuce 
> job with too many mappers may fail with OOM after running for long time. It 
> will be a big waste.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org

[jira] [Commented] (MAPREDUCE-6686) Add a way to download the job config from the mapred CLI

2016-05-17 Thread Harsh J (JIRA)


[ 
https://issues.apache.org/jira/browse/MAPREDUCE-6686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15288067#comment-15288067
 ] 

Harsh J commented on MAPREDUCE-6686:


+1, thanks [~rkanter]

> Add a way to download the job config from the mapred CLI
> 
>
> Key: MAPREDUCE-6686
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6686
> Project: Hadoop Map/Reduce
>  Issue Type: Improvement
>  Components: client
>Affects Versions: 2.9.0
>Reporter: Robert Kanter
>Assignee: Robert Kanter
> Attachments: MAPREDUCE-6686.001.patch, MAPREDUCE-6686.002.patch
>
>
> It would be convenient if there was a way to easily grab the job 
> configuration via the CLI instead of having to find and go to the specific 
> HDFS location to grab it.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org

[jira] [Commented] (MAPREDUCE-6696) Add a configuration to limit the number of map tasks allowed per job.

2016-05-17 Thread Jian He (JIRA)


[ 
https://issues.apache.org/jira/browse/MAPREDUCE-6696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15287888#comment-15287888
 ] 

Jian He commented on MAPREDUCE-6696:


I think the MRJobConfig.NUM_MAPS is giving a hint about, not the actual, number 
of maps.
Btw, seems JobImpl#checkTaskLimits was the very initial code for the task 
limit. I guess it was removed when YARN got created based on git history

> Add a configuration to limit the number of map tasks allowed per job.
> -
>
> Key: MAPREDUCE-6696
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6696
> Project: Hadoop Map/Reduce
>  Issue Type: Improvement
>  Components: job submission
>Affects Versions: 2.8.0
>Reporter: zhihai xu
>Assignee: zhihai xu
> Attachments: MAPREDUCE-6696.000.patch, MAPREDUCE-6696.001.patch, 
> MAPREDUCE-6696.002.patch
>
>
> Add a configuration "mapreduce.job.max.map" to limit the number of map tasks 
> allowed per job. It will be useful for Hadoop admin to save Hadoop cluster 
> resource by preventing users from submitting big mapreduce jobs. A mapredeuce 
> job with too many mappers may fail with OOM after running for long time. It 
> will be a big waste.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org

[jira] [Created] (MAPREDUCE-6703) Add flag to allow MapReduce AM to request for OPPORTUNISTIC containers

2016-05-17 Thread Arun Suresh (JIRA)

Arun Suresh created MAPREDUCE-6703:
--

 Summary: Add flag to allow MapReduce AM to request for 
OPPORTUNISTIC containers
 Key: MAPREDUCE-6703
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6703
 Project: Hadoop Map/Reduce
  Issue Type: Bug
Reporter: Arun Suresh


YARN-2882 and YARN-4335 introduces the concept of container ExecutionTypes and 
specifically OPPORTUNISTIC containers.
The default ExecutionType is GUARANTEED. This JIRA proposes to allow users to 
provide hints via config to the MR framework as to the number of containers it 
would like to schedule as OPPORTUNISTIC.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org

[jira] [Commented] (MAPREDUCE-6696) Add a configuration to limit the number of map tasks allowed per job.

2016-05-17 Thread Hadoop QA (JIRA)


[ 
https://issues.apache.org/jira/browse/MAPREDUCE-6696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15287736#comment-15287736
 ] 

Hadoop QA commented on MAPREDUCE-6696:
--

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 12s 
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s 
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 
0s {color} | {color:green} The patch appears to include 1 new or modified test 
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 11s 
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 10m 
30s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 19s 
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 
40s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 13s 
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 
30s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 
35s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 48s 
{color} | {color:green} trunk passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 10s 
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 
3s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 18s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m 18s 
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 37s 
{color} | {color:red} hadoop-mapreduce-project/hadoop-mapreduce-client: patch 
generated 2 new + 539 unchanged - 1 fixed = 541 total (was 540) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 9s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 
25s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m 2s 
{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 
51s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 40s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m 37s 
{color} | {color:green} hadoop-mapreduce-client-core in the patch passed. 
{color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 98m 5s {color} 
| {color:red} hadoop-mapreduce-client-jobclient in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 
40s {color} | {color:green} Patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 128m 34s {color} 
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.mapreduce.v2.TestSpeculativeExecution |
|   | hadoop.mapreduce.security.ssl.TestEncryptedShuffle |
|   | hadoop.mapred.TestMiniMRChildTask |
|   | hadoop.mapreduce.v2.TestSpeculativeExecutionWithMRApp |
|   | hadoop.mapred.TestJobSysDirWithDFS |
|   | hadoop.mapred.TestMiniMRClientCluster |
|   | hadoop.mapred.TestClusterMRNotification |
|   | hadoop.mapred.TestMRTimelineEventHandling |
|   | hadoop.mapreduce.v2.TestMRJobsWithHistoryService |
|   | hadoop.mapreduce.security.TestBinaryTokenFile |
|   | hadoop.mapreduce.v2.TestNonExistentJob |
|   | hadoop.mapreduce.v2.TestMiniMRProxyUser |
|   | hadoop.mapred.TestNetworkedJob |
|   | hadoop.mapreduce.v2.TestMROldApiJobs |
|   | hadoop.mapred.TestClusterMapReduceTestCase |
|   | hadoop.mapred.TestReduceFetchFromPartialMem |
|   | hadoop.mapreduce.lib.output.TestJobOutputCommitter |
|   | hadoop.mapreduce.TestLargeSort |
|   |

[jira] [Updated] (MAPREDUCE-6702) TestMiniMRChildTask.testTaskEnv and TestMiniMRChildTask.testTaskOldEnv are failing

2016-05-17 Thread Daniel Templeton (JIRA)


 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-6702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Daniel Templeton updated MAPREDUCE-6702:

Description: 
---
 T E S T S
---
Running org.apache.hadoop.mapred.TestMiniMRChildTask
Tests run: 3, Failures: 2, Errors: 0, Skipped: 0, Time elapsed: 92.48 sec <<< 
FAILURE! - in org.apache.hadoop.mapred.TestMiniMRChildTask
testTaskEnv(org.apache.hadoop.mapred.TestMiniMRChildTask)  Time elapsed: 21.906 
sec  <<< FAILURE!
java.lang.AssertionError: The environment checker job failed.
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.assertTrue(Assert.java:41)
at 
org.apache.hadoop.mapred.TestMiniMRChildTask.runTestTaskEnv(TestMiniMRChildTask.java:550)
at 
org.apache.hadoop.mapred.TestMiniMRChildTask.testTaskEnv(TestMiniMRChildTask.java:472)

testTaskOldEnv(org.apache.hadoop.mapred.TestMiniMRChildTask)  Time elapsed: 
17.452 sec  <<< FAILURE!
java.lang.AssertionError: The environment checker job failed.
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.assertTrue(Assert.java:41)
at 
org.apache.hadoop.mapred.TestMiniMRChildTask.runTestTaskEnv(TestMiniMRChildTask.java:550)
at 
org.apache.hadoop.mapred.TestMiniMRChildTask.testTaskOldEnv(TestMiniMRChildTask.java:496)

> TestMiniMRChildTask.testTaskEnv and TestMiniMRChildTask.testTaskOldEnv are 
> failing
> --
>
> Key: MAPREDUCE-6702
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6702
> Project: Hadoop Map/Reduce
>  Issue Type: Test
>  Components: client
>Affects Versions: 3.0.0-alpha1
>Reporter: Daniel Templeton
>
> ---
>  T E S T S
> ---
> Running org.apache.hadoop.mapred.TestMiniMRChildTask
> Tests run: 3, Failures: 2, Errors: 0, Skipped: 0, Time elapsed: 92.48 sec <<< 
> FAILURE! - in org.apache.hadoop.mapred.TestMiniMRChildTask
> testTaskEnv(org.apache.hadoop.mapred.TestMiniMRChildTask)  Time elapsed: 
> 21.906 sec  <<< FAILURE!
> java.lang.AssertionError: The environment checker job failed.
>   at org.junit.Assert.fail(Assert.java:88)
>   at org.junit.Assert.assertTrue(Assert.java:41)
>   at 
> org.apache.hadoop.mapred.TestMiniMRChildTask.runTestTaskEnv(TestMiniMRChildTask.java:550)
>   at 
> org.apache.hadoop.mapred.TestMiniMRChildTask.testTaskEnv(TestMiniMRChildTask.java:472)
> testTaskOldEnv(org.apache.hadoop.mapred.TestMiniMRChildTask)  Time elapsed: 
> 17.452 sec  <<< FAILURE!
> java.lang.AssertionError: The environment checker job failed.
>   at org.junit.Assert.fail(Assert.java:88)
>   at org.junit.Assert.assertTrue(Assert.java:41)
>   at 
> org.apache.hadoop.mapred.TestMiniMRChildTask.runTestTaskEnv(TestMiniMRChildTask.java:550)
>   at 
> org.apache.hadoop.mapred.TestMiniMRChildTask.testTaskOldEnv(TestMiniMRChildTask.java:496)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org

[jira] [Created] (MAPREDUCE-6702) TestMiniMRChildTask.testTaskEnv and TestMiniMRChildTask.testTaskOldEnv are failing

2016-05-17 Thread Daniel Templeton (JIRA)

Daniel Templeton created MAPREDUCE-6702:
---

 Summary: TestMiniMRChildTask.testTaskEnv and 
TestMiniMRChildTask.testTaskOldEnv are failing
 Key: MAPREDUCE-6702
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6702
 Project: Hadoop Map/Reduce
  Issue Type: Test
  Components: client
Affects Versions: 3.0.0-alpha1
Reporter: Daniel Templeton






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org

[jira] [Commented] (MAPREDUCE-6657) job history server can fail on startup when NameNode is in start phase

2016-05-17 Thread Hudson (JIRA)


[ 
https://issues.apache.org/jira/browse/MAPREDUCE-6657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15287686#comment-15287686
 ] 

Hudson commented on MAPREDUCE-6657:
---

SUCCESS: Integrated in Hadoop-trunk-Commit #9807 (See 
[https://builds.apache.org/job/Hadoop-trunk-Commit/9807/])
MAPREDUCE-6657. Job history server can fail on startup when NameNode is 
(junping_du: rev f6ef876fe158a5334cad7075f1966573a1c4dec9)
* 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/pom.xml
* 
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeRpcServer.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/java/org/apache/hadoop/mapreduce/v2/hs/TestHistoryFileManagerInitWithNonRunningDFS.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/HistoryFileManager.java
* 
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java


> job history server can fail on startup when NameNode is in start phase
> --
>
> Key: MAPREDUCE-6657
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6657
> Project: Hadoop Map/Reduce
>  Issue Type: Bug
>  Components: jobhistoryserver
>Reporter: Haibo Chen
>Assignee: Haibo Chen
> Fix For: 2.9.0
>
> Attachments: mapreduce6657.001.patch, mapreduce6657.002.patch, 
> mapreduce6657.003.patch, mapreduce6657.004.patch, mapreduce6657.005.patch, 
> mapreduce6657.006.patch, mapreduce6657.007.patch
>
>
> Job history server will try to create a history directory in HDFS on startup. 
> When NameNode is in safe mode, it will keep retrying for a configurable time 
> period.  However, it should also keeps retrying if the name node is in start 
> state. Safe mode does not happen until the NN is out of the startup phase. 
> A RetriableException with the text "NameNode still not started" is thrown 
> when the NN is in its internal service startup phase. We should add the check 
> for this specific exception in isBecauseSafeMode() to account for that.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org

[jira] [Updated] (MAPREDUCE-6657) job history server can fail on startup when NameNode is in start phase

2016-05-17 Thread Junping Du (JIRA)


 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-6657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Junping Du updated MAPREDUCE-6657:
--
   Resolution: Fixed
 Hadoop Flags: Reviewed
Fix Version/s: 2.9.0
   Status: Resolved  (was: Patch Available)

I have commit the patch to trunk and branch-2. Thanks [~haibochen] for 
contributing the patch and [~templedf] and [~rkanter] for review and comments!

> job history server can fail on startup when NameNode is in start phase
> --
>
> Key: MAPREDUCE-6657
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6657
> Project: Hadoop Map/Reduce
>  Issue Type: Bug
>  Components: jobhistoryserver
>Reporter: Haibo Chen
>Assignee: Haibo Chen
> Fix For: 2.9.0
>
> Attachments: mapreduce6657.001.patch, mapreduce6657.002.patch, 
> mapreduce6657.003.patch, mapreduce6657.004.patch, mapreduce6657.005.patch, 
> mapreduce6657.006.patch, mapreduce6657.007.patch
>
>
> Job history server will try to create a history directory in HDFS on startup. 
> When NameNode is in safe mode, it will keep retrying for a configurable time 
> period.  However, it should also keeps retrying if the name node is in start 
> state. Safe mode does not happen until the NN is out of the startup phase. 
> A RetriableException with the text "NameNode still not started" is thrown 
> when the NN is in its internal service startup phase. We should add the check 
> for this specific exception in isBecauseSafeMode() to account for that.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org

[jira] [Updated] (MAPREDUCE-6696) Add a configuration to limit the number of map tasks allowed per job.

2016-05-17 Thread zhihai xu (JIRA)


 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-6696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

zhihai xu updated MAPREDUCE-6696:
-
Status: Open  (was: Patch Available)

> Add a configuration to limit the number of map tasks allowed per job.
> -
>
> Key: MAPREDUCE-6696
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6696
> Project: Hadoop Map/Reduce
>  Issue Type: Improvement
>  Components: job submission
>Affects Versions: 2.8.0
>Reporter: zhihai xu
>Assignee: zhihai xu
> Attachments: MAPREDUCE-6696.000.patch, MAPREDUCE-6696.001.patch, 
> MAPREDUCE-6696.002.patch
>
>
> Add a configuration "mapreduce.job.max.map" to limit the number of map tasks 
> allowed per job. It will be useful for Hadoop admin to save Hadoop cluster 
> resource by preventing users from submitting big mapreduce jobs. A mapredeuce 
> job with too many mappers may fail with OOM after running for long time. It 
> will be a big waste.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org

[jira] [Updated] (MAPREDUCE-6696) Add a configuration to limit the number of map tasks allowed per job.

2016-05-17 Thread zhihai xu (JIRA)


 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-6696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

zhihai xu updated MAPREDUCE-6696:
-
Status: Patch Available  (was: Open)

> Add a configuration to limit the number of map tasks allowed per job.
> -
>
> Key: MAPREDUCE-6696
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6696
> Project: Hadoop Map/Reduce
>  Issue Type: Improvement
>  Components: job submission
>Affects Versions: 2.8.0
>Reporter: zhihai xu
>Assignee: zhihai xu
> Attachments: MAPREDUCE-6696.000.patch, MAPREDUCE-6696.001.patch, 
> MAPREDUCE-6696.002.patch
>
>
> Add a configuration "mapreduce.job.max.map" to limit the number of map tasks 
> allowed per job. It will be useful for Hadoop admin to save Hadoop cluster 
> resource by preventing users from submitting big mapreduce jobs. A mapredeuce 
> job with too many mappers may fail with OOM after running for long time. It 
> will be a big waste.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org

[jira] [Comment Edited] (MAPREDUCE-6696) Add a configuration to limit the number of map tasks allowed per job.

2016-05-17 Thread zhihai xu (JIRA)


[ 
https://issues.apache.org/jira/browse/MAPREDUCE-6696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15287387#comment-15287387
 ] 

zhihai xu edited comment on MAPREDUCE-6696 at 5/17/16 7:50 PM:
---

All these test failures are not related to my changes:

TestMRCJCFileOutputCommitter is passed in my local build also the failure for 
TestMRCJCFileOutputCommitter is due to test environment problem:
{code}
2016-05-17 17:29:33,792 WARN  [main] util.NativeCodeLoader 
(NativeCodeLoader.java:(60)) - Unable to load native-hadoop library for 
your platform... using builtin-java classes where applicable.
{code}
TestMiniMRChildTask and TestMiniMRChildTask.testTaskOldEnv failure happened at 
launchContainer which already pass job submission phase after my code change.
{code}
2016-05-17 17:45:48,781 WARN  [ContainersLauncher #1] 
nodemanager.DefaultContainerExecutor 
(DefaultContainerExecutor.java:launchContainer(245)) - Exception from 
container-launch with container ID: container_1463507138005_0001_01_02 and 
exit code: 127
ExitCodeException exitCode=127: nice: bash: No such file or directory

at org.apache.hadoop.util.Shell.runCommand(Shell.java:946)
at org.apache.hadoop.util.Shell.run(Shell.java:850)
at 
org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:1144)
at 
org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:227)
at 
org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.launchContainer(ContainerLaunch.java:385)
at 
org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:281)
at 
org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:89)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
{code}


was (Author: zxu):
All these test failures are not related to my changes:

TestMRCJCFileOutputCommitter is passed in my local build also the failure for 
TestMRCJCFileOutputCommitter is due to test environment problem:
2016-05-17 17:29:33,792 WARN  [main] util.NativeCodeLoader 
(NativeCodeLoader.java:(60)) - Unable to load native-hadoop library for 
your platform... using builtin-java classes where applicable.

TestMiniMRChildTask and TestMiniMRChildTask.testTaskOldEnv failure happened at 
launchContainer which already pass job submission phase after my code change.
2016-05-17 17:45:48,781 WARN  [ContainersLauncher #1] 
nodemanager.DefaultContainerExecutor 
(DefaultContainerExecutor.java:launchContainer(245)) - Exception from 
container-launch with container ID: container_1463507138005_0001_01_02 and 
exit code: 127
ExitCodeException exitCode=127: nice: bash: No such file or directory

at org.apache.hadoop.util.Shell.runCommand(Shell.java:946)
at org.apache.hadoop.util.Shell.run(Shell.java:850)
at 
org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:1144)
at 
org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:227)
at 
org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.launchContainer(ContainerLaunch.java:385)
at 
org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:281)
at 
org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:89)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)


> Add a configuration to limit the number of map tasks allowed per job.
> -
>
> Key: MAPREDUCE-6696
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6696
> Project: Hadoop Map/Reduce
>  Issue Type: Improvement
>  Components: job submission
>Affects Versions: 2.8.0
>Reporter: zhihai xu
>Assignee: zhihai xu
> Attachments: MAPREDUCE-6696.000.patch, MAPREDUCE-6696.001.patch, 
> MAPREDUCE-6696.002.patch
>
>
> Add a configuration "mapreduce.job.max.map" to limit the number of map tasks 
> allowed per job. It will be useful for Hadoop admin to save Hadoop cluster 
> resource by preventing users from submitting big mapreduce jobs. A mapredeuce 
>

[jira] [Updated] (MAPREDUCE-6696) Add a configuration to limit the number of map tasks allowed per job.

2016-05-17 Thread zhihai xu (JIRA)


 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-6696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

zhihai xu updated MAPREDUCE-6696:
-
Attachment: MAPREDUCE-6696.002.patch

> Add a configuration to limit the number of map tasks allowed per job.
> -
>
> Key: MAPREDUCE-6696
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6696
> Project: Hadoop Map/Reduce
>  Issue Type: Improvement
>  Components: job submission
>Affects Versions: 2.8.0
>Reporter: zhihai xu
>Assignee: zhihai xu
> Attachments: MAPREDUCE-6696.000.patch, MAPREDUCE-6696.001.patch, 
> MAPREDUCE-6696.002.patch
>
>
> Add a configuration "mapreduce.job.max.map" to limit the number of map tasks 
> allowed per job. It will be useful for Hadoop admin to save Hadoop cluster 
> resource by preventing users from submitting big mapreduce jobs. A mapredeuce 
> job with too many mappers may fail with OOM after running for long time. It 
> will be a big waste.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org

[jira] [Commented] (MAPREDUCE-6696) Add a configuration to limit the number of map tasks allowed per job.

2016-05-17 Thread zhihai xu (JIRA)


[ 
https://issues.apache.org/jira/browse/MAPREDUCE-6696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15287387#comment-15287387
 ] 

zhihai xu commented on MAPREDUCE-6696:
--

All these test failures are not related to my changes:

TestMRCJCFileOutputCommitter is passed in my local build also the failure for 
TestMRCJCFileOutputCommitter is due to test environment problem:
2016-05-17 17:29:33,792 WARN  [main] util.NativeCodeLoader 
(NativeCodeLoader.java:(60)) - Unable to load native-hadoop library for 
your platform... using builtin-java classes where applicable.

TestMiniMRChildTask and TestMiniMRChildTask.testTaskOldEnv failure happened at 
launchContainer which already pass job submission phase after my code change.
2016-05-17 17:45:48,781 WARN  [ContainersLauncher #1] 
nodemanager.DefaultContainerExecutor 
(DefaultContainerExecutor.java:launchContainer(245)) - Exception from 
container-launch with container ID: container_1463507138005_0001_01_02 and 
exit code: 127
ExitCodeException exitCode=127: nice: bash: No such file or directory

at org.apache.hadoop.util.Shell.runCommand(Shell.java:946)
at org.apache.hadoop.util.Shell.run(Shell.java:850)
at 
org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:1144)
at 
org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:227)
at 
org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.launchContainer(ContainerLaunch.java:385)
at 
org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:281)
at 
org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:89)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)


> Add a configuration to limit the number of map tasks allowed per job.
> -
>
> Key: MAPREDUCE-6696
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6696
> Project: Hadoop Map/Reduce
>  Issue Type: Improvement
>  Components: job submission
>Affects Versions: 2.8.0
>Reporter: zhihai xu
>Assignee: zhihai xu
> Attachments: MAPREDUCE-6696.000.patch, MAPREDUCE-6696.001.patch
>
>
> Add a configuration "mapreduce.job.max.map" to limit the number of map tasks 
> allowed per job. It will be useful for Hadoop admin to save Hadoop cluster 
> resource by preventing users from submitting big mapreduce jobs. A mapredeuce 
> job with too many mappers may fail with OOM after running for long time. It 
> will be a big waste.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org

[jira] [Assigned] (MAPREDUCE-6682) TestMRCJCFileOutputCommitter fais intermittently

2016-05-17 Thread Haibo Chen (JIRA)


 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-6682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Haibo Chen reassigned MAPREDUCE-6682:
-

Assignee: Haibo Chen

> TestMRCJCFileOutputCommitter fais intermittently
> 
>
> Key: MAPREDUCE-6682
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6682
> Project: Hadoop Map/Reduce
>  Issue Type: Bug
>  Components: test
>Reporter: Brahma Reddy Battula
>Assignee: Haibo Chen
>
> {noformat}
> java.lang.AssertionError: Output directory not empty expected:<0> but was:<4>
>   at org.junit.Assert.fail(Assert.java:88)
>   at org.junit.Assert.failNotEquals(Assert.java:743)
>   at org.junit.Assert.assertEquals(Assert.java:118)
>   at org.junit.Assert.assertEquals(Assert.java:555)
>   at 
> org.apache.hadoop.mapred.TestMRCJCFileOutputCommitter.testAbort(TestMRCJCFileOutputCommitter.java:153)
> {noformat}
> *PreCommit Report* 
> https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6434/testReport/



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org

[jira] [Assigned] (MAPREDUCE-6669) Jobs with encrypted spills don't tolerate AM failures

2016-05-17 Thread Haibo Chen (JIRA)


 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-6669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Haibo Chen reassigned MAPREDUCE-6669:
-

Assignee: Haibo Chen

> Jobs with encrypted spills don't tolerate AM failures
> -
>
> Key: MAPREDUCE-6669
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6669
> Project: Hadoop Map/Reduce
>  Issue Type: Bug
>  Components: mr-am
>Affects Versions: 2.8.0
>Reporter: Karthik Kambatla
>Assignee: Haibo Chen
>Priority: Critical
>
> The key used for encrypting intermediate data is not persisted anywhere, and 
> hence can't be recovered the same way other MR jobs can be. We should support 
> recovering these jobs as well, hopefully without having to re-run completed 
> tasks.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org

[jira] [Commented] (MAPREDUCE-6441) LocalDistributedCacheManager for concurrent sqoop processes fails to create unique directories

2016-05-17 Thread Ray Chiang (JIRA)


[ 
https://issues.apache.org/jira/browse/MAPREDUCE-6441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15287160#comment-15287160
 ] 

Ray Chiang commented on MAPREDUCE-6441:
---

[~wattsinabox] let me know if you can fix the patch for trunk.  Thanks.

> LocalDistributedCacheManager for concurrent sqoop processes fails to create 
> unique directories
> --
>
> Key: MAPREDUCE-6441
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6441
> Project: Hadoop Map/Reduce
>  Issue Type: Bug
>Reporter: William Watson
>Assignee: William Watson
> Attachments: HADOOP-10924.02.patch, 
> HADOOP-10924.03.jobid-plus-uuid.patch
>
>
> Kicking off many sqoop processes in different threads results in:
> {code}
> 2014-08-01 13:47:24 -0400:  INFO - 14/08/01 13:47:22 ERROR tool.ImportTool: 
> Encountered IOException running import job: java.io.IOException: 
> java.util.concurrent.ExecutionException: java.io.IOException: Rename cannot 
> overwrite non empty destination directory 
> /tmp/hadoop-hadoop/mapred/local/1406915233073
> 2014-08-01 13:47:24 -0400:  INFO -at 
> org.apache.hadoop.mapred.LocalDistributedCacheManager.setup(LocalDistributedCacheManager.java:149)
> 2014-08-01 13:47:24 -0400:  INFO -at 
> org.apache.hadoop.mapred.LocalJobRunner$Job.(LocalJobRunner.java:163)
> 2014-08-01 13:47:24 -0400:  INFO -at 
> org.apache.hadoop.mapred.LocalJobRunner.submitJob(LocalJobRunner.java:731)
> 2014-08-01 13:47:24 -0400:  INFO -at 
> org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:432)
> 2014-08-01 13:47:24 -0400:  INFO -at 
> org.apache.hadoop.mapreduce.Job$10.run(Job.java:1285)
> 2014-08-01 13:47:24 -0400:  INFO -at 
> org.apache.hadoop.mapreduce.Job$10.run(Job.java:1282)
> 2014-08-01 13:47:24 -0400:  INFO -at 
> java.security.AccessController.doPrivileged(Native Method)
> 2014-08-01 13:47:24 -0400:  INFO -at 
> javax.security.auth.Subject.doAs(Subject.java:415)
> 2014-08-01 13:47:24 -0400:  INFO -at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
> 2014-08-01 13:47:24 -0400:  INFO -at 
> org.apache.hadoop.mapreduce.Job.submit(Job.java:1282)
> 2014-08-01 13:47:24 -0400:  INFO -at 
> org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1303)
> 2014-08-01 13:47:24 -0400:  INFO -at 
> org.apache.sqoop.mapreduce.ImportJobBase.doSubmitJob(ImportJobBase.java:186)
> 2014-08-01 13:47:24 -0400:  INFO -at 
> org.apache.sqoop.mapreduce.ImportJobBase.runJob(ImportJobBase.java:159)
> 2014-08-01 13:47:24 -0400:  INFO -at 
> org.apache.sqoop.mapreduce.ImportJobBase.runImport(ImportJobBase.java:239)
> 2014-08-01 13:47:24 -0400:  INFO -at 
> org.apache.sqoop.manager.SqlManager.importQuery(SqlManager.java:645)
> 2014-08-01 13:47:24 -0400:  INFO -at 
> org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:415)
> 2014-08-01 13:47:24 -0400:  INFO -at 
> org.apache.sqoop.tool.ImportTool.run(ImportTool.java:502)
> 2014-08-01 13:47:24 -0400:  INFO -at 
> org.apache.sqoop.Sqoop.run(Sqoop.java:145)
> 2014-08-01 13:47:24 -0400:  INFO -at 
> org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
> 2014-08-01 13:47:24 -0400:  INFO -at 
> org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:181)
> 2014-08-01 13:47:24 -0400:  INFO -at 
> org.apache.sqoop.Sqoop.runTool(Sqoop.java:220)
> 2014-08-01 13:47:24 -0400:  INFO -at 
> org.apache.sqoop.Sqoop.runTool(Sqoop.java:229)
> 2014-08-01 13:47:24 -0400:  INFO -at 
> org.apache.sqoop.Sqoop.main(Sqoop.java:238)
> {code}
> If two are kicked off in the same second. The issue is the following lines of 
> code in the org.apache.hadoop.mapred.LocalDistributedCacheManager class: 
> {code}
> // Generating unique numbers for FSDownload.
> AtomicLong uniqueNumberGenerator =
>new AtomicLong(System.currentTimeMillis());
> {code}
> and 
> {code}
> Long.toString(uniqueNumberGenerator.incrementAndGet())),
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org

[jira] [Commented] (MAPREDUCE-6696) Add a configuration to limit the number of map tasks allowed per job.

2016-05-17 Thread Hadoop QA (JIRA)


[ 
https://issues.apache.org/jira/browse/MAPREDUCE-6696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15287104#comment-15287104
 ] 

Hadoop QA commented on MAPREDUCE-6696:
--

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 11s 
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s 
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 
0s {color} | {color:green} The patch appears to include 1 new or modified test 
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 7s 
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m 
58s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 28s 
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 
30s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 51s 
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 
22s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 5s 
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 33s 
{color} | {color:green} trunk passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 7s 
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 
39s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 27s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 27s 
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 28s 
{color} | {color:red} hadoop-mapreduce-project/hadoop-mapreduce-client: patch 
generated 2 new + 539 unchanged - 1 fixed = 541 total (was 540) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 46s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 
18s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m 1s 
{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 
15s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 29s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 55s 
{color} | {color:green} hadoop-mapreduce-client-core in the patch passed. 
{color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 102m 26s 
{color} | {color:red} hadoop-mapreduce-client-jobclient in the patch failed. 
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 
24s {color} | {color:green} Patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 122m 2s {color} 
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.mapred.TestMRCJCFileOutputCommitter |
|   | hadoop.mapred.TestMiniMRChildTask |
\\
\\
|| Subsystem || Report/Notes ||
| Docker |  Image:yetus/hadoop:2c91fd8 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12804448/MAPREDUCE-6696.001.patch
 |
| JIRA Issue | MAPREDUCE-6696 |
| Optional Tests |  asflicense  compile  javac  javadoc  mvninstall  mvnsite  
unit  findbugs  checkstyle  xml  |
| uname | Linux 7ff4c2a0c4af 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed 
Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh 
|
| git revision | trunk / 34fddd1 |
| Default Java | 1.8.0_91 |
| findbugs | v3.0.0 |
| checkstyle |

[jira] [Commented] (MAPREDUCE-6700) Jobhistory server attempt and task table not loading maps/reduce

2016-05-17 Thread Rohith Sharma K S (JIRA)


[ 
https://issues.apache.org/jira/browse/MAPREDUCE-6700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15286980#comment-15286980
 ] 

Rohith Sharma K S commented on MAPREDUCE-6700:
--

ah.. I tried to cherry-pick which could not happened. Let me try applying patch 
directly.

> Jobhistory server attempt and task table not loading maps/reduce
> 
>
> Key: MAPREDUCE-6700
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6700
> Project: Hadoop Map/Reduce
>  Issue Type: Bug
>Reporter: Bibin A Chundatt
>Priority: Blocker
>
> Browser
> ===
> Chrome
> Steps to reproduce
> ==
> # Submit mapreduce application with 20 maps
> # Wait till completion of mapreduce application
> # Check maps attempts page
> {{jobhistory/attempts/job_1463446678437_0003/m/SUCCESSFUL}}
> and {{jobhistory/tasks/job_1463446678437_0003/m}} page
> Actual
> =
> Table not loading.
> Sort based on any column other than attempt contents are loaded.
> Column 0 is of *natural sorting* and not working. So waiting for ever to be 
> sorted.
> {noformat}
> SCRIPT438: Object doesn't support property or method 'natural-asc' 
> jquery.dataTables.min.js, line 86 character 179
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org

[jira] [Commented] (MAPREDUCE-6700) Jobhistory server attempt and task table not loading maps/reduce

2016-05-17 Thread Varun Saxena (JIRA)


[ 
https://issues.apache.org/jira/browse/MAPREDUCE-6700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15286963#comment-15286963
 ] 

Varun Saxena commented on MAPREDUCE-6700:
-

The latest patch I had given on YARN-3840 was YARN-3840.reopened.001.patch
It applies cleanly on branch-2.8

> Jobhistory server attempt and task table not loading maps/reduce
> 
>
> Key: MAPREDUCE-6700
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6700
> Project: Hadoop Map/Reduce
>  Issue Type: Bug
>Reporter: Bibin A Chundatt
>Priority: Blocker
>
> Browser
> ===
> Chrome
> Steps to reproduce
> ==
> # Submit mapreduce application with 20 maps
> # Wait till completion of mapreduce application
> # Check maps attempts page
> {{jobhistory/attempts/job_1463446678437_0003/m/SUCCESSFUL}}
> and {{jobhistory/tasks/job_1463446678437_0003/m}} page
> Actual
> =
> Table not loading.
> Sort based on any column other than attempt contents are loaded.
> Column 0 is of *natural sorting* and not working. So waiting for ever to be 
> sorted.
> {noformat}
> SCRIPT438: Object doesn't support property or method 'natural-asc' 
> jquery.dataTables.min.js, line 86 character 179
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org

[jira] [Commented] (MAPREDUCE-6700) Jobhistory server attempt and task table not loading maps/reduce

2016-05-17 Thread Varun Saxena (JIRA)


[ 
https://issues.apache.org/jira/browse/MAPREDUCE-6700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15286930#comment-15286930
 ] 

Varun Saxena commented on MAPREDUCE-6700:
-

Not able to backport YARN-3840 ?

> Jobhistory server attempt and task table not loading maps/reduce
> 
>
> Key: MAPREDUCE-6700
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6700
> Project: Hadoop Map/Reduce
>  Issue Type: Bug
>Reporter: Bibin A Chundatt
>Priority: Blocker
>
> Browser
> ===
> Chrome
> Steps to reproduce
> ==
> # Submit mapreduce application with 20 maps
> # Wait till completion of mapreduce application
> # Check maps attempts page
> {{jobhistory/attempts/job_1463446678437_0003/m/SUCCESSFUL}}
> and {{jobhistory/tasks/job_1463446678437_0003/m}} page
> Actual
> =
> Table not loading.
> Sort based on any column other than attempt contents are loaded.
> Column 0 is of *natural sorting* and not working. So waiting for ever to be 
> sorted.
> {noformat}
> SCRIPT438: Object doesn't support property or method 'natural-asc' 
> jquery.dataTables.min.js, line 86 character 179
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org

[jira] [Commented] (MAPREDUCE-6700) Jobhistory server attempt and task table not loading maps/reduce

2016-05-17 Thread Rohith Sharma K S (JIRA)


[ 
https://issues.apache.org/jira/browse/MAPREDUCE-6700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15286925#comment-15286925
 ] 

Rohith Sharma K S commented on MAPREDUCE-6700:
--

It is not able to back port to branch-2.8 now since many commits have been 
done. [~bibinchundatt] would you mind providing rebase patch to branch-2.8 
please?

> Jobhistory server attempt and task table not loading maps/reduce
> 
>
> Key: MAPREDUCE-6700
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6700
> Project: Hadoop Map/Reduce
>  Issue Type: Bug
>Reporter: Bibin A Chundatt
>Priority: Blocker
>
> Browser
> ===
> Chrome
> Steps to reproduce
> ==
> # Submit mapreduce application with 20 maps
> # Wait till completion of mapreduce application
> # Check maps attempts page
> {{jobhistory/attempts/job_1463446678437_0003/m/SUCCESSFUL}}
> and {{jobhistory/tasks/job_1463446678437_0003/m}} page
> Actual
> =
> Table not loading.
> Sort based on any column other than attempt contents are loaded.
> Column 0 is of *natural sorting* and not working. So waiting for ever to be 
> sorted.
> {noformat}
> SCRIPT438: Object doesn't support property or method 'natural-asc' 
> jquery.dataTables.min.js, line 86 character 179
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org

[jira] [Commented] (MAPREDUCE-6700) Jobhistory server attempt and task table not loading maps/reduce

2016-05-17 Thread Rohith Sharma K S (JIRA)


[ 
https://issues.apache.org/jira/browse/MAPREDUCE-6700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15286915#comment-15286915
 ] 

Rohith Sharma K S commented on MAPREDUCE-6700:
--

IIRC, During December month branch-2.8 was created. When first commit of 
YARN-3840 happened i.e 10Nov branch-2 was pointing out to branch-2.8-snapshot 
and got reverted. But later in month first week of Dec, 2015 branch-2.8 got 
created. I think since newly created branch-2.8 was new, it got missed 
unknowingly. This would be the reason for missing commit in branch-2.8

Anyway I will try to backport YARN-3840 to branch-2.8. If any rebase is to be 
done, then this JIRA can be moved to YARN project for uploading branch-2.8 patch

> Jobhistory server attempt and task table not loading maps/reduce
> 
>
> Key: MAPREDUCE-6700
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6700
> Project: Hadoop Map/Reduce
>  Issue Type: Bug
>Reporter: Bibin A Chundatt
>Priority: Blocker
>
> Browser
> ===
> Chrome
> Steps to reproduce
> ==
> # Submit mapreduce application with 20 maps
> # Wait till completion of mapreduce application
> # Check maps attempts page
> {{jobhistory/attempts/job_1463446678437_0003/m/SUCCESSFUL}}
> and {{jobhistory/tasks/job_1463446678437_0003/m}} page
> Actual
> =
> Table not loading.
> Sort based on any column other than attempt contents are loaded.
> Column 0 is of *natural sorting* and not working. So waiting for ever to be 
> sorted.
> {noformat}
> SCRIPT438: Object doesn't support property or method 'natural-asc' 
> jquery.dataTables.min.js, line 86 character 179
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org

[jira] [Commented] (MAPREDUCE-6700) Jobhistory server attempt and task table not loading maps/reduce

2016-05-17 Thread Rohith Sharma K S (JIRA)


[ 
https://issues.apache.org/jira/browse/MAPREDUCE-6700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15286885#comment-15286885
 ] 

Rohith Sharma K S commented on MAPREDUCE-6700:
--

You are right, it got reverted!! I am not sure why it was reverted, I wait for 
[~jianhe]

> Jobhistory server attempt and task table not loading maps/reduce
> 
>
> Key: MAPREDUCE-6700
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6700
> Project: Hadoop Map/Reduce
>  Issue Type: Bug
>Reporter: Bibin A Chundatt
>Priority: Blocker
>
> Browser
> ===
> Chrome
> Steps to reproduce
> ==
> # Submit mapreduce application with 20 maps
> # Wait till completion of mapreduce application
> # Check maps attempts page
> {{jobhistory/attempts/job_1463446678437_0003/m/SUCCESSFUL}}
> and {{jobhistory/tasks/job_1463446678437_0003/m}} page
> Actual
> =
> Table not loading.
> Sort based on any column other than attempt contents are loaded.
> Column 0 is of *natural sorting* and not working. So waiting for ever to be 
> sorted.
> {noformat}
> SCRIPT438: Object doesn't support property or method 'natural-asc' 
> jquery.dataTables.min.js, line 86 character 179
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org

[jira] [Commented] (MAPREDUCE-6700) Jobhistory server attempt and task table not loading maps/reduce

2016-05-17 Thread Varun Saxena (JIRA)


[ 
https://issues.apache.org/jira/browse/MAPREDUCE-6700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15286878#comment-15286878
 ] 

Varun Saxena commented on MAPREDUCE-6700:
-

I mean MAPREDUCE-6419 should be reverted from branch-2.8

> Jobhistory server attempt and task table not loading maps/reduce
> 
>
> Key: MAPREDUCE-6700
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6700
> Project: Hadoop Map/Reduce
>  Issue Type: Bug
>Reporter: Bibin A Chundatt
>Priority: Blocker
>
> Browser
> ===
> Chrome
> Steps to reproduce
> ==
> # Submit mapreduce application with 20 maps
> # Wait till completion of mapreduce application
> # Check maps attempts page
> {{jobhistory/attempts/job_1463446678437_0003/m/SUCCESSFUL}}
> and {{jobhistory/tasks/job_1463446678437_0003/m}} page
> Actual
> =
> Table not loading.
> Sort based on any column other than attempt contents are loaded.
> Column 0 is of *natural sorting* and not working. So waiting for ever to be 
> sorted.
> {noformat}
> SCRIPT438: Object doesn't support property or method 'natural-asc' 
> jquery.dataTables.min.js, line 86 character 179
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org

[jira] [Commented] (MAPREDUCE-6700) Jobhistory server attempt and task table not loading maps/reduce

2016-05-17 Thread Varun Saxena (JIRA)


[ 
https://issues.apache.org/jira/browse/MAPREDUCE-6700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15286874#comment-15286874
 ] 

Varun Saxena commented on MAPREDUCE-6700:
-

Refer to history :
https://github.com/apache/hadoop/commits/branch-2.8/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/webapp/view/JQueryUI.java

> Jobhistory server attempt and task table not loading maps/reduce
> 
>
> Key: MAPREDUCE-6700
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6700
> Project: Hadoop Map/Reduce
>  Issue Type: Bug
>Reporter: Bibin A Chundatt
>Priority: Blocker
>
> Browser
> ===
> Chrome
> Steps to reproduce
> ==
> # Submit mapreduce application with 20 maps
> # Wait till completion of mapreduce application
> # Check maps attempts page
> {{jobhistory/attempts/job_1463446678437_0003/m/SUCCESSFUL}}
> and {{jobhistory/tasks/job_1463446678437_0003/m}} page
> Actual
> =
> Table not loading.
> Sort based on any column other than attempt contents are loaded.
> Column 0 is of *natural sorting* and not working. So waiting for ever to be 
> sorted.
> {noformat}
> SCRIPT438: Object doesn't support property or method 'natural-asc' 
> jquery.dataTables.min.js, line 86 character 179
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org

[jira] [Commented] (MAPREDUCE-6700) Jobhistory server attempt and task table not loading maps/reduce

2016-05-17 Thread Varun Saxena (JIRA)


[ 
https://issues.apache.org/jira/browse/MAPREDUCE-6700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15286873#comment-15286873
 ] 

Varun Saxena commented on MAPREDUCE-6700:
-

It seems YARN-3840 was reverted from branch-2.8
This means MAPREDUCE-6419 should be reverted too.as it depends on YARN-3840.
This is why [~bibinchundatt] was getting natural js related error.

cc [~jianhe], [~rohithsharma]

> Jobhistory server attempt and task table not loading maps/reduce
> 
>
> Key: MAPREDUCE-6700
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6700
> Project: Hadoop Map/Reduce
>  Issue Type: Bug
>Reporter: Bibin A Chundatt
>Priority: Blocker
>
> Browser
> ===
> Chrome
> Steps to reproduce
> ==
> # Submit mapreduce application with 20 maps
> # Wait till completion of mapreduce application
> # Check maps attempts page
> {{jobhistory/attempts/job_1463446678437_0003/m/SUCCESSFUL}}
> and {{jobhistory/tasks/job_1463446678437_0003/m}} page
> Actual
> =
> Table not loading.
> Sort based on any column other than attempt contents are loaded.
> Column 0 is of *natural sorting* and not working. So waiting for ever to be 
> sorted.
> {noformat}
> SCRIPT438: Object doesn't support property or method 'natural-asc' 
> jquery.dataTables.min.js, line 86 character 179
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org

[jira] [Updated] (MAPREDUCE-6696) Add a configuration to limit the number of map tasks allowed per job.

2016-05-17 Thread zhihai xu (JIRA)


 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-6696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

zhihai xu updated MAPREDUCE-6696:
-
Attachment: MAPREDUCE-6696.001.patch

> Add a configuration to limit the number of map tasks allowed per job.
> -
>
> Key: MAPREDUCE-6696
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6696
> Project: Hadoop Map/Reduce
>  Issue Type: Improvement
>  Components: job submission
>Affects Versions: 2.8.0
>Reporter: zhihai xu
>Assignee: zhihai xu
> Attachments: MAPREDUCE-6696.000.patch, MAPREDUCE-6696.001.patch
>
>
> Add a configuration "mapreduce.job.max.map" to limit the number of map tasks 
> allowed per job. It will be useful for Hadoop admin to save Hadoop cluster 
> resource by preventing users from submitting big mapreduce jobs. A mapredeuce 
> job with too many mappers may fail with OOM after running for long time. It 
> will be a big waste.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org

[jira] [Updated] (MAPREDUCE-6696) Add a configuration to limit the number of map tasks allowed per job.

2016-05-17 Thread zhihai xu (JIRA)


 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-6696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

zhihai xu updated MAPREDUCE-6696:
-
Attachment: (was: MAPREDUCE-6696.001.patch)

> Add a configuration to limit the number of map tasks allowed per job.
> -
>
> Key: MAPREDUCE-6696
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6696
> Project: Hadoop Map/Reduce
>  Issue Type: Improvement
>  Components: job submission
>Affects Versions: 2.8.0
>Reporter: zhihai xu
>Assignee: zhihai xu
> Attachments: MAPREDUCE-6696.000.patch
>
>
> Add a configuration "mapreduce.job.max.map" to limit the number of map tasks 
> allowed per job. It will be useful for Hadoop admin to save Hadoop cluster 
> resource by preventing users from submitting big mapreduce jobs. A mapredeuce 
> job with too many mappers may fail with OOM after running for long time. It 
> will be a big waste.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org

[jira] [Commented] (MAPREDUCE-6701) application master log can not be available when clicking jobhistory's am logs link

2016-05-17 Thread Hudson (JIRA)


[ 
https://issues.apache.org/jira/browse/MAPREDUCE-6701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15286869#comment-15286869
 ] 

Hudson commented on MAPREDUCE-6701:
---

ABORTED: Integrated in Hadoop-trunk-Commit #9793 (See 
[https://builds.apache.org/job/Hadoop-trunk-Commit/9793/])
MAPREDUCE-6701. application master log can not be available when (jlowe: rev 
12fa4ec141f8d0cb105b6f60c3739a4b26890fd5)
* 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/HsJobBlock.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/java/org/apache/hadoop/mapreduce/v2/hs/webapp/TestHsWebServicesJobs.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/dao/AMAttemptInfo.java


> application master log can not be available when clicking jobhistory's am 
> logs link
> ---
>
> Key: MAPREDUCE-6701
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6701
> Project: Hadoop Map/Reduce
>  Issue Type: Bug
>  Components: jobhistoryserver
>Affects Versions: 2.9.0
>Reporter: chenyukang
>Assignee: Haibo Chen
>Priority: Critical
> Fix For: 2.9.0
>
> Attachments: yarn5041.001.patch, yarn5041.002.patch
>
>
> In history server webapp, application master logs link is wrong. it shows "No 
> logs available for container container_1462419429440_0003_01_01".  It 
> direct to a wrong nodemanager http port instead of a node manager' container 
> managerment port. I think YARN-4701 brought this bug



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org

[jira] [Commented] (MAPREDUCE-6698) Increase timeout on TestUnnecessaryBlockingOnHistoryFileInfo.testTwoThreadsQueryingDifferentJobOfSameUser

2016-05-17 Thread Hudson (JIRA)


[ 
https://issues.apache.org/jira/browse/MAPREDUCE-6698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15286870#comment-15286870
 ] 

Hudson commented on MAPREDUCE-6698:
---

ABORTED: Integrated in Hadoop-trunk-Commit #9793 (See 
[https://builds.apache.org/job/Hadoop-trunk-Commit/9793/])
MAPREDUCE-6698. Increase timeout on (jlowe: rev 
34fddd1e912dcd9f898e500b8df11295ee49c0d8)
* 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/java/org/apache/hadoop/mapreduce/v2/hs/TestUnnecessaryBlockingOnHistoryFileInfo.java


> Increase timeout on 
> TestUnnecessaryBlockingOnHistoryFileInfo.testTwoThreadsQueryingDifferentJobOfSameUser
> -
>
> Key: MAPREDUCE-6698
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6698
> Project: Hadoop Map/Reduce
>  Issue Type: Bug
>  Components: jobhistoryserver
>Affects Versions: 2.8.0
>Reporter: Haibo Chen
>Assignee: Haibo Chen
> Fix For: 2.8.0
>
> Attachments: mapreduce6698.001.patch
>
>
> The timeout on 
> TestUnnecessaryBlockingOnHistoryFileInfo.testTwoThreadsQueryingDifferentJobOfSameUser
>  is added to verify the fix of MAPREDUCE-6684 works. When two thread are 
> requesting different jobs owned by the same user, one thread request jobA 
> should not be blocked by the other that is processing a large job jobB. The 
> timeout exception, if happened, should ideally indicate the fix does not 
> work. But the timeout period is set too aggressive, so the test always fails  
> on slow VMs.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org

[jira] [Commented] (MAPREDUCE-6700) Jobhistory server attempt and task table not loading maps/reduce

2016-05-17 Thread Sunil G (JIRA)


[ 
https://issues.apache.org/jira/browse/MAPREDUCE-6700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15286846#comment-15286846
 ] 

Sunil G commented on MAPREDUCE-6700:


Oh ok.. Thats very interesting, we need to do some retrospection for this. I ll 
also check.

> Jobhistory server attempt and task table not loading maps/reduce
> 
>
> Key: MAPREDUCE-6700
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6700
> Project: Hadoop Map/Reduce
>  Issue Type: Bug
>Reporter: Bibin A Chundatt
>Priority: Blocker
>
> Browser
> ===
> Chrome
> Steps to reproduce
> ==
> # Submit mapreduce application with 20 maps
> # Wait till completion of mapreduce application
> # Check maps attempts page
> {{jobhistory/attempts/job_1463446678437_0003/m/SUCCESSFUL}}
> and {{jobhistory/tasks/job_1463446678437_0003/m}} page
> Actual
> =
> Table not loading.
> Sort based on any column other than attempt contents are loaded.
> Column 0 is of *natural sorting* and not working. So waiting for ever to be 
> sorted.
> {noformat}
> SCRIPT438: Object doesn't support property or method 'natural-asc' 
> jquery.dataTables.min.js, line 86 character 179
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org

[jira] [Commented] (MAPREDUCE-6700) Jobhistory server attempt and task table not loading maps/reduce

2016-05-17 Thread Varun Saxena (JIRA)


[ 
https://issues.apache.org/jira/browse/MAPREDUCE-6700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15286837#comment-15286837
 ] 

Varun Saxena commented on MAPREDUCE-6700:
-

IIUC, [~bibinchundatt] was saying this is an issue only on branch-2.8

> Jobhistory server attempt and task table not loading maps/reduce
> 
>
> Key: MAPREDUCE-6700
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6700
> Project: Hadoop Map/Reduce
>  Issue Type: Bug
>Reporter: Bibin A Chundatt
>Priority: Blocker
>
> Browser
> ===
> Chrome
> Steps to reproduce
> ==
> # Submit mapreduce application with 20 maps
> # Wait till completion of mapreduce application
> # Check maps attempts page
> {{jobhistory/attempts/job_1463446678437_0003/m/SUCCESSFUL}}
> and {{jobhistory/tasks/job_1463446678437_0003/m}} page
> Actual
> =
> Table not loading.
> Sort based on any column other than attempt contents are loaded.
> Column 0 is of *natural sorting* and not working. So waiting for ever to be 
> sorted.
> {noformat}
> SCRIPT438: Object doesn't support property or method 'natural-asc' 
> jquery.dataTables.min.js, line 86 character 179
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org

[jira] [Commented] (MAPREDUCE-6700) Jobhistory server attempt and task table not loading maps/reduce

2016-05-17 Thread Sunil G (JIRA)


[ 
https://issues.apache.org/jira/browse/MAPREDUCE-6700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15286832#comment-15286832
 ] 

Sunil G commented on MAPREDUCE-6700:


[~bibinchundatt] I dont have permission to upload screen shot, So I  will send 
screenshot to you or [~rohithsharma] so that it can be uploaded here.

> Jobhistory server attempt and task table not loading maps/reduce
> 
>
> Key: MAPREDUCE-6700
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6700
> Project: Hadoop Map/Reduce
>  Issue Type: Bug
>Reporter: Bibin A Chundatt
>Priority: Blocker
>
> Browser
> ===
> Chrome
> Steps to reproduce
> ==
> # Submit mapreduce application with 20 maps
> # Wait till completion of mapreduce application
> # Check maps attempts page
> {{jobhistory/attempts/job_1463446678437_0003/m/SUCCESSFUL}}
> and {{jobhistory/tasks/job_1463446678437_0003/m}} page
> Actual
> =
> Table not loading.
> Sort based on any column other than attempt contents are loaded.
> Column 0 is of *natural sorting* and not working. So waiting for ever to be 
> sorted.
> {noformat}
> SCRIPT438: Object doesn't support property or method 'natural-asc' 
> jquery.dataTables.min.js, line 86 character 179
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org

[jira] [Commented] (MAPREDUCE-6700) Jobhistory server attempt and task table not loading maps/reduce

2016-05-17 Thread Sunil G (JIRA)


[ 
https://issues.apache.org/jira/browse/MAPREDUCE-6700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15286803#comment-15286803
 ] 

Sunil G commented on MAPREDUCE-6700:


Hi [~bibinchundatt]
I checked this case, and I am able to get the page. I couldnt upload a screen 
shot, but I used below URL.
{{http://localhost:19888/jobhistory/attempts/job_1463495738783_0004/m/SUCCESSFUL}}.
 I tried in trunk

> Jobhistory server attempt and task table not loading maps/reduce
> 
>
> Key: MAPREDUCE-6700
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6700
> Project: Hadoop Map/Reduce
>  Issue Type: Bug
>Reporter: Bibin A Chundatt
>Priority: Blocker
>
> Browser
> ===
> Chrome
> Steps to reproduce
> ==
> # Submit mapreduce application with 20 maps
> # Wait till completion of mapreduce application
> # Check maps attempts page
> {{jobhistory/attempts/job_1463446678437_0003/m/SUCCESSFUL}}
> and {{jobhistory/tasks/job_1463446678437_0003/m}} page
> Actual
> =
> Table not loading.
> Sort based on any column other than attempt contents are loaded.
> Column 0 is of *natural sorting* and not working. So waiting for ever to be 
> sorted.
> {noformat}
> SCRIPT438: Object doesn't support property or method 'natural-asc' 
> jquery.dataTables.min.js, line 86 character 179
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org

[jira] [Updated] (MAPREDUCE-6698) Increase timeout on TestUnnecessaryBlockingOnHistoryFileInfo.testTwoThreadsQueryingDifferentJobOfSameUser

2016-05-17 Thread Jason Lowe (JIRA)


 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-6698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jason Lowe updated MAPREDUCE-6698:
--
   Resolution: Fixed
 Hadoop Flags: Reviewed
Fix Version/s: 2.8.0
   Status: Resolved  (was: Patch Available)

Thanks, [~haibochen]!  I committed this to trunk, branch-2, and branch-2.8.

> Increase timeout on 
> TestUnnecessaryBlockingOnHistoryFileInfo.testTwoThreadsQueryingDifferentJobOfSameUser
> -
>
> Key: MAPREDUCE-6698
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6698
> Project: Hadoop Map/Reduce
>  Issue Type: Bug
>  Components: jobhistoryserver
>Affects Versions: 2.8.0
>Reporter: Haibo Chen
>Assignee: Haibo Chen
> Fix For: 2.8.0
>
> Attachments: mapreduce6698.001.patch
>
>
> The timeout on 
> TestUnnecessaryBlockingOnHistoryFileInfo.testTwoThreadsQueryingDifferentJobOfSameUser
>  is added to verify the fix of MAPREDUCE-6684 works. When two thread are 
> requesting different jobs owned by the same user, one thread request jobA 
> should not be blocked by the other that is processing a large job jobB. The 
> timeout exception, if happened, should ideally indicate the fix does not 
> work. But the timeout period is set too aggressive, so the test always fails  
> on slow VMs.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org

[jira] [Updated] (MAPREDUCE-6698) Increase timeout on TestUnnecessaryBlockingOnHistoryFileInfo.testTwoThreadsQueryingDifferentJobOfSameUser

2016-05-17 Thread Jason Lowe (JIRA)


 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-6698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jason Lowe updated MAPREDUCE-6698:
--
Affects Version/s: (was: 2.7.3)
   2.8.0
 Target Version/s: 2.8.0  (was: 2.7.3)

TestUnnecessaryBlockingOnHistoryFileInfo doesn't exist in branch-2.7.  
Adjusting the affects/target versions accordingly.

> Increase timeout on 
> TestUnnecessaryBlockingOnHistoryFileInfo.testTwoThreadsQueryingDifferentJobOfSameUser
> -
>
> Key: MAPREDUCE-6698
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6698
> Project: Hadoop Map/Reduce
>  Issue Type: Bug
>  Components: jobhistoryserver
>Affects Versions: 2.8.0
>Reporter: Haibo Chen
>Assignee: Haibo Chen
> Attachments: mapreduce6698.001.patch
>
>
> The timeout on 
> TestUnnecessaryBlockingOnHistoryFileInfo.testTwoThreadsQueryingDifferentJobOfSameUser
>  is added to verify the fix of MAPREDUCE-6684 works. When two thread are 
> requesting different jobs owned by the same user, one thread request jobA 
> should not be blocked by the other that is processing a large job jobB. The 
> timeout exception, if happened, should ideally indicate the fix does not 
> work. But the timeout period is set too aggressive, so the test always fails  
> on slow VMs.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org

[jira] [Commented] (MAPREDUCE-6698) Increase timeout on TestUnnecessaryBlockingOnHistoryFileInfo.testTwoThreadsQueryingDifferentJobOfSameUser

2016-05-17 Thread Jason Lowe (JIRA)


[ 
https://issues.apache.org/jira/browse/MAPREDUCE-6698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15286782#comment-15286782
 ] 

Jason Lowe commented on MAPREDUCE-6698:
---

+1 lgtm.  Committing this.

> Increase timeout on 
> TestUnnecessaryBlockingOnHistoryFileInfo.testTwoThreadsQueryingDifferentJobOfSameUser
> -
>
> Key: MAPREDUCE-6698
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6698
> Project: Hadoop Map/Reduce
>  Issue Type: Bug
>  Components: jobhistoryserver
>Affects Versions: 2.7.3
>Reporter: Haibo Chen
>Assignee: Haibo Chen
> Attachments: mapreduce6698.001.patch
>
>
> The timeout on 
> TestUnnecessaryBlockingOnHistoryFileInfo.testTwoThreadsQueryingDifferentJobOfSameUser
>  is added to verify the fix of MAPREDUCE-6684 works. When two thread are 
> requesting different jobs owned by the same user, one thread request jobA 
> should not be blocked by the other that is processing a large job jobB. The 
> timeout exception, if happened, should ideally indicate the fix does not 
> work. But the timeout period is set too aggressive, so the test always fails  
> on slow VMs.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org

[jira] [Updated] (MAPREDUCE-6701) application master log can not be available when clicking jobhistory's am logs link

2016-05-17 Thread Jason Lowe (JIRA)


 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-6701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jason Lowe updated MAPREDUCE-6701:
--
   Resolution: Fixed
 Hadoop Flags: Reviewed
Fix Version/s: 2.9.0
   Status: Resolved  (was: Patch Available)

Thanks to [~haibochen] for the contribution and to [~rkanter] for additional 
review!  I committed this to trunk and branch-2.


> application master log can not be available when clicking jobhistory's am 
> logs link
> ---
>
> Key: MAPREDUCE-6701
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6701
> Project: Hadoop Map/Reduce
>  Issue Type: Bug
>  Components: jobhistoryserver
>Affects Versions: 2.9.0
>Reporter: chenyukang
>Assignee: Haibo Chen
>Priority: Critical
> Fix For: 2.9.0
>
> Attachments: yarn5041.001.patch, yarn5041.002.patch
>
>
> In history server webapp, application master logs link is wrong. it shows "No 
> logs available for container container_1462419429440_0003_01_01".  It 
> direct to a wrong nodemanager http port instead of a node manager' container 
> managerment port. I think YARN-4701 brought this bug



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org

[jira] [Moved] (MAPREDUCE-6701) application master log can not be available when clicking jobhistory's am logs link

2016-05-17 Thread Jason Lowe (JIRA)


 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-6701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jason Lowe moved YARN-5041 to MAPREDUCE-6701:
-

Affects Version/s: (was: 2.9.0)
   2.9.0
 Target Version/s: 2.9.0  (was: 2.9.0)
  Component/s: (was: yarn)
   jobhistoryserver
  Key: MAPREDUCE-6701  (was: YARN-5041)
  Project: Hadoop Map/Reduce  (was: Hadoop YARN)

> application master log can not be available when clicking jobhistory's am 
> logs link
> ---
>
> Key: MAPREDUCE-6701
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6701
> Project: Hadoop Map/Reduce
>  Issue Type: Bug
>  Components: jobhistoryserver
>Affects Versions: 2.9.0
>Reporter: chenyukang
>Assignee: Haibo Chen
>Priority: Critical
> Attachments: yarn5041.001.patch, yarn5041.002.patch
>
>
> In history server webapp, application master logs link is wrong. it shows "No 
> logs available for container container_1462419429440_0003_01_01".  It 
> direct to a wrong nodemanager http port instead of a node manager' container 
> managerment port. I think YARN-4701 brought this bug



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org

[jira] [Updated] (MAPREDUCE-6700) Jobhistory server attempt and task table not loading maps/reduce

2016-05-17 Thread Bibin A Chundatt (JIRA)


 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-6700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Bibin A Chundatt updated MAPREDUCE-6700:

Description: 
Browser
===
Chrome

Steps to reproduce
==
# Submit mapreduce application with 20 maps
# Wait till completion of mapreduce application
# Check maps attempts page
{{jobhistory/attempts/job_1463446678437_0003/m/SUCCESSFUL}}
and {{jobhistory/tasks/job_1463446678437_0003/m}} page

Actual
=
Table not loading.
Sort based on any column other than attempt contents are loaded.

Column 0 is of *natural sorting* and not working. So waiting for ever to be 
sorted.


{noformat}
SCRIPT438: Object doesn't support property or method 'natural-asc' 
jquery.dataTables.min.js, line 86 character 179

{noformat}


  was:
Browser
===
Chrome

Steps to reproduce
==
# Submit mapreduce application with 20 maps
# Wait till completion of mapreduce application
# Check maps attempts page
{{jobhistory/attempts/job_1463446678437_0003/m/SUCCESSFUL}}
and {{jobhistory/tasks/job_1463446678437_0003/m}} page

Actual
=
Table not loading.
Sort based on any column other than attempt contents are loaded.

Column 0 is of *natural sorting* and not working. So waiting for ever to be 
sorted.




> Jobhistory server attempt and task table not loading maps/reduce
> 
>
> Key: MAPREDUCE-6700
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6700
> Project: Hadoop Map/Reduce
>  Issue Type: Bug
>Reporter: Bibin A Chundatt
>Priority: Blocker
>
> Browser
> ===
> Chrome
> Steps to reproduce
> ==
> # Submit mapreduce application with 20 maps
> # Wait till completion of mapreduce application
> # Check maps attempts page
> {{jobhistory/attempts/job_1463446678437_0003/m/SUCCESSFUL}}
> and {{jobhistory/tasks/job_1463446678437_0003/m}} page
> Actual
> =
> Table not loading.
> Sort based on any column other than attempt contents are loaded.
> Column 0 is of *natural sorting* and not working. So waiting for ever to be 
> sorted.
> {noformat}
> SCRIPT438: Object doesn't support property or method 'natural-asc' 
> jquery.dataTables.min.js, line 86 character 179
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org

[jira] [Created] (MAPREDUCE-6700) Jobhistory server attempt and task table not loading maps/reduce

2016-05-17 Thread Bibin A Chundatt (JIRA)

Bibin A Chundatt created MAPREDUCE-6700:
---

 Summary: Jobhistory server attempt and task table not loading 
maps/reduce
 Key: MAPREDUCE-6700
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6700
 Project: Hadoop Map/Reduce
  Issue Type: Bug
Reporter: Bibin A Chundatt
Priority: Blocker


Browser
===
Chrome

Steps to reproduce
==
# Submit mapreduce application with 20 maps
# Wait till completion of mapreduce application
# Check maps attempts page
{{jobhistory/attempts/job_1463446678437_0003/m/SUCCESSFUL}}
and {{jobhistory/tasks/job_1463446678437_0003/m}} page

Actual
=
Table not loading.
Sort based on any column other than attempt contents are loaded.

Column 0 is of *natural sorting* and not working. So waiting for ever to be 
sorted.





--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org

[jira] [Commented] (MAPREDUCE-6693) ArrayIndexOutOfBoundsException occurs when the length of the job name is equal to mapreduce.jobhistory.jobname.limit

2016-05-17 Thread Hudson (JIRA)


[ 
https://issues.apache.org/jira/browse/MAPREDUCE-6693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15286453#comment-15286453
 ] 

Hudson commented on MAPREDUCE-6693:
---

FAILURE: Integrated in Hadoop-trunk-Commit #9789 (See 
[https://builds.apache.org/job/Hadoop-trunk-Commit/9789/])
MAPREDUCE-6693. ArrayIndexOutOfBoundsException occurs when the length of 
(aajisaka: rev b3579305268bffc44a8041c5b75f15f01ebb9ce4)
* 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/jobhistory/FileNameIndexUtils.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/test/java/org/apache/hadoop/mapreduce/v2/jobhistory/TestFileNameIndexUtils.java


> ArrayIndexOutOfBoundsException occurs when the length of the job name is 
> equal to mapreduce.jobhistory.jobname.limit
> 
>
> Key: MAPREDUCE-6693
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6693
> Project: Hadoop Map/Reduce
>  Issue Type: Bug
>Reporter: Bibin A Chundatt
>Assignee: Ajith S
>Priority: Critical
> Fix For: 2.8.0
>
> Attachments: MAPREDUCE-6693.branch-2.patch, MAPREDUCE-6693.patch, 
> MAPREDUCE-6693.patch
>
>
> Job history entry missing when JOB name is of 
> {{mapreduce.jobhistory.jobname.limit}} character
> {noformat}
> 2016-05-10 06:51:00,674 DEBUG [Thread-73] 
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Interrupting 
> Event Handling thread
> 2016-05-10 06:51:00,674 DEBUG [Thread-73] 
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Waiting for 
> Event Handling thread to complete
> 2016-05-10 06:51:00,674 ERROR [eventHandlingThread] 
> org.apache.hadoop.yarn.YarnUncaughtExceptionHandler: Thread 
> Thread[eventHandlingThread,5,main] threw an Exception.
> java.lang.ArrayIndexOutOfBoundsException: 50
>   at 
> org.apache.hadoop.mapreduce.v2.jobhistory.FileNameIndexUtils.trimURLEncodedString(FileNameIndexUtils.java:326)
>   at 
> org.apache.hadoop.mapreduce.v2.jobhistory.FileNameIndexUtils.getDoneFileName(FileNameIndexUtils.java:86)
>   at 
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler.processDoneFiles(JobHistoryEventHandler.java:1147)
>   at 
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler.handleEvent(JobHistoryEventHandler.java:635)
>   at 
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler$1.run(JobHistoryEventHandler.java:341)
>   at java.lang.Thread.run(Thread.java:745)
> 2016-05-10 06:51:00,675 DEBUG [Thread-73] 
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Shutting down 
> timer for Job MetaInfo for job_1462840033869_0009 history file 
> hdfs://hacluster:9820/staging-dir/dsperf/.staging/job_1462840033869_0009/job_1462840033869_0009_1.jhist
> 2016-05-10 06:51:00,675 DEBUG [Thread-73] 
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Shutting down 
> timer Job MetaInfo for job_1462840033869_0009 history file 
> hdfs://hacluster:9820/staging-dir/dsperf/.staging/job_1462840033869_0009/job_1462840033869_0009_1.jhist
> 2016-05-10 06:51:00,676 DEBUG [Thread-73] 
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Closing Writer
> {noformat}
> Looks like 50 character check is going wrong



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org

[jira] [Updated] (MAPREDUCE-6693) ArrayIndexOutOfBoundsException occurs when the length of the job name is equal to mapreduce.jobhistory.jobname.limit

2016-05-17 Thread Akira AJISAKA (JIRA)


 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-6693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Akira AJISAKA updated MAPREDUCE-6693:
-
   Resolution: Fixed
Fix Version/s: 2.8.0
   Status: Resolved  (was: Patch Available)

Committed this to trunk, branch-2, and branch-2.8. Thanks Ajith for the 
contribution, and thanks Kosuke and Bibin for the reviews!

> ArrayIndexOutOfBoundsException occurs when the length of the job name is 
> equal to mapreduce.jobhistory.jobname.limit
> 
>
> Key: MAPREDUCE-6693
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6693
> Project: Hadoop Map/Reduce
>  Issue Type: Bug
>Reporter: Bibin A Chundatt
>Assignee: Ajith S
>Priority: Critical
> Fix For: 2.8.0
>
> Attachments: MAPREDUCE-6693.branch-2.patch, MAPREDUCE-6693.patch, 
> MAPREDUCE-6693.patch
>
>
> Job history entry missing when JOB name is of 
> {{mapreduce.jobhistory.jobname.limit}} character
> {noformat}
> 2016-05-10 06:51:00,674 DEBUG [Thread-73] 
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Interrupting 
> Event Handling thread
> 2016-05-10 06:51:00,674 DEBUG [Thread-73] 
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Waiting for 
> Event Handling thread to complete
> 2016-05-10 06:51:00,674 ERROR [eventHandlingThread] 
> org.apache.hadoop.yarn.YarnUncaughtExceptionHandler: Thread 
> Thread[eventHandlingThread,5,main] threw an Exception.
> java.lang.ArrayIndexOutOfBoundsException: 50
>   at 
> org.apache.hadoop.mapreduce.v2.jobhistory.FileNameIndexUtils.trimURLEncodedString(FileNameIndexUtils.java:326)
>   at 
> org.apache.hadoop.mapreduce.v2.jobhistory.FileNameIndexUtils.getDoneFileName(FileNameIndexUtils.java:86)
>   at 
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler.processDoneFiles(JobHistoryEventHandler.java:1147)
>   at 
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler.handleEvent(JobHistoryEventHandler.java:635)
>   at 
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler$1.run(JobHistoryEventHandler.java:341)
>   at java.lang.Thread.run(Thread.java:745)
> 2016-05-10 06:51:00,675 DEBUG [Thread-73] 
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Shutting down 
> timer for Job MetaInfo for job_1462840033869_0009 history file 
> hdfs://hacluster:9820/staging-dir/dsperf/.staging/job_1462840033869_0009/job_1462840033869_0009_1.jhist
> 2016-05-10 06:51:00,675 DEBUG [Thread-73] 
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Shutting down 
> timer Job MetaInfo for job_1462840033869_0009 history file 
> hdfs://hacluster:9820/staging-dir/dsperf/.staging/job_1462840033869_0009/job_1462840033869_0009_1.jhist
> 2016-05-10 06:51:00,676 DEBUG [Thread-73] 
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Closing Writer
> {noformat}
> Looks like 50 character check is going wrong



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org

[jira] [Updated] (MAPREDUCE-6693) ArrayIndexOutOfBoundsException occurs when the length of the job name is equal to mapreduce.jobhistory.jobname.limit

2016-05-17 Thread Akira AJISAKA (JIRA)


 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-6693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Akira AJISAKA updated MAPREDUCE-6693:
-
Hadoop Flags: Reviewed
 Summary: ArrayIndexOutOfBoundsException occurs when the length of the 
job name is equal to mapreduce.jobhistory.jobname.limit  (was: Job history 
entry missing when JOB name is of mapreduce.jobhistory.jobname.limit length)

> ArrayIndexOutOfBoundsException occurs when the length of the job name is 
> equal to mapreduce.jobhistory.jobname.limit
> 
>
> Key: MAPREDUCE-6693
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6693
> Project: Hadoop Map/Reduce
>  Issue Type: Bug
>Reporter: Bibin A Chundatt
>Assignee: Ajith S
>Priority: Critical
> Attachments: MAPREDUCE-6693.branch-2.patch, MAPREDUCE-6693.patch, 
> MAPREDUCE-6693.patch
>
>
> Job history entry missing when JOB name is of 
> {{mapreduce.jobhistory.jobname.limit}} character
> {noformat}
> 2016-05-10 06:51:00,674 DEBUG [Thread-73] 
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Interrupting 
> Event Handling thread
> 2016-05-10 06:51:00,674 DEBUG [Thread-73] 
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Waiting for 
> Event Handling thread to complete
> 2016-05-10 06:51:00,674 ERROR [eventHandlingThread] 
> org.apache.hadoop.yarn.YarnUncaughtExceptionHandler: Thread 
> Thread[eventHandlingThread,5,main] threw an Exception.
> java.lang.ArrayIndexOutOfBoundsException: 50
>   at 
> org.apache.hadoop.mapreduce.v2.jobhistory.FileNameIndexUtils.trimURLEncodedString(FileNameIndexUtils.java:326)
>   at 
> org.apache.hadoop.mapreduce.v2.jobhistory.FileNameIndexUtils.getDoneFileName(FileNameIndexUtils.java:86)
>   at 
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler.processDoneFiles(JobHistoryEventHandler.java:1147)
>   at 
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler.handleEvent(JobHistoryEventHandler.java:635)
>   at 
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler$1.run(JobHistoryEventHandler.java:341)
>   at java.lang.Thread.run(Thread.java:745)
> 2016-05-10 06:51:00,675 DEBUG [Thread-73] 
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Shutting down 
> timer for Job MetaInfo for job_1462840033869_0009 history file 
> hdfs://hacluster:9820/staging-dir/dsperf/.staging/job_1462840033869_0009/job_1462840033869_0009_1.jhist
> 2016-05-10 06:51:00,675 DEBUG [Thread-73] 
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Shutting down 
> timer Job MetaInfo for job_1462840033869_0009 history file 
> hdfs://hacluster:9820/staging-dir/dsperf/.staging/job_1462840033869_0009/job_1462840033869_0009_1.jhist
> 2016-05-10 06:51:00,676 DEBUG [Thread-73] 
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Closing Writer
> {noformat}
> Looks like 50 character check is going wrong



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org

[jira] [Commented] (MAPREDUCE-6693) Job history entry missing when JOB name is of mapreduce.jobhistory.jobname.limit length

2016-05-17 Thread Akira AJISAKA (JIRA)


[ 
https://issues.apache.org/jira/browse/MAPREDUCE-6693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15286415#comment-15286415
 ] 

Akira AJISAKA commented on MAPREDUCE-6693:
--

Very good to see the precommit job uses only JDK8 to test against trunk. +1, 
checking this in.

> Job history entry missing when JOB name is of 
> mapreduce.jobhistory.jobname.limit length
> ---
>
> Key: MAPREDUCE-6693
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6693
> Project: Hadoop Map/Reduce
>  Issue Type: Bug
>Reporter: Bibin A Chundatt
>Assignee: Ajith S
>Priority: Critical
> Attachments: MAPREDUCE-6693.branch-2.patch, MAPREDUCE-6693.patch, 
> MAPREDUCE-6693.patch
>
>
> Job history entry missing when JOB name is of 
> {{mapreduce.jobhistory.jobname.limit}} character
> {noformat}
> 2016-05-10 06:51:00,674 DEBUG [Thread-73] 
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Interrupting 
> Event Handling thread
> 2016-05-10 06:51:00,674 DEBUG [Thread-73] 
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Waiting for 
> Event Handling thread to complete
> 2016-05-10 06:51:00,674 ERROR [eventHandlingThread] 
> org.apache.hadoop.yarn.YarnUncaughtExceptionHandler: Thread 
> Thread[eventHandlingThread,5,main] threw an Exception.
> java.lang.ArrayIndexOutOfBoundsException: 50
>   at 
> org.apache.hadoop.mapreduce.v2.jobhistory.FileNameIndexUtils.trimURLEncodedString(FileNameIndexUtils.java:326)
>   at 
> org.apache.hadoop.mapreduce.v2.jobhistory.FileNameIndexUtils.getDoneFileName(FileNameIndexUtils.java:86)
>   at 
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler.processDoneFiles(JobHistoryEventHandler.java:1147)
>   at 
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler.handleEvent(JobHistoryEventHandler.java:635)
>   at 
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler$1.run(JobHistoryEventHandler.java:341)
>   at java.lang.Thread.run(Thread.java:745)
> 2016-05-10 06:51:00,675 DEBUG [Thread-73] 
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Shutting down 
> timer for Job MetaInfo for job_1462840033869_0009 history file 
> hdfs://hacluster:9820/staging-dir/dsperf/.staging/job_1462840033869_0009/job_1462840033869_0009_1.jhist
> 2016-05-10 06:51:00,675 DEBUG [Thread-73] 
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Shutting down 
> timer Job MetaInfo for job_1462840033869_0009 history file 
> hdfs://hacluster:9820/staging-dir/dsperf/.staging/job_1462840033869_0009/job_1462840033869_0009_1.jhist
> 2016-05-10 06:51:00,676 DEBUG [Thread-73] 
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Closing Writer
> {noformat}
> Looks like 50 character check is going wrong



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org

[jira] [Commented] (MAPREDUCE-6693) Job history entry missing when JOB name is of mapreduce.jobhistory.jobname.limit length

2016-05-17 Thread Hadoop QA (JIRA)


[ 
https://issues.apache.org/jira/browse/MAPREDUCE-6693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15286331#comment-15286331
 ] 

Hadoop QA commented on MAPREDUCE-6693:
--

| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 19m 1s 
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s 
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 
0s {color} | {color:green} The patch appears to include 1 new or modified test 
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m 
37s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 25s 
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 
17s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 31s 
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 
15s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m 
50s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 29s 
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 
25s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 23s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 23s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 
14s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 27s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 
12s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 2s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 26s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 55s 
{color} | {color:green} hadoop-mapreduce-client-common in the patch passed. 
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 
19s {color} | {color:green} Patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 36m 36s {color} 
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker |  Image:yetus/hadoop:2c91fd8 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12804375/MAPREDUCE-6693.patch |
| JIRA Issue | MAPREDUCE-6693 |
| Optional Tests |  asflicense  compile  javac  javadoc  mvninstall  mvnsite  
unit  findbugs  checkstyle  |
| uname | Linux a5edbc7ee1a0 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed 
Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh 
|
| git revision | trunk / 9fe5828 |
| Default Java | 1.8.0_91 |
| findbugs | v3.0.0 |
|  Test Results | 
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6509/testReport/ |
| modules | C: 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common 
U: 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common 
|
| Console output | 
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6509/console |
| Powered by | Apache Yetus 0.2.0   http://yetus.apache.org |


This message was automatically generated.



> Job history entry missing when JOB name is of 
> mapreduce.jobhistory.jobname.limit length
> ---
>
> Key: MAPREDUCE-6693
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6693
> Project: Hadoop Map/Reduce
>  Issue Type: Bug
>Reporter: Bibin A Chundatt
>Assignee: Ajith S
>Priority: Critical
> Attachments:

[jira] [Commented] (MAPREDUCE-6695) Problem accessing /jobtracker.jsp

2016-05-17 Thread Mina Samir Yacoub (JIRA)


[ 
https://issues.apache.org/jira/browse/MAPREDUCE-6695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15286288#comment-15286288
 ] 

Mina Samir Yacoub commented on MAPREDUCE-6695:
--

Thanks so much for your support.

> Problem accessing /jobtracker.jsp
> -
>
> Key: MAPREDUCE-6695
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6695
> Project: Hadoop Map/Reduce
>  Issue Type: Bug
>  Components: jobtracker
>Affects Versions: 2.4.1
> Environment: MapR Cluster 
>Reporter: Mina Samir Yacoub
> Attachments: screenshot-1.png
>
>
> Problem accessing /jobtracker.jsp. Reason:
> Too many counters: 121 max=120
> Caused by:
> org.apache.hadoop.mapreduce.counters.LimitExceededException: Too many 
> counters: 121 max=120
>   at 
> org.apache.hadoop.mapreduce.counters.Limits.checkCounters(Limits.java:61)
>   at 
> org.apache.hadoop.mapreduce.counters.Limits.incrCounters(Limits.java:68)
>   at 
> org.apache.hadoop.mapreduce.counters.AbstractCounterGroup.addCounter(AbstractCounterGroup.java:77)
>   at 
> org.apache.hadoop.mapreduce.counters.AbstractCounterGroup.addCounterImpl(AbstractCounterGroup.java:94)
>   at 
> org.apache.hadoop.mapreduce.counters.AbstractCounterGroup.findCounterImpl(AbstractCounterGroup.java:122)
>   at 
> org.apache.hadoop.mapreduce.counters.AbstractCounterGroup.findCounter(AbstractCounterGroup.java:112)
>   at 
> org.apache.hadoop.mapreduce.counters.AbstractCounterGroup.findCounter(AbstractCounterGroup.java:129)
>   at 
> org.apache.hadoop.mapred.Counters$Group.findCounter(Counters.java:323)
>   at 
> org.apache.hadoop.mapred.Counters$Group.getCounterForName(Counters.java:268)
>   at org.apache.hadoop.mapred.Counters.incrAllCounters(Counters.java:533)
>   at 
> org.apache.hadoop.mapred.JobInProgress.incrementTaskCounters(JobInProgress.java:1705)
>   at 
> org.apache.hadoop.mapred.JobInProgress.getMapCounters(JobInProgress.java:1667)
>   at org.apache.hadoop.mapred.JSPUtil.generateJobTable(JSPUtil.java:428)
>   at 
> org.apache.hadoop.mapred.jobtracker_jsp._jspService(jobtracker_jsp.java:207)
>   at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:98)
>   at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
>   at 
> org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org

[jira] [Updated] (MAPREDUCE-6693) Job history entry missing when JOB name is of mapreduce.jobhistory.jobname.limit length

2016-05-17 Thread Akira AJISAKA (JIRA)


 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-6693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Akira AJISAKA updated MAPREDUCE-6693:
-
Attachment: MAPREDUCE-6693.patch

Jenkins +1ed to branch-2. Attaching the same patch to test against trunk.

> Job history entry missing when JOB name is of 
> mapreduce.jobhistory.jobname.limit length
> ---
>
> Key: MAPREDUCE-6693
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6693
> Project: Hadoop Map/Reduce
>  Issue Type: Bug
>Reporter: Bibin A Chundatt
>Assignee: Ajith S
>Priority: Critical
> Attachments: MAPREDUCE-6693.branch-2.patch, MAPREDUCE-6693.patch, 
> MAPREDUCE-6693.patch
>
>
> Job history entry missing when JOB name is of 
> {{mapreduce.jobhistory.jobname.limit}} character
> {noformat}
> 2016-05-10 06:51:00,674 DEBUG [Thread-73] 
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Interrupting 
> Event Handling thread
> 2016-05-10 06:51:00,674 DEBUG [Thread-73] 
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Waiting for 
> Event Handling thread to complete
> 2016-05-10 06:51:00,674 ERROR [eventHandlingThread] 
> org.apache.hadoop.yarn.YarnUncaughtExceptionHandler: Thread 
> Thread[eventHandlingThread,5,main] threw an Exception.
> java.lang.ArrayIndexOutOfBoundsException: 50
>   at 
> org.apache.hadoop.mapreduce.v2.jobhistory.FileNameIndexUtils.trimURLEncodedString(FileNameIndexUtils.java:326)
>   at 
> org.apache.hadoop.mapreduce.v2.jobhistory.FileNameIndexUtils.getDoneFileName(FileNameIndexUtils.java:86)
>   at 
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler.processDoneFiles(JobHistoryEventHandler.java:1147)
>   at 
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler.handleEvent(JobHistoryEventHandler.java:635)
>   at 
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler$1.run(JobHistoryEventHandler.java:341)
>   at java.lang.Thread.run(Thread.java:745)
> 2016-05-10 06:51:00,675 DEBUG [Thread-73] 
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Shutting down 
> timer for Job MetaInfo for job_1462840033869_0009 history file 
> hdfs://hacluster:9820/staging-dir/dsperf/.staging/job_1462840033869_0009/job_1462840033869_0009_1.jhist
> 2016-05-10 06:51:00,675 DEBUG [Thread-73] 
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Shutting down 
> timer Job MetaInfo for job_1462840033869_0009 history file 
> hdfs://hacluster:9820/staging-dir/dsperf/.staging/job_1462840033869_0009/job_1462840033869_0009_1.jhist
> 2016-05-10 06:51:00,676 DEBUG [Thread-73] 
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Closing Writer
> {noformat}
> Looks like 50 character check is going wrong



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org

[jira] [Commented] (MAPREDUCE-6693) Job history entry missing when JOB name is of mapreduce.jobhistory.jobname.limit length

2016-05-17 Thread Hadoop QA (JIRA)


[ 
https://issues.apache.org/jira/browse/MAPREDUCE-6693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15286239#comment-15286239
 ] 

Hadoop QA commented on MAPREDUCE-6693:
--

| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 14m 7s 
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s 
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 
0s {color} | {color:green} The patch appears to include 1 new or modified test 
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 19m 
43s {color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 24s 
{color} | {color:green} branch-2 passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 25s 
{color} | {color:green} branch-2 passed with JDK v1.7.0_101 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 
22s {color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 33s 
{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 
21s {color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 8s 
{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 30s 
{color} | {color:green} branch-2 passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 31s 
{color} | {color:green} branch-2 passed with JDK v1.7.0_101 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 
25s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 21s 
{color} | {color:green} the patch passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 21s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 22s 
{color} | {color:green} the patch passed with JDK v1.7.0_101 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 22s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 
14s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 28s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 
13s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 
11s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 25s 
{color} | {color:green} the patch passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 28s 
{color} | {color:green} the patch passed with JDK v1.7.0_101 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 54s 
{color} | {color:green} hadoop-mapreduce-client-common in the patch passed with 
JDK v1.8.0_91. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 55s 
{color} | {color:green} hadoop-mapreduce-client-common in the patch passed with 
JDK v1.7.0_101. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 
20s {color} | {color:green} Patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 45m 31s {color} 
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker |  Image:yetus/hadoop:babe025 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12804356/MAPREDUCE-6693.branch-2.patch
 |
| JIRA Issue | MAPREDUCE-6693 |
| Optional Tests |  asflicense  compile  javac  javadoc  mvninstall  mvnsite  
unit  findbugs  checkstyle  |
| uname | Linux a78f22f841fc 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed 
Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh 
|
| git revision | branch-2 /

[jira] [Commented] (MAPREDUCE-6693) Job history entry missing when JOB name is of mapreduce.jobhistory.jobname.limit length

2016-05-17 Thread Hadoop QA (JIRA)


[ 
https://issues.apache.org/jira/browse/MAPREDUCE-6693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15286164#comment-15286164
 ] 

Hadoop QA commented on MAPREDUCE-6693:
--

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s 
{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} docker {color} | {color:red} 35m 41s 
{color} | {color:red} Docker failed to build yetus/hadoop:babe025. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12804356/MAPREDUCE-6693.branch-2.patch
 |
| JIRA Issue | MAPREDUCE-6693 |
| Console output | 
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6507/console |
| Powered by | Apache Yetus 0.2.0   http://yetus.apache.org |


This message was automatically generated.



> Job history entry missing when JOB name is of 
> mapreduce.jobhistory.jobname.limit length
> ---
>
> Key: MAPREDUCE-6693
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6693
> Project: Hadoop Map/Reduce
>  Issue Type: Bug
>Reporter: Bibin A Chundatt
>Assignee: Ajith S
>Priority: Critical
> Attachments: MAPREDUCE-6693.branch-2.patch, MAPREDUCE-6693.patch
>
>
> Job history entry missing when JOB name is of 
> {{mapreduce.jobhistory.jobname.limit}} character
> {noformat}
> 2016-05-10 06:51:00,674 DEBUG [Thread-73] 
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Interrupting 
> Event Handling thread
> 2016-05-10 06:51:00,674 DEBUG [Thread-73] 
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Waiting for 
> Event Handling thread to complete
> 2016-05-10 06:51:00,674 ERROR [eventHandlingThread] 
> org.apache.hadoop.yarn.YarnUncaughtExceptionHandler: Thread 
> Thread[eventHandlingThread,5,main] threw an Exception.
> java.lang.ArrayIndexOutOfBoundsException: 50
>   at 
> org.apache.hadoop.mapreduce.v2.jobhistory.FileNameIndexUtils.trimURLEncodedString(FileNameIndexUtils.java:326)
>   at 
> org.apache.hadoop.mapreduce.v2.jobhistory.FileNameIndexUtils.getDoneFileName(FileNameIndexUtils.java:86)
>   at 
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler.processDoneFiles(JobHistoryEventHandler.java:1147)
>   at 
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler.handleEvent(JobHistoryEventHandler.java:635)
>   at 
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler$1.run(JobHistoryEventHandler.java:341)
>   at java.lang.Thread.run(Thread.java:745)
> 2016-05-10 06:51:00,675 DEBUG [Thread-73] 
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Shutting down 
> timer for Job MetaInfo for job_1462840033869_0009 history file 
> hdfs://hacluster:9820/staging-dir/dsperf/.staging/job_1462840033869_0009/job_1462840033869_0009_1.jhist
> 2016-05-10 06:51:00,675 DEBUG [Thread-73] 
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Shutting down 
> timer Job MetaInfo for job_1462840033869_0009 history file 
> hdfs://hacluster:9820/staging-dir/dsperf/.staging/job_1462840033869_0009/job_1462840033869_0009_1.jhist
> 2016-05-10 06:51:00,676 DEBUG [Thread-73] 
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Closing Writer
> {noformat}
> Looks like 50 character check is going wrong



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org

[jira] [Commented] (MAPREDUCE-6693) Job history entry missing when JOB name is of mapreduce.jobhistory.jobname.limit length

2016-05-17 Thread Hadoop QA (JIRA)


[ 
https://issues.apache.org/jira/browse/MAPREDUCE-6693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15286146#comment-15286146
 ] 

Hadoop QA commented on MAPREDUCE-6693:
--

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s 
{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} docker {color} | {color:red} 26m 13s 
{color} | {color:red} Docker failed to build yetus/hadoop:2c91fd8. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12804120/MAPREDUCE-6693.patch |
| JIRA Issue | MAPREDUCE-6693 |
| Console output | 
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6506/console |
| Powered by | Apache Yetus 0.2.0   http://yetus.apache.org |


This message was automatically generated.



> Job history entry missing when JOB name is of 
> mapreduce.jobhistory.jobname.limit length
> ---
>
> Key: MAPREDUCE-6693
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6693
> Project: Hadoop Map/Reduce
>  Issue Type: Bug
>Reporter: Bibin A Chundatt
>Assignee: Ajith S
>Priority: Critical
> Attachments: MAPREDUCE-6693.branch-2.patch, MAPREDUCE-6693.patch
>
>
> Job history entry missing when JOB name is of 
> {{mapreduce.jobhistory.jobname.limit}} character
> {noformat}
> 2016-05-10 06:51:00,674 DEBUG [Thread-73] 
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Interrupting 
> Event Handling thread
> 2016-05-10 06:51:00,674 DEBUG [Thread-73] 
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Waiting for 
> Event Handling thread to complete
> 2016-05-10 06:51:00,674 ERROR [eventHandlingThread] 
> org.apache.hadoop.yarn.YarnUncaughtExceptionHandler: Thread 
> Thread[eventHandlingThread,5,main] threw an Exception.
> java.lang.ArrayIndexOutOfBoundsException: 50
>   at 
> org.apache.hadoop.mapreduce.v2.jobhistory.FileNameIndexUtils.trimURLEncodedString(FileNameIndexUtils.java:326)
>   at 
> org.apache.hadoop.mapreduce.v2.jobhistory.FileNameIndexUtils.getDoneFileName(FileNameIndexUtils.java:86)
>   at 
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler.processDoneFiles(JobHistoryEventHandler.java:1147)
>   at 
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler.handleEvent(JobHistoryEventHandler.java:635)
>   at 
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler$1.run(JobHistoryEventHandler.java:341)
>   at java.lang.Thread.run(Thread.java:745)
> 2016-05-10 06:51:00,675 DEBUG [Thread-73] 
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Shutting down 
> timer for Job MetaInfo for job_1462840033869_0009 history file 
> hdfs://hacluster:9820/staging-dir/dsperf/.staging/job_1462840033869_0009/job_1462840033869_0009_1.jhist
> 2016-05-10 06:51:00,675 DEBUG [Thread-73] 
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Shutting down 
> timer Job MetaInfo for job_1462840033869_0009 history file 
> hdfs://hacluster:9820/staging-dir/dsperf/.staging/job_1462840033869_0009/job_1462840033869_0009_1.jhist
> 2016-05-10 06:51:00,676 DEBUG [Thread-73] 
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Closing Writer
> {noformat}
> Looks like 50 character check is going wrong



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org

[jira] [Updated] (MAPREDUCE-6693) Job history entry missing when JOB name is of mapreduce.jobhistory.jobname.limit length

2016-05-17 Thread Akira AJISAKA (JIRA)


 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-6693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Akira AJISAKA updated MAPREDUCE-6693:
-
Attachment: MAPREDUCE-6693.branch-2.patch

Attaching the same patch to run precommit job against branch-2.

> Job history entry missing when JOB name is of 
> mapreduce.jobhistory.jobname.limit length
> ---
>
> Key: MAPREDUCE-6693
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6693
> Project: Hadoop Map/Reduce
>  Issue Type: Bug
>Reporter: Bibin A Chundatt
>Assignee: Ajith S
>Priority: Critical
> Attachments: MAPREDUCE-6693.branch-2.patch, MAPREDUCE-6693.patch
>
>
> Job history entry missing when JOB name is of 
> {{mapreduce.jobhistory.jobname.limit}} character
> {noformat}
> 2016-05-10 06:51:00,674 DEBUG [Thread-73] 
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Interrupting 
> Event Handling thread
> 2016-05-10 06:51:00,674 DEBUG [Thread-73] 
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Waiting for 
> Event Handling thread to complete
> 2016-05-10 06:51:00,674 ERROR [eventHandlingThread] 
> org.apache.hadoop.yarn.YarnUncaughtExceptionHandler: Thread 
> Thread[eventHandlingThread,5,main] threw an Exception.
> java.lang.ArrayIndexOutOfBoundsException: 50
>   at 
> org.apache.hadoop.mapreduce.v2.jobhistory.FileNameIndexUtils.trimURLEncodedString(FileNameIndexUtils.java:326)
>   at 
> org.apache.hadoop.mapreduce.v2.jobhistory.FileNameIndexUtils.getDoneFileName(FileNameIndexUtils.java:86)
>   at 
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler.processDoneFiles(JobHistoryEventHandler.java:1147)
>   at 
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler.handleEvent(JobHistoryEventHandler.java:635)
>   at 
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler$1.run(JobHistoryEventHandler.java:341)
>   at java.lang.Thread.run(Thread.java:745)
> 2016-05-10 06:51:00,675 DEBUG [Thread-73] 
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Shutting down 
> timer for Job MetaInfo for job_1462840033869_0009 history file 
> hdfs://hacluster:9820/staging-dir/dsperf/.staging/job_1462840033869_0009/job_1462840033869_0009_1.jhist
> 2016-05-10 06:51:00,675 DEBUG [Thread-73] 
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Shutting down 
> timer Job MetaInfo for job_1462840033869_0009 history file 
> hdfs://hacluster:9820/staging-dir/dsperf/.staging/job_1462840033869_0009/job_1462840033869_0009_1.jhist
> 2016-05-10 06:51:00,676 DEBUG [Thread-73] 
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Closing Writer
> {noformat}
> Looks like 50 character check is going wrong



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org

[jira] [Updated] (MAPREDUCE-6693) Job history entry missing when JOB name is of mapreduce.jobhistory.jobname.limit length

2016-05-17 Thread Akira AJISAKA (JIRA)


 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-6693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Akira AJISAKA updated MAPREDUCE-6693:
-
Status: Patch Available  (was: Open)

> Job history entry missing when JOB name is of 
> mapreduce.jobhistory.jobname.limit length
> ---
>
> Key: MAPREDUCE-6693
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6693
> Project: Hadoop Map/Reduce
>  Issue Type: Bug
>Reporter: Bibin A Chundatt
>Assignee: Ajith S
>Priority: Critical
> Attachments: MAPREDUCE-6693.patch
>
>
> Job history entry missing when JOB name is of 
> {{mapreduce.jobhistory.jobname.limit}} character
> {noformat}
> 2016-05-10 06:51:00,674 DEBUG [Thread-73] 
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Interrupting 
> Event Handling thread
> 2016-05-10 06:51:00,674 DEBUG [Thread-73] 
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Waiting for 
> Event Handling thread to complete
> 2016-05-10 06:51:00,674 ERROR [eventHandlingThread] 
> org.apache.hadoop.yarn.YarnUncaughtExceptionHandler: Thread 
> Thread[eventHandlingThread,5,main] threw an Exception.
> java.lang.ArrayIndexOutOfBoundsException: 50
>   at 
> org.apache.hadoop.mapreduce.v2.jobhistory.FileNameIndexUtils.trimURLEncodedString(FileNameIndexUtils.java:326)
>   at 
> org.apache.hadoop.mapreduce.v2.jobhistory.FileNameIndexUtils.getDoneFileName(FileNameIndexUtils.java:86)
>   at 
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler.processDoneFiles(JobHistoryEventHandler.java:1147)
>   at 
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler.handleEvent(JobHistoryEventHandler.java:635)
>   at 
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler$1.run(JobHistoryEventHandler.java:341)
>   at java.lang.Thread.run(Thread.java:745)
> 2016-05-10 06:51:00,675 DEBUG [Thread-73] 
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Shutting down 
> timer for Job MetaInfo for job_1462840033869_0009 history file 
> hdfs://hacluster:9820/staging-dir/dsperf/.staging/job_1462840033869_0009/job_1462840033869_0009_1.jhist
> 2016-05-10 06:51:00,675 DEBUG [Thread-73] 
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Shutting down 
> timer Job MetaInfo for job_1462840033869_0009 history file 
> hdfs://hacluster:9820/staging-dir/dsperf/.staging/job_1462840033869_0009/job_1462840033869_0009_1.jhist
> 2016-05-10 06:51:00,676 DEBUG [Thread-73] 
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Closing Writer
> {noformat}
> Looks like 50 character check is going wrong



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org

[jira] [Updated] (MAPREDUCE-6693) Job history entry missing when JOB name is of mapreduce.jobhistory.jobname.limit length

2016-05-17 Thread Akira AJISAKA (JIRA)


 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-6693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Akira AJISAKA updated MAPREDUCE-6693:
-
Target Version/s: 2.8.0

> Job history entry missing when JOB name is of 
> mapreduce.jobhistory.jobname.limit length
> ---
>
> Key: MAPREDUCE-6693
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6693
> Project: Hadoop Map/Reduce
>  Issue Type: Bug
>Reporter: Bibin A Chundatt
>Assignee: Ajith S
>Priority: Critical
> Attachments: MAPREDUCE-6693.patch
>
>
> Job history entry missing when JOB name is of 
> {{mapreduce.jobhistory.jobname.limit}} character
> {noformat}
> 2016-05-10 06:51:00,674 DEBUG [Thread-73] 
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Interrupting 
> Event Handling thread
> 2016-05-10 06:51:00,674 DEBUG [Thread-73] 
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Waiting for 
> Event Handling thread to complete
> 2016-05-10 06:51:00,674 ERROR [eventHandlingThread] 
> org.apache.hadoop.yarn.YarnUncaughtExceptionHandler: Thread 
> Thread[eventHandlingThread,5,main] threw an Exception.
> java.lang.ArrayIndexOutOfBoundsException: 50
>   at 
> org.apache.hadoop.mapreduce.v2.jobhistory.FileNameIndexUtils.trimURLEncodedString(FileNameIndexUtils.java:326)
>   at 
> org.apache.hadoop.mapreduce.v2.jobhistory.FileNameIndexUtils.getDoneFileName(FileNameIndexUtils.java:86)
>   at 
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler.processDoneFiles(JobHistoryEventHandler.java:1147)
>   at 
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler.handleEvent(JobHistoryEventHandler.java:635)
>   at 
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler$1.run(JobHistoryEventHandler.java:341)
>   at java.lang.Thread.run(Thread.java:745)
> 2016-05-10 06:51:00,675 DEBUG [Thread-73] 
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Shutting down 
> timer for Job MetaInfo for job_1462840033869_0009 history file 
> hdfs://hacluster:9820/staging-dir/dsperf/.staging/job_1462840033869_0009/job_1462840033869_0009_1.jhist
> 2016-05-10 06:51:00,675 DEBUG [Thread-73] 
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Shutting down 
> timer Job MetaInfo for job_1462840033869_0009 history file 
> hdfs://hacluster:9820/staging-dir/dsperf/.staging/job_1462840033869_0009/job_1462840033869_0009_1.jhist
> 2016-05-10 06:51:00,676 DEBUG [Thread-73] 
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Closing Writer
> {noformat}
> Looks like 50 character check is going wrong



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org

57 matches

Mail list logo