[jira] [Commented] (MAPREDUCE-6637) Testcase Failure : TestFileInputFormat.testSplitLocationInfo

2016-02-19 Thread Brahma Reddy Battula (JIRA) 

[ 
https://issues.apache.org/jira/browse/MAPREDUCE-6637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15155298#comment-15155298
 ] 

Brahma Reddy Battula commented on MAPREDUCE-6637:
-

[~andrew.wang] thanks for review and commit.

> Testcase Failure : TestFileInputFormat.testSplitLocationInfo
> 
>
> Key: MAPREDUCE-6637
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6637
> Project: Hadoop Map/Reduce
>  Issue Type: Bug
>  Components: test
>Reporter: Brahma Reddy Battula
>Assignee: Brahma Reddy Battula
> Fix For: 2.7.3
>
> Attachments: MAPREDUCE-6637.patch
>
>
> Following testcase is failing after HADOOP-12810
> {noformat}
> FAILED:  org.apache.hadoop.mapred.TestFileInputFormat.testSplitLocationInfo[0]
> Error Message:
> expected:<2> but was:<1>
> Stack Trace:
> java.lang.AssertionError: expected:<2> but was:<1>
>   at org.junit.Assert.fail(Assert.java:88)
>   at org.junit.Assert.failNotEquals(Assert.java:743)
>   at org.junit.Assert.assertEquals(Assert.java:118)
>   at org.junit.Assert.assertEquals(Assert.java:555)
>   at org.junit.Assert.assertEquals(Assert.java:542)
>   at 
> org.apache.hadoop.mapred.TestFileInputFormat.testSplitLocationInfo(TestFileInputFormat.java:115)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (MAPREDUCE-6527) Data race on field org.apache.hadoop.mapred.JobConf.credentials

2016-02-19 Thread Haibo Chen (JIRA) 

[ 
https://issues.apache.org/jira/browse/MAPREDUCE-6527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15155278#comment-15155278
 ] 

Haibo Chen commented on MAPREDUCE-6527:
---

within the construtor

> Data race on field org.apache.hadoop.mapred.JobConf.credentials
> ---
>
> Key: MAPREDUCE-6527
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6527
> Project: Hadoop Map/Reduce
>  Issue Type: Bug
>Affects Versions: 2.7.1
>Reporter: Ali Kheradmand
>Assignee: Haibo Chen
> Attachments: mapreduce6527.001.patch
>
>
> I am running the test suite against a dynamic race detector called 
> RV-Predict. Here is a race report that I got: 
> {noformat}
> Data race on field org.apache.hadoop.mapred.JobConf.credentials: {{{
> Concurrent read in thread T327 (locks held: {})
>  >  at org.apache.hadoop.mapred.JobConf.getCredentials(JobConf.java:505)
> at 
> org.apache.hadoop.mapreduce.task.JobContextImpl.(JobContextImpl.java:70)
> at 
> org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:524)
> T327 is created by T22
> at 
> org.apache.hadoop.mapred.LocalJobRunner$Job.(LocalJobRunner.java:218)
> Concurrent write in thread T22 (locks held: {Monitor@496c673a, 
> Monitor@496319b0})
>  >  at org.apache.hadoop.mapred.JobConf.setCredentials(JobConf.java:510)
> at 
> org.apache.hadoop.mapred.LocalJobRunner.submitJob(LocalJobRunner.java:787)
> at 
> org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:241)
> at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1290)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1669)
> at org.apache.hadoop.mapreduce.Job.submit(Job.java:1287)
> at 
> org.apache.hadoop.mapreduce.lib.jobcontrol.ControlledJob.submit(ControlledJob.java:335)
>  locked Monitor@496319b0 at 
> org.apache.hadoop.mapreduce.lib.jobcontrol.ControlledJob.submit(ControlledJob.java:n/a)
>  
> at 
> org.apache.hadoop.mapreduce.lib.jobcontrol.JobControl.run(JobControl.java:245)
>  locked Monitor@496c673a at 
> org.apache.hadoop.mapreduce.lib.jobcontrol.JobControl.run(JobControl.java:229)
>  
> T22 is created by T1
> at 
> org.apache.hadoop.mapred.jobcontrol.TestJobControl.doJobControlTest(TestJobControl.java:111)
> }}}
> {noformat}
> In the source code of org.apache.hadoop.mapreduce.JobStatus.submitJob 
> function, we have the following lines:
> {code}
> Job job = new Job(JobID.downgrade(jobid), jobSubmitDir);
> job.job.setCredentials(credentials);
> {code}
> It looks a bit suspicious: Job extends thread and at the end of its 
> constructor it starts a new thread which creates a new instance of 
> JobContextImpl which reads credentials. However, the first thread 
> concurrently sets credentials after a creating the Job instance. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (MAPREDUCE-6527) Data race on field org.apache.hadoop.mapred.JobConf.credentials

2016-02-19 Thread Haibo Chen (JIRA) 

 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-6527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Haibo Chen updated MAPREDUCE-6527:
--
Status: Patch Available  (was: Open)

The race condition is caused because the thread is started with the constructor.

> Data race on field org.apache.hadoop.mapred.JobConf.credentials
> ---
>
> Key: MAPREDUCE-6527
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6527
> Project: Hadoop Map/Reduce
>  Issue Type: Bug
>Affects Versions: 2.7.1
>Reporter: Ali Kheradmand
>Assignee: Haibo Chen
> Attachments: mapreduce6527.001.patch
>
>
> I am running the test suite against a dynamic race detector called 
> RV-Predict. Here is a race report that I got: 
> {noformat}
> Data race on field org.apache.hadoop.mapred.JobConf.credentials: {{{
> Concurrent read in thread T327 (locks held: {})
>  >  at org.apache.hadoop.mapred.JobConf.getCredentials(JobConf.java:505)
> at 
> org.apache.hadoop.mapreduce.task.JobContextImpl.(JobContextImpl.java:70)
> at 
> org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:524)
> T327 is created by T22
> at 
> org.apache.hadoop.mapred.LocalJobRunner$Job.(LocalJobRunner.java:218)
> Concurrent write in thread T22 (locks held: {Monitor@496c673a, 
> Monitor@496319b0})
>  >  at org.apache.hadoop.mapred.JobConf.setCredentials(JobConf.java:510)
> at 
> org.apache.hadoop.mapred.LocalJobRunner.submitJob(LocalJobRunner.java:787)
> at 
> org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:241)
> at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1290)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1669)
> at org.apache.hadoop.mapreduce.Job.submit(Job.java:1287)
> at 
> org.apache.hadoop.mapreduce.lib.jobcontrol.ControlledJob.submit(ControlledJob.java:335)
>  locked Monitor@496319b0 at 
> org.apache.hadoop.mapreduce.lib.jobcontrol.ControlledJob.submit(ControlledJob.java:n/a)
>  
> at 
> org.apache.hadoop.mapreduce.lib.jobcontrol.JobControl.run(JobControl.java:245)
>  locked Monitor@496c673a at 
> org.apache.hadoop.mapreduce.lib.jobcontrol.JobControl.run(JobControl.java:229)
>  
> T22 is created by T1
> at 
> org.apache.hadoop.mapred.jobcontrol.TestJobControl.doJobControlTest(TestJobControl.java:111)
> }}}
> {noformat}
> In the source code of org.apache.hadoop.mapreduce.JobStatus.submitJob 
> function, we have the following lines:
> {code}
> Job job = new Job(JobID.downgrade(jobid), jobSubmitDir);
> job.job.setCredentials(credentials);
> {code}
> It looks a bit suspicious: Job extends thread and at the end of its 
> constructor it starts a new thread which creates a new instance of 
> JobContextImpl which reads credentials. However, the first thread 
> concurrently sets credentials after a creating the Job instance. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (MAPREDUCE-6613) Change mapreduce.jobhistory.jhist.format default from json to binary

2016-02-19 Thread Hudson (JIRA) 

[ 
https://issues.apache.org/jira/browse/MAPREDUCE-6613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15155266#comment-15155266
 ] 

Hudson commented on MAPREDUCE-6613:
---

FAILURE: Integrated in Hadoop-trunk-Commit #9333 (See 
[https://builds.apache.org/job/Hadoop-trunk-Commit/9333/])
MAPREDUCE-6613. Change mapreduce.jobhistory.jhist.format default from (rkanter: 
rev 6eae4337d1929077ffa74734327775fb987ba910)
* 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml
* hadoop-mapreduce-project/CHANGES.txt
* 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/jobhistory/JHAdminConfig.java


> Change mapreduce.jobhistory.jhist.format default from json to binary
> 
>
> Key: MAPREDUCE-6613
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6613
> Project: Hadoop Map/Reduce
>  Issue Type: New Feature
>Affects Versions: 2.8.0
>Reporter: Ray Chiang
>Assignee: Ray Chiang
>Priority: Minor
> Fix For: 3.0.0
>
> Attachments: MAPREDUCE-6613.001.patch
>
>
> MAPREDUCE-6376 added a configuration setting to set up .jhist internal format:
> mapreduce.jobhistory.jhist.format
> Currently, the default is "json".  Changing the default to "binary" allows 
> faster parsing, but with the downside of making the file not output friendly 
> by using "hadoop fs cat".



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (MAPREDUCE-6613) Change mapreduce.jobhistory.jhist.format default from json to binary

2016-02-19 Thread Robert Kanter (JIRA) 

 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-6613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robert Kanter updated MAPREDUCE-6613:
-
   Resolution: Fixed
 Hadoop Flags: Incompatible change,Reviewed  (was: Incompatible change)
Fix Version/s: 3.0.0
   Status: Resolved  (was: Patch Available)

Thanks everyone.  Committed to trunk!

> Change mapreduce.jobhistory.jhist.format default from json to binary
> 
>
> Key: MAPREDUCE-6613
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6613
> Project: Hadoop Map/Reduce
>  Issue Type: New Feature
>Affects Versions: 2.8.0
>Reporter: Ray Chiang
>Assignee: Ray Chiang
>Priority: Minor
> Fix For: 3.0.0
>
> Attachments: MAPREDUCE-6613.001.patch
>
>
> MAPREDUCE-6376 added a configuration setting to set up .jhist internal format:
> mapreduce.jobhistory.jhist.format
> Currently, the default is "json".  Changing the default to "binary" allows 
> faster parsing, but with the downside of making the file not output friendly 
> by using "hadoop fs cat".



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (MAPREDUCE-6637) Testcase Failure : TestFileInputFormat.testSplitLocationInfo

2016-02-19 Thread Hudson (JIRA) 

[ 
https://issues.apache.org/jira/browse/MAPREDUCE-6637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15155227#comment-15155227
 ] 

Hudson commented on MAPREDUCE-6637:
---

FAILURE: Integrated in Hadoop-trunk-Commit #9332 (See 
[https://builds.apache.org/job/Hadoop-trunk-Commit/9332/])
MAPREDUCE-6637. Testcase Failure : (wang: rev 
37577852ba54ba506b0b6c4db5e95d176f5e9b63)
* hadoop-mapreduce-project/CHANGES.txt
* 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/lib/input/TestFileInputFormat.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapred/TestFileInputFormat.java


> Testcase Failure : TestFileInputFormat.testSplitLocationInfo
> 
>
> Key: MAPREDUCE-6637
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6637
> Project: Hadoop Map/Reduce
>  Issue Type: Bug
>  Components: test
>Reporter: Brahma Reddy Battula
>Assignee: Brahma Reddy Battula
> Fix For: 2.7.3
>
> Attachments: MAPREDUCE-6637.patch
>
>
> Following testcase is failing after HADOOP-12810
> {noformat}
> FAILED:  org.apache.hadoop.mapred.TestFileInputFormat.testSplitLocationInfo[0]
> Error Message:
> expected:<2> but was:<1>
> Stack Trace:
> java.lang.AssertionError: expected:<2> but was:<1>
>   at org.junit.Assert.fail(Assert.java:88)
>   at org.junit.Assert.failNotEquals(Assert.java:743)
>   at org.junit.Assert.assertEquals(Assert.java:118)
>   at org.junit.Assert.assertEquals(Assert.java:555)
>   at org.junit.Assert.assertEquals(Assert.java:542)
>   at 
> org.apache.hadoop.mapred.TestFileInputFormat.testSplitLocationInfo(TestFileInputFormat.java:115)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (MAPREDUCE-6637) Testcase Failure : TestFileInputFormat.testSplitLocationInfo

2016-02-19 Thread Andrew Wang (JIRA) 

[ 
https://issues.apache.org/jira/browse/MAPREDUCE-6637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15155215#comment-15155215
 ] 

Andrew Wang commented on MAPREDUCE-6637:


+1 LGTM thanks Brahma! Committing shortly.

> Testcase Failure : TestFileInputFormat.testSplitLocationInfo
> 
>
> Key: MAPREDUCE-6637
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6637
> Project: Hadoop Map/Reduce
>  Issue Type: Bug
>  Components: test
>Reporter: Brahma Reddy Battula
>Assignee: Brahma Reddy Battula
> Attachments: MAPREDUCE-6637.patch
>
>
> Following testcase is failing after HADOOP-12810
> {noformat}
> FAILED:  org.apache.hadoop.mapred.TestFileInputFormat.testSplitLocationInfo[0]
> Error Message:
> expected:<2> but was:<1>
> Stack Trace:
> java.lang.AssertionError: expected:<2> but was:<1>
>   at org.junit.Assert.fail(Assert.java:88)
>   at org.junit.Assert.failNotEquals(Assert.java:743)
>   at org.junit.Assert.assertEquals(Assert.java:118)
>   at org.junit.Assert.assertEquals(Assert.java:555)
>   at org.junit.Assert.assertEquals(Assert.java:542)
>   at 
> org.apache.hadoop.mapred.TestFileInputFormat.testSplitLocationInfo(TestFileInputFormat.java:115)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (MAPREDUCE-6637) Testcase Failure : TestFileInputFormat.testSplitLocationInfo

2016-02-19 Thread Andrew Wang (JIRA) 

 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-6637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Wang updated MAPREDUCE-6637:
---
   Resolution: Fixed
Fix Version/s: 2.7.3
   Status: Resolved  (was: Patch Available)

Pushed to trunk, branch-2, branch-2.8, branch-2.7 for 2.7.3. Thanks for find 
and fix Brahma!

> Testcase Failure : TestFileInputFormat.testSplitLocationInfo
> 
>
> Key: MAPREDUCE-6637
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6637
> Project: Hadoop Map/Reduce
>  Issue Type: Bug
>  Components: test
>Reporter: Brahma Reddy Battula
>Assignee: Brahma Reddy Battula
> Fix For: 2.7.3
>
> Attachments: MAPREDUCE-6637.patch
>
>
> Following testcase is failing after HADOOP-12810
> {noformat}
> FAILED:  org.apache.hadoop.mapred.TestFileInputFormat.testSplitLocationInfo[0]
> Error Message:
> expected:<2> but was:<1>
> Stack Trace:
> java.lang.AssertionError: expected:<2> but was:<1>
>   at org.junit.Assert.fail(Assert.java:88)
>   at org.junit.Assert.failNotEquals(Assert.java:743)
>   at org.junit.Assert.assertEquals(Assert.java:118)
>   at org.junit.Assert.assertEquals(Assert.java:555)
>   at org.junit.Assert.assertEquals(Assert.java:542)
>   at 
> org.apache.hadoop.mapred.TestFileInputFormat.testSplitLocationInfo(TestFileInputFormat.java:115)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (MAPREDUCE-6622) Add capability to set JHS job cache to a task-based limit

2016-02-19 Thread Ray Chiang (JIRA) 

[ 
https://issues.apache.org/jira/browse/MAPREDUCE-6622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15154995#comment-15154995
 ] 

Ray Chiang commented on MAPREDUCE-6622:
---

For LinkedHashMap implementations, I have tried the following:

A. I first did an attempt at updating LinkedHashMap by just modifying 
removeEldestEntry().  That didn't work.

B. I just did a second attempt this morning at extending LinkedHashMap and 
adding the ability to track the total task count.  It tracks the task count 
correctly, but removeEldestEntry() only gets triggered once per put.  I don't 
see a good way to improve on that.


> Add capability to set JHS job cache to a task-based limit
> -
>
> Key: MAPREDUCE-6622
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6622
> Project: Hadoop Map/Reduce
>  Issue Type: Improvement
>  Components: jobhistoryserver
>Affects Versions: 2.7.2
>Reporter: Ray Chiang
>Assignee: Ray Chiang
>  Labels: supportability
> Attachments: MAPREDUCE-6622.001.patch, MAPREDUCE-6622.002.patch
>
>
> When setting the property mapreduce.jobhistory.loadedjobs.cache.size the jobs 
> can be of varying size.  This is generally not a problem when the jobs sizes 
> are uniform or small, but when the job sizes can be very large (say greater 
> than 250k tasks), then the JHS heap size can grow tremendously.
> In cases, where multiple jobs are very large, then the JHS can lock up and 
> spend all its time in GC.  However, since the cache is holding on to all the 
> jobs, not much heap space can be freed up.
> By setting a property that sets a cap on the number of tasks allowed in the 
> cache and since the total number of tasks loaded is directly proportional to 
> the amount of heap used, this should help prevent the JHS from locking up.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (MAPREDUCE-5044) Have AM trigger jstack on task attempts that timeout before killing them

2016-02-19 Thread Eric Payne (JIRA) 

[ 
https://issues.apache.org/jira/browse/MAPREDUCE-5044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15154866#comment-15154866
 ] 

Eric Payne commented on MAPREDUCE-5044:
---

Thanks, [~jira.shegalov]. Would it be okay if I upmerged 
{{MAPREDUCE-5044.v06.patch}} and integrated it with the 
{{SignalContainerRequest}} that was added as part of YARN-445 and its children?


> Have AM trigger jstack on task attempts that timeout before killing them
> 
>
> Key: MAPREDUCE-5044
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5044
> Project: Hadoop Map/Reduce
>  Issue Type: Improvement
>  Components: mr-am
>Affects Versions: 2.1.0-beta
>Reporter: Jason Lowe
>Assignee: Gera Shegalov
> Attachments: MAPREDUCE-5044.v01.patch, MAPREDUCE-5044.v02.patch, 
> MAPREDUCE-5044.v03.patch, MAPREDUCE-5044.v04.patch, MAPREDUCE-5044.v05.patch, 
> MAPREDUCE-5044.v06.patch, Screen Shot 2013-11-12 at 1.05.32 PM.png, Screen 
> Shot 2013-11-12 at 1.06.04 PM.png
>
>
> When an AM expires a task attempt it would be nice if it triggered a jstack 
> output via SIGQUIT before killing the task attempt.  This would be invaluable 
> for helping users debug their hung tasks, especially if they do not have 
> shell access to the nodes.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (MAPREDUCE-4785) TestMRApp occasionally fails

2016-02-19 Thread Haibo Chen (JIRA) 

 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-4785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Haibo Chen reassigned MAPREDUCE-4785:
-

Assignee: Haibo Chen

> TestMRApp occasionally fails
> 
>
> Key: MAPREDUCE-4785
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4785
> Project: Hadoop Map/Reduce
>  Issue Type: Bug
>  Components: mrv2, test
>Affects Versions: 2.0.3-alpha
>Reporter: Jason Lowe
>Assignee: Haibo Chen
>
> TestMRApp is failing occasionally with this error:
> {noformat}
> testUpdatedNodes(org.apache.hadoop.mapreduce.v2.app.TestMRApp): Expecting 2 
> more completion events for killed expected:<4> but was:<2>
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (MAPREDUCE-6639) Process hangs in LocatedFileStatusFetcher if FileSystem.get throws

2016-02-19 Thread Ryan Blue (JIRA) 

 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-6639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ryan Blue updated MAPREDUCE-6639:
-
Description: 
ListLocatedFileStatusFetcher uses a thread pool, but one of the Callable thread 
functions, 
[{{ProcessInitialInputPathCallable}}|https://github.com/apache/hadoop/blob/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/LocatedFileStatusFetcher.java#L306],
 doesn't catch exceptions (the callbacks do). When an exception is thrown, the 
thread exists and doesn't signal the error to the calling thread, which 
continues waiting to be signaled. This can happen when a FS implementation 
cannot be found.

The solution is to use a try-catch around the implementation of 
{{ProcessInitialInputPathCallable#call}} that calls {{registerError}} when any 
exception is caught.

  was:
ListLocatedFileStatusFetcher uses a thread pool, but one of the Callable thread 
functions, 
[{{ProcessInitialInputPathCallable}}|https://github.com/apache/hadoop/blob/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/LocatedFileStatusFetcher.java#L306],
 doesn't catch exceptions (the others do). When an exception is thrown, the 
thread exists and doesn't signal the error to the calling thread, which 
continues waiting to be signaled. This can happen when a FS implementation 
cannot be found.

The solution is to use a try-catch around the implementation of 
{{ProcessInitialInputPathCallable#call}} that calls {{registerError}} when any 
exception is caught.


> Process hangs in LocatedFileStatusFetcher if FileSystem.get throws
> --
>
> Key: MAPREDUCE-6639
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6639
> Project: Hadoop Map/Reduce
>  Issue Type: Bug
>  Components: mrv2
>Affects Versions: 2.7.2
>Reporter: Ryan Blue
>Assignee: Ryan Blue
> Attachments: MAPREDUCE-6639.1.patch
>
>
> ListLocatedFileStatusFetcher uses a thread pool, but one of the Callable 
> thread functions, 
> [{{ProcessInitialInputPathCallable}}|https://github.com/apache/hadoop/blob/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/LocatedFileStatusFetcher.java#L306],
>  doesn't catch exceptions (the callbacks do). When an exception is thrown, 
> the thread exists and doesn't signal the error to the calling thread, which 
> continues waiting to be signaled. This can happen when a FS implementation 
> cannot be found.
> The solution is to use a try-catch around the implementation of 
> {{ProcessInitialInputPathCallable#call}} that calls {{registerError}} when 
> any exception is caught.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (MAPREDUCE-6639) Process hangs in LocatedFileStatusFetcher if FileSystem.get throws

2016-02-19 Thread Ryan Blue (JIRA) 

 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-6639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ryan Blue updated MAPREDUCE-6639:
-
Attachment: MAPREDUCE-6639.1.patch

Attaching a fix. This is slightly different from what I suggested above. 
Because a result needs to be returned for the future, I've added an 
unknownError field to the results that the futures check. If there is an error, 
the futures call registerError.

> Process hangs in LocatedFileStatusFetcher if FileSystem.get throws
> --
>
> Key: MAPREDUCE-6639
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6639
> Project: Hadoop Map/Reduce
>  Issue Type: Bug
>  Components: mrv2
>Affects Versions: 2.7.2
>Reporter: Ryan Blue
>Assignee: Ryan Blue
> Attachments: MAPREDUCE-6639.1.patch
>
>
> ListLocatedFileStatusFetcher uses a thread pool, but one of the Callable 
> thread functions, 
> [{{ProcessInitialInputPathCallable}}|https://github.com/apache/hadoop/blob/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/LocatedFileStatusFetcher.java#L306],
>  doesn't catch exceptions (the others do). When an exception is thrown, the 
> thread exists and doesn't signal the error to the calling thread, which 
> continues waiting to be signaled. This can happen when a FS implementation 
> cannot be found.
> The solution is to use a try-catch around the implementation of 
> {{ProcessInitialInputPathCallable#call}} that calls {{registerError}} when 
> any exception is caught.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (MAPREDUCE-6639) Process hangs in LocatedFileStatusFetcher if FileSystem.get throws

2016-02-19 Thread Ryan Blue (JIRA) 

 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-6639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ryan Blue updated MAPREDUCE-6639:
-
 Assignee: Ryan Blue
Affects Version/s: 2.7.2
   Status: Patch Available  (was: Open)

> Process hangs in LocatedFileStatusFetcher if FileSystem.get throws
> --
>
> Key: MAPREDUCE-6639
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6639
> Project: Hadoop Map/Reduce
>  Issue Type: Bug
>  Components: mrv2
>Affects Versions: 2.7.2
>Reporter: Ryan Blue
>Assignee: Ryan Blue
>
> ListLocatedFileStatusFetcher uses a thread pool, but one of the Callable 
> thread functions, 
> [{{ProcessInitialInputPathCallable}}|https://github.com/apache/hadoop/blob/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/LocatedFileStatusFetcher.java#L306],
>  doesn't catch exceptions (the others do). When an exception is thrown, the 
> thread exists and doesn't signal the error to the calling thread, which 
> continues waiting to be signaled. This can happen when a FS implementation 
> cannot be found.
> The solution is to use a try-catch around the implementation of 
> {{ProcessInitialInputPathCallable#call}} that calls {{registerError}} when 
> any exception is caught.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (MAPREDUCE-6639) Process hangs in LocatedFileStatusFetcher if FileSystem.get throws

2016-02-19 Thread Ryan Blue (JIRA) 

 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-6639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ryan Blue updated MAPREDUCE-6639:
-
Description: 
ListLocatedFileStatusFetcher uses a thread pool, but one of the Callable thread 
functions, 
[{{ProcessInitialInputPathCallable}}|https://github.com/apache/hadoop/blob/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/LocatedFileStatusFetcher.java#L306],
 doesn't catch exceptions (the others do). When an exception is thrown, the 
thread exists and doesn't signal the error to the calling thread, which 
continues waiting to be signaled. This can happen when a FS implementation 
cannot be found.

The solution is to use a try-catch around the implementation of 
{{ProcessInitialInputPathCallable#call}} that calls {{registerError}} when any 
exception is caught.

  was:
ListLocatedFileStatusFetcher uses a thread pool, but one of the Callable thread 
functions, 
[ProcessInitialInputPathCallable|https://github.com/apache/hadoop/blob/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/LocatedFileStatusFetcher.java#L306],
 doesn't catch exceptions (the others do). When an exception is thrown, the 
thread exists and doesn't signal the error to the calling thread, which 
continues waiting to be signaled. This can happen when a FS implementation 
cannot be found.

The solution is to use a try-catch around the implementation of 
{{ProcessInitialInputPathCallable#call}} that calls {{registerError}} when any 
exception is caught.


> Process hangs in LocatedFileStatusFetcher if FileSystem.get throws
> --
>
> Key: MAPREDUCE-6639
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6639
> Project: Hadoop Map/Reduce
>  Issue Type: Bug
>  Components: mrv2
>Reporter: Ryan Blue
>
> ListLocatedFileStatusFetcher uses a thread pool, but one of the Callable 
> thread functions, 
> [{{ProcessInitialInputPathCallable}}|https://github.com/apache/hadoop/blob/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/LocatedFileStatusFetcher.java#L306],
>  doesn't catch exceptions (the others do). When an exception is thrown, the 
> thread exists and doesn't signal the error to the calling thread, which 
> continues waiting to be signaled. This can happen when a FS implementation 
> cannot be found.
> The solution is to use a try-catch around the implementation of 
> {{ProcessInitialInputPathCallable#call}} that calls {{registerError}} when 
> any exception is caught.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (MAPREDUCE-6639) Process hangs in LocatedFileStatusFetcher if FileSystem.get throws

2016-02-19 Thread Ryan Blue (JIRA) 
Ryan Blue created MAPREDUCE-6639:


 Summary: Process hangs in LocatedFileStatusFetcher if 
FileSystem.get throws
 Key: MAPREDUCE-6639
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6639
 Project: Hadoop Map/Reduce
  Issue Type: Bug
  Components: mrv2
Reporter: Ryan Blue


ListLocatedFileStatusFetcher uses a thread pool, but one of the Callable thread 
functions, 
[ProcessInitialInputPathCallable|https://github.com/apache/hadoop/blob/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/LocatedFileStatusFetcher.java#L306],
 doesn't catch exceptions (the others do). When an exception is thrown, the 
thread exists and doesn't signal the error to the calling thread, which 
continues waiting to be signaled. This can happen when a FS implementation 
cannot be found.

The solution is to use a try-catch around the implementation of 
{{ProcessInitialInputPathCallable#call}} that calls {{registerError}} when any 
exception is caught.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (MAPREDUCE-6638) Jobs with encrypted spills don't recover if the AM goes down

2016-02-19 Thread Karthik Kambatla (JIRA) 

[ 
https://issues.apache.org/jira/browse/MAPREDUCE-6638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15154459#comment-15154459
 ] 

Karthik Kambatla commented on MAPREDUCE-6638:
-

Can't we store the key on KMS or encrypted HDFS so the second AM can recover it 
using delegation tokens? 

[~asuresh], [~vinodkv] - thoughts? 

> Jobs with encrypted spills don't recover if the AM goes down
> 
>
> Key: MAPREDUCE-6638
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6638
> Project: Hadoop Map/Reduce
>  Issue Type: Bug
>  Components: applicationmaster
>Affects Versions: 2.7.2
>Reporter: Karthik Kambatla
>Priority: Critical
>
> Post the fix to CVE-2015-1776, jobs with ecrypted spills enabled cannot be 
> recovered if the AM fails. We should store the key some place safe so they 
> can actually be recovered. If there is no "safe" place, at least we should 
> restart the job by re-running all mappers/reducers. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (MAPREDUCE-6638) Jobs with encrypted spills don't recover if the AM goes down

2016-02-19 Thread Karthik Kambatla (JIRA) 
Karthik Kambatla created MAPREDUCE-6638:
---

 Summary: Jobs with encrypted spills don't recover if the AM goes 
down
 Key: MAPREDUCE-6638
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6638
 Project: Hadoop Map/Reduce
  Issue Type: Bug
  Components: applicationmaster
Affects Versions: 2.7.2
Reporter: Karthik Kambatla
Priority: Critical


Post the fix to CVE-2015-1776, jobs with ecrypted spills enabled cannot be 
recovered if the AM fails. We should store the key some place safe so they can 
actually be recovered. If there is no "safe" place, at least we should restart 
the job by re-running all mappers/reducers. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (MAPREDUCE-6637) Testcase Failure : TestFileInputFormat.testSplitLocationInfo

2016-02-19 Thread Brahma Reddy Battula (JIRA) 

 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-6637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brahma Reddy Battula updated MAPREDUCE-6637:

Component/s: test

> Testcase Failure : TestFileInputFormat.testSplitLocationInfo
> 
>
> Key: MAPREDUCE-6637
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6637
> Project: Hadoop Map/Reduce
>  Issue Type: Bug
>  Components: test
>Reporter: Brahma Reddy Battula
>Assignee: Brahma Reddy Battula
> Attachments: MAPREDUCE-6637.patch
>
>
> Following testcase is failing after HADOOP-12810
> {noformat}
> FAILED:  org.apache.hadoop.mapred.TestFileInputFormat.testSplitLocationInfo[0]
> Error Message:
> expected:<2> but was:<1>
> Stack Trace:
> java.lang.AssertionError: expected:<2> but was:<1>
>   at org.junit.Assert.fail(Assert.java:88)
>   at org.junit.Assert.failNotEquals(Assert.java:743)
>   at org.junit.Assert.assertEquals(Assert.java:118)
>   at org.junit.Assert.assertEquals(Assert.java:555)
>   at org.junit.Assert.assertEquals(Assert.java:542)
>   at 
> org.apache.hadoop.mapred.TestFileInputFormat.testSplitLocationInfo(TestFileInputFormat.java:115)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (MAPREDUCE-6637) Testcase Failure : TestFileInputFormat.testSplitLocationInfo

2016-02-19 Thread Hadoop QA (JIRA) 

[ 
https://issues.apache.org/jira/browse/MAPREDUCE-6637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15154069#comment-15154069
 ] 

Hadoop QA commented on MAPREDUCE-6637:
--

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 10s 
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s 
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 
0s {color} | {color:green} The patch appears to include 2 new or modified test 
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m 
5s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 23s 
{color} | {color:green} trunk passed with JDK v1.8.0_72 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 25s 
{color} | {color:green} trunk passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 
18s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 35s 
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 
13s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 9s 
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 24s 
{color} | {color:green} trunk passed with JDK v1.8.0_72 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 30s 
{color} | {color:green} trunk passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 
27s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 21s 
{color} | {color:green} the patch passed with JDK v1.8.0_72 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 21s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 24s 
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 24s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 
17s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 33s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 
11s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 
25s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 23s 
{color} | {color:green} the patch passed with JDK v1.8.0_72 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 26s 
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 2m 12s {color} 
| {color:red} hadoop-mapreduce-client-core in the patch failed with JDK 
v1.8.0_72. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m 25s 
{color} | {color:green} hadoop-mapreduce-client-core in the patch passed with 
JDK v1.7.0_95. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 
24s {color} | {color:green} Patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 21m 42s {color} 
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| JDK v1.8.0_72 Failed junit tests | hadoop.mapreduce.tools.TestCLI |
\\
\\
|| Subsystem || Report/Notes ||
| Docker |  Image:yetus/hadoop:0ca8df7 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12788656/MAPREDUCE-6637.patch |
| JIRA Issue | MAPREDUCE-6637 |
| Optional Tests |  asflicense  compile  javac  javadoc  mvninstall  mvnsite  
unit  findbugs  checkstyle  |
| uname | Linux d75dc09a833a 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed 
Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |

[jira] [Updated] (MAPREDUCE-6637) Testcase Failure : TestFileInputFormat.testSplitLocationInfo

2016-02-19 Thread Brahma Reddy Battula (JIRA) 

 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-6637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brahma Reddy Battula updated MAPREDUCE-6637:

Status: Patch Available  (was: Open)

> Testcase Failure : TestFileInputFormat.testSplitLocationInfo
> 
>
> Key: MAPREDUCE-6637
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6637
> Project: Hadoop Map/Reduce
>  Issue Type: Bug
>Reporter: Brahma Reddy Battula
>Assignee: Brahma Reddy Battula
> Attachments: MAPREDUCE-6637.patch
>
>
> Following testcase is failing after HADOOP-12810
> {noformat}
> FAILED:  org.apache.hadoop.mapred.TestFileInputFormat.testSplitLocationInfo[0]
> Error Message:
> expected:<2> but was:<1>
> Stack Trace:
> java.lang.AssertionError: expected:<2> but was:<1>
>   at org.junit.Assert.fail(Assert.java:88)
>   at org.junit.Assert.failNotEquals(Assert.java:743)
>   at org.junit.Assert.assertEquals(Assert.java:118)
>   at org.junit.Assert.assertEquals(Assert.java:555)
>   at org.junit.Assert.assertEquals(Assert.java:542)
>   at 
> org.apache.hadoop.mapred.TestFileInputFormat.testSplitLocationInfo(TestFileInputFormat.java:115)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (MAPREDUCE-6637) Testcase Failure : TestFileInputFormat.testSplitLocationInfo

2016-02-19 Thread Brahma Reddy Battula (JIRA) 

 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-6637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brahma Reddy Battula updated MAPREDUCE-6637:

Attachment: MAPREDUCE-6637.patch

> Testcase Failure : TestFileInputFormat.testSplitLocationInfo
> 
>
> Key: MAPREDUCE-6637
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6637
> Project: Hadoop Map/Reduce
>  Issue Type: Bug
>Reporter: Brahma Reddy Battula
>Assignee: Brahma Reddy Battula
> Attachments: MAPREDUCE-6637.patch
>
>
> Following testcase is failing after HADOOP-12810
> {noformat}
> FAILED:  org.apache.hadoop.mapred.TestFileInputFormat.testSplitLocationInfo[0]
> Error Message:
> expected:<2> but was:<1>
> Stack Trace:
> java.lang.AssertionError: expected:<2> but was:<1>
>   at org.junit.Assert.fail(Assert.java:88)
>   at org.junit.Assert.failNotEquals(Assert.java:743)
>   at org.junit.Assert.assertEquals(Assert.java:118)
>   at org.junit.Assert.assertEquals(Assert.java:555)
>   at org.junit.Assert.assertEquals(Assert.java:542)
>   at 
> org.apache.hadoop.mapred.TestFileInputFormat.testSplitLocationInfo(TestFileInputFormat.java:115)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (MAPREDUCE-6637) Testcase Failure : TestFileInputFormat.testSplitLocationInfo

2016-02-19 Thread Brahma Reddy Battula (JIRA) 
Brahma Reddy Battula created MAPREDUCE-6637:
---

 Summary: Testcase Failure : 
TestFileInputFormat.testSplitLocationInfo
 Key: MAPREDUCE-6637
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6637
 Project: Hadoop Map/Reduce
  Issue Type: Bug
Reporter: Brahma Reddy Battula
Assignee: Brahma Reddy Battula


Following testcase is failing after HADOOP-12810
{noformat}
FAILED:  org.apache.hadoop.mapred.TestFileInputFormat.testSplitLocationInfo[0]

Error Message:
expected:<2> but was:<1>

Stack Trace:
java.lang.AssertionError: expected:<2> but was:<1>
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.failNotEquals(Assert.java:743)
at org.junit.Assert.assertEquals(Assert.java:118)
at org.junit.Assert.assertEquals(Assert.java:555)
at org.junit.Assert.assertEquals(Assert.java:542)
at 
org.apache.hadoop.mapred.TestFileInputFormat.testSplitLocationInfo(TestFileInputFormat.java:115)

{noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (MAPREDUCE-6579) JobStatus#getFailureInfo should not output diagnostic information when the job is running

2016-02-19 Thread Hadoop QA (JIRA) 

[ 
https://issues.apache.org/jira/browse/MAPREDUCE-6579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15153936#comment-15153936
 ] 

Hadoop QA commented on MAPREDUCE-6579:
--

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 11s 
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s 
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 
0s {color} | {color:green} The patch appears to include 1 new or modified test 
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 11s 
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m 
27s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 30s 
{color} | {color:green} trunk passed with JDK v1.8.0_72 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 46s 
{color} | {color:green} trunk passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 
26s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 1s 
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 
27s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 
37s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 34s 
{color} | {color:green} trunk passed with JDK v1.8.0_72 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 42s 
{color} | {color:green} trunk passed with JDK v1.7.0_95 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 9s 
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 
54s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 40s 
{color} | {color:green} the patch passed with JDK v1.8.0_72 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 40s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 49s 
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 49s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 
23s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 57s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 
25s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 2s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 32s 
{color} | {color:green} the patch passed with JDK v1.8.0_72 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 37s 
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 2m 4s {color} | 
{color:red} hadoop-mapreduce-client-core in the patch failed with JDK 
v1.8.0_72. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 107m 21s 
{color} | {color:red} hadoop-mapreduce-client-jobclient in the patch failed 
with JDK v1.8.0_72. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 2m 17s {color} 
| {color:red} hadoop-mapreduce-client-core in the patch failed with JDK 
v1.7.0_95. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 101m 47s 
{color} | {color:green} hadoop-mapreduce-client-jobclient in the patch passed 
with JDK v1.7.0_95. {color} |
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m 36s 
{color} | {color:red} Patch generated 19 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 240m 44s {color} 
| {color:black} {color} |
\\
\\
||