Jira: https://issues.apache.org/jira/browse/TEZ-3887
Build: https://builds.apache.org/job/PreCommit-TEZ-Build/2772/
###################################################################################
########################## LAST 60 LINES OF THE CONSOLE
###########################
[...truncated 360.56 KB...]
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12906487/TEZ-3887.1.patch
against master revision 2e66f3c.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 7 new
or modified test files.
{color:red}-1 javac{color}. The applied patch generated 25 javac
compiler warnings (more than the master's current 24 warnings).
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 3.0.1) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in :
org.apache.tez.mapreduce.processor.map.TestMapProcessor
org.apache.tez.runtime.library.common.writers.TestUnorderedPartitionedKVWriter
org.apache.tez.runtime.library.common.sort.impl.TestPipelinedSorter
The following test timeouts occurred in :
org.apache.tez.runtime.api.impl.TestTaskSpec
org.apache.tez.runtime.task.TestTezTaskRuTests
org.apache.tez.runtime.library.common.soTests
Test results:
https://builds.apache.org/job/PreCommit-TEZ-Build/2772//testReport/
Javac warnings:
https://builds.apache.org/job/PreCommit-TEZ-Build/2772//artifact/patchprocess/diffJavacWarnings.txt
Console output: https://builds.apache.org/job/PreCommit-TEZ-Build/2772//console
This message is automatically generated.
======================================================================
======================================================================
Adding comment to Jira.
======================================================================
======================================================================
======================================================================
======================================================================
Finished build.
======================================================================
======================================================================
Build step 'Execute shell' marked build as failure
Archiving artifacts
[description-setter] Could not determine description.
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
###################################################################################
############################## FAILED TESTS (if any)
##############################
10 tests failed.
FAILED:
org.apache.tez.mapreduce.processor.map.TestMapProcessor.testMapProcessorProgress
Error Message:
Input path does not exist: file:/tmp/TestMapProcessor
Stack Trace:
org.apache.hadoop.mapred.InvalidInputException: Input path does not exist:
file:/tmp/TestMapProcessor
at
org.apache.hadoop.mapred.FileInputFormat.singleThreadedListStatus(FileInputFormat.java:287)
at
org.apache.hadoop.mapred.FileInputFormat.listStatus(FileInputFormat.java:229)
at
org.apache.hadoop.mapred.SequenceFileInputFormat.listStatus(SequenceFileInputFormat.java:45)
at
org.apache.hadoop.mapred.FileInputFormat.getSplits(FileInputFormat.java:315)
at
org.apache.tez.mapreduce.processor.MapUtils.createInputSplit(MapUtils.java:149)
at
org.apache.tez.mapreduce.processor.MapUtils.generateInputSplit(MapUtils.java:197)
at
org.apache.tez.mapreduce.processor.map.TestMapProcessor.testMapProcessorProgress(TestMapProcessor.java:216)
FAILED:
org.apache.tez.runtime.library.common.sort.impl.TestPipelinedSorter.basicTest
Error Message:
File
/tmp/org.apache.tez.runtime.library.common.sort.impl.TestPipelinedSorter/output/a520f920-4118-4f94-b5c1-c6211d457e8e/file.out
does not exist
Stack Trace:
java.io.FileNotFoundException: File
/tmp/org.apache.tez.runtime.library.common.sort.impl.TestPipelinedSorter/output/a520f920-4118-4f94-b5c1-c6211d457e8e/file.out
does not exist
at
org.apache.hadoop.fs.RawLocalFileSystem.deprecatedGetFileStatus(RawLocalFileSystem.java:609)
at
org.apache.hadoop.fs.RawLocalFileSystem.getFileLinkStatusInternal(RawLocalFileSystem.java:822)
at
org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:599)
at
org.apache.tez.runtime.library.common.sort.impl.PipelinedSorter.flush(PipelinedSorter.java:831)
at
org.apache.tez.runtime.library.common.sort.impl.TestPipelinedSorter.closeSorter(TestPipelinedSorter.java:880)
at
org.apache.tez.runtime.library.common.sort.impl.TestPipelinedSorter.writeData(TestPipelinedSorter.java:874)
at
org.apache.tez.runtime.library.common.sort.impl.TestPipelinedSorter.writeData(TestPipelinedSorter.java:842)
at
org.apache.tez.runtime.library.common.sort.impl.TestPipelinedSorter.basicTest(TestPipelinedSorter.java:534)
at
org.apache.tez.runtime.library.common.sort.impl.TestPipelinedSorter.basicTest(TestPipelinedSorter.java:151)
FAILED:
org.apache.tez.runtime.library.common.sort.impl.TestPipelinedSorter.testMultipleSpills
Error Message:
/tmp/org.apache.tez.runtime.library.common.sort.impl.TestPipelinedSorter/output/e2bcbce0-9281-4c72-870c-1a9c16910764_0/file.out
Stack Trace:
java.io.FileNotFoundException:
/tmp/org.apache.tez.runtime.library.common.sort.impl.TestPipelinedSorter/output/e2bcbce0-9281-4c72-870c-1a9c16910764_0/file.out
at
org.apache.hadoop.fs.RawLocalFileSystem.open(RawLocalFileSystem.java:200)
at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:767)
at
org.apache.tez.runtime.library.common.sort.impl.TezMerger$DiskSegment.init(TezMerger.java:403)
at
org.apache.tez.runtime.library.common.sort.impl.TezMerger$MergeQueue.merge(TezMerger.java:760)
at
org.apache.tez.runtime.library.common.sort.impl.TezMerger$MergeQueue.merge(TezMerger.java:699)
at
org.apache.tez.runtime.library.common.sort.impl.TezMerger.merge(TezMerger.java:147)
at
org.apache.tez.runtime.library.common.sort.impl.PipelinedSorter.flush(PipelinedSorter.java:791)
at
org.apache.tez.runtime.library.common.sort.impl.TestPipelinedSorter.closeSorter(TestPipelinedSorter.java:880)
at
org.apache.tez.runtime.library.common.sort.impl.TestPipelinedSorter.writeData(TestPipelinedSorter.java:874)
at
org.apache.tez.runtime.library.common.sort.impl.TestPipelinedSorter.writeData(TestPipelinedSorter.java:842)
at
org.apache.tez.runtime.library.common.sort.impl.TestPipelinedSorter.testMultipleSpills(TestPipelinedSorter.java:438)
FAILED:
org.apache.tez.runtime.library.common.sort.impl.TestPipelinedSorter.testMultipleSpills_WithRLE
Error Message:
/tmp/org.apache.tez.runtime.library.common.sort.impl.TestPipelinedSorter/output/c0d7d2f7-e55e-4b35-9e0f-9b4c8b7e896f_0/file.out
Stack Trace:
java.io.FileNotFoundException:
/tmp/org.apache.tez.runtime.library.common.sort.impl.TestPipelinedSorter/output/c0d7d2f7-e55e-4b35-9e0f-9b4c8b7e896f_0/file.out
at
org.apache.hadoop.fs.RawLocalFileSystem.open(RawLocalFileSystem.java:200)
at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:767)
at
org.apache.tez.runtime.library.common.sort.impl.TezMerger$DiskSegment.init(TezMerger.java:403)
at
org.apache.tez.runtime.library.common.sort.impl.TezMerger$MergeQueue.merge(TezMerger.java:760)
at
org.apache.tez.runtime.library.common.sort.impl.TezMerger$MergeQueue.merge(TezMerger.java:699)
at
org.apache.tez.runtime.library.common.sort.impl.TezMerger.merge(TezMerger.java:147)
at
org.apache.tez.runtime.library.common.sort.impl.PipelinedSorter.flush(PipelinedSorter.java:791)
at
org.apache.tez.runtime.library.common.sort.impl.TestPipelinedSorter.closeSorter(TestPipelinedSorter.java:880)
at
org.apache.tez.runtime.library.common.sort.impl.TestPipelinedSorter.writeSimilarKeys(TestPipelinedSorter.java:860)
at
org.apache.tez.runtime.library.common.sort.impl.TestPipelinedSorter.testMultipleSpills_WithRLE(TestPipelinedSorter.java:491)
FAILED:
org.apache.tez.runtime.library.common.sort.impl.TestPipelinedSorter.testExceedsKVWithMultiplePartitions
Error Message:
/tmp/org.apache.tez.runtime.library.common.sort.impl.TestPipelinedSorter/output/dcf995d2-b81e-4a34-912d-058bc2034868_7/file.out
Stack Trace:
java.io.FileNotFoundException:
/tmp/org.apache.tez.runtime.library.common.sort.impl.TestPipelinedSorter/output/dcf995d2-b81e-4a34-912d-058bc2034868_7/file.out
at
org.apache.hadoop.fs.RawLocalFileSystem.open(RawLocalFileSystem.java:200)
at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:767)
at
org.apache.tez.runtime.library.common.sort.impl.TezMerger$DiskSegment.init(TezMerger.java:403)
at
org.apache.tez.runtime.library.common.sort.impl.TezMerger$MergeQueue.merge(TezMerger.java:760)
at
org.apache.tez.runtime.library.common.sort.impl.TezMerger$MergeQueue.merge(TezMerger.java:699)
at
org.apache.tez.runtime.library.common.sort.impl.TezMerger.merge(TezMerger.java:147)
at
org.apache.tez.runtime.library.common.sort.impl.PipelinedSorter.flush(PipelinedSorter.java:791)
at
org.apache.tez.runtime.library.common.sort.impl.TestPipelinedSorter.closeSorter(TestPipelinedSorter.java:880)
at
org.apache.tez.runtime.library.common.sort.impl.TestPipelinedSorter.writeData(TestPipelinedSorter.java:874)
at
org.apache.tez.runtime.library.common.sort.impl.TestPipelinedSorter.writeData(TestPipelinedSorter.java:842)
at
org.apache.tez.runtime.library.common.sort.impl.TestPipelinedSorter.testExceedsKVWithMultiplePartitions(TestPipelinedSorter.java:279)
FAILED:
org.apache.tez.runtime.library.common.sort.impl.TestPipelinedSorter.testWithCustomComparator
Error Message:
/tmp/org.apache.tez.runtime.library.common.sort.impl.TestPipelinedSorter/output/ee6f275a-ed9f-44a8-95d5-1de748e6236c_0/file.out
Stack Trace:
java.io.FileNotFoundException:
/tmp/org.apache.tez.runtime.library.common.sort.impl.TestPipelinedSorter/output/ee6f275a-ed9f-44a8-95d5-1de748e6236c_0/file.out
at
org.apache.hadoop.fs.RawLocalFileSystem.open(RawLocalFileSystem.java:200)
at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:767)
at
org.apache.tez.runtime.library.common.sort.impl.TezMerger$DiskSegment.init(TezMerger.java:403)
at
org.apache.tez.runtime.library.common.sort.impl.TezMerger$MergeQueue.merge(TezMerger.java:760)
at
org.apache.tez.runtime.library.common.sort.impl.TezMerger$MergeQueue.merge(TezMerger.java:699)
at
org.apache.tez.runtime.library.common.sort.impl.TezMerger.merge(TezMerger.java:147)
at
org.apache.tez.runtime.library.common.sort.impl.PipelinedSorter.flush(PipelinedSorter.java:791)
at
org.apache.tez.runtime.library.common.sort.impl.TestPipelinedSorter.closeSorter(TestPipelinedSorter.java:880)
at
org.apache.tez.runtime.library.common.sort.impl.TestPipelinedSorter.writeData(TestPipelinedSorter.java:874)
at
org.apache.tez.runtime.library.common.sort.impl.TestPipelinedSorter.writeData(TestPipelinedSorter.java:842)
at
org.apache.tez.runtime.library.common.sort.impl.TestPipelinedSorter.basicTest(TestPipelinedSorter.java:534)
at
org.apache.tez.runtime.library.common.sort.impl.TestPipelinedSorter.testWithCustomComparator(TestPipelinedSorter.java:390)
FAILED:
org.apache.tez.runtime.library.common.sort.impl.TestPipelinedSorter.test_TEZ_2602_50mb
Error Message:
/tmp/org.apache.tez.runtime.library.common.sort.impl.TestPipelinedSorter/output/29a14fe6-1fde-453b-b598-6a0db1138bbf_0/file.out
Stack Trace:
java.io.FileNotFoundException:
/tmp/org.apache.tez.runtime.library.common.sort.impl.TestPipelinedSorter/output/29a14fe6-1fde-453b-b598-6a0db1138bbf_0/file.out
at
org.apache.hadoop.fs.RawLocalFileSystem.open(RawLocalFileSystem.java:200)
at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:767)
at
org.apache.tez.runtime.library.common.sort.impl.TezMerger$DiskSegment.init(TezMerger.java:403)
at
org.apache.tez.runtime.library.common.sort.impl.TezMerger$MergeQueue.merge(TezMerger.java:760)
at
org.apache.tez.runtime.library.common.sort.impl.TezMerger$MergeQueue.merge(TezMerger.java:699)
at
org.apache.tez.runtime.library.common.sort.impl.TezMerger.merge(TezMerger.java:147)
at
org.apache.tez.runtime.library.common.sort.impl.PipelinedSorter.flush(PipelinedSorter.java:791)
at
org.apache.tez.runtime.library.common.sort.impl.TestPipelinedSorter.test_TEZ_2602_50mb(TestPipelinedSorter.java:321)
FAILED:
org.apache.tez.runtime.library.common.writers.TestUnorderedPartitionedKVWriter.testSingleSpill_WithPipelinedShuffle[test[false,
DISABLED]]
Error Message:
Could not find any valid local directory for
output/817a5cd6-83e7-43a4-b529-640bf3e5914c_0/file.out
Stack Trace:
org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find any valid
local directory for output/817a5cd6-83e7-43a4-b529-640bf3e5914c_0/file.out
at
org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPathForWrite(LocalDirAllocator.java:402)
at
org.apache.hadoop.fs.LocalDirAllocator.getLocalPathForWrite(LocalDirAllocator.java:150)
at
org.apache.hadoop.fs.LocalDirAllocator.getLocalPathForWrite(LocalDirAllocator.java:131)
at
org.apache.tez.runtime.library.common.task.local.output.TezTaskOutputFiles.getSpillFileForWrite(TezTaskOutputFiles.java:211)
at
org.apache.tez.runtime.library.common.writers.TestUnorderedPartitionedKVWriter.baseTestWithPipelinedTransfer(TestUnorderedPartitionedKVWriter.java:803)
at
org.apache.tez.runtime.library.common.writers.TestUnorderedPartitionedKVWriter.testSingleSpill_WithPipelinedShuffle(TestUnorderedPartitionedKVWriter.java:616)
FAILED:
org.apache.tez.runtime.library.common.writers.TestUnorderedPartitionedKVWriter.testMultipleSpills_WithPipelinedShuffle[test[true,
DISABLED]]
Error Message:
test timed out after 10000 milliseconds
Stack Trace:
java.lang.Exception: test timed out after 10000 milliseconds
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at
org.apache.tez.runtime.library.common.writers.UnorderedPartitionedKVWriter.getNextAvailableBuffer(UnorderedPartitionedKVWriter.java:545)
at
org.apache.tez.runtime.library.common.writers.UnorderedPartitionedKVWriter.setupNextBuffer(UnorderedPartitionedKVWriter.java:470)
at
org.apache.tez.runtime.library.common.writers.UnorderedPartitionedKVWriter.write(UnorderedPartitionedKVWriter.java:416)
at
org.apache.tez.runtime.library.common.writers.UnorderedPartitionedKVWriter.write(UnorderedPartitionedKVWriter.java:368)
at
org.apache.tez.runtime.library.common.writers.TestUnorderedPartitionedKVWriter.baseTestWithPipelinedTransfer(TestUnorderedPartitionedKVWriter.java:694)
at
org.apache.tez.runtime.library.common.writers.TestUnorderedPartitionedKVWriter.testMultipleSpills_WithPipelinedShuffle(TestUnorderedPartitionedKVWriter.java:621)
FAILED:
org.apache.tez.runtime.library.common.writers.TestUnorderedPartitionedKVWriter.testLargevalues[test[true,
DISABLED]]
Error Message:
chmod: cannot access
'/tmp/TestUnorderedPartitionedKVWriter/outDir_5280a0e9-886d-4d66-beeb-1c3829908f69/output/5280a0e9-886d-4d66-beeb-1c3829908f69_1/file.out':
No such file or directory
Stack Trace:
org.apache.hadoop.util.Shell$ExitCodeException: chmod: cannot access
'/tmp/TestUnorderedPartitionedKVWriter/outDir_5280a0e9-886d-4d66-beeb-1c3829908f69/output/5280a0e9-886d-4d66-beeb-1c3829908f69_1/file.out':
No such file or directory
at org.apache.hadoop.util.Shell.runCommand(Shell.java:545)
at org.apache.hadoop.util.Shell.run(Shell.java:456)
at
org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:722)
at org.apache.hadoop.util.Shell.execCommand(Shell.java:815)
at org.apache.hadoop.util.Shell.execCommand(Shell.java:798)
at
org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:731)
at
org.apache.tez.runtime.library.common.writers.UnorderedPartitionedKVWriter.writeLargeRecord(UnorderedPartitionedKVWriter.java:1086)
at
org.apache.tez.runtime.library.common.writers.UnorderedPartitionedKVWriter.write(UnorderedPartitionedKVWriter.java:412)
at
org.apache.tez.runtime.library.common.writers.UnorderedPartitionedKVWriter.write(UnorderedPartitionedKVWriter.java:368)
at
org.apache.tez.runtime.library.common.writers.TestUnorderedPartitionedKVWriter.textTest(TestUnorderedPartitionedKVWriter.java:424)
at
org.apache.tez.runtime.library.common.writers.TestUnorderedPartitionedKVWriter.testLargevalues(TestUnorderedPartitionedKVWriter.java:309)
