[
https://issues.apache.org/jira/browse/TEZ-3927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Eagles updated TEZ-3927:
---------------------------------
Target Version/s: 0.9.2, 0.10.0 (was: 0.10.0)
> TestReduceProcessor fails on Hadoop 3.x
> ---------------------------------------
>
> Key: TEZ-3927
> URL: https://issues.apache.org/jira/browse/TEZ-3927
> Project: Apache Tez
> Issue Type: Sub-task
> Reporter: Jonathan Eagles
> Assignee: Jonathan Eagles
> Priority: Major
> Attachments: TEZ-3927.001.patch
>
>
> TestReduceProcessor makes an assumption about the location of pre committed
> output files based on fileoutputcommiter version 1. In hadoop3, the default
> fileoutputcommiter version moved to 2, changing the output file locations.
> {noformat}
> org.apache.tez.mapreduce.processor.reduce.TestReduceProcessor.testReduceProcessor
> Failing for the past 1 build (Since Failed#2782 )
> Took 1 sec.
> Error Message
> File
> file:/tmp/TestReduceProcessor/output/_temporary/0/task_01_0000_r_000000/part-v001-o000-00000
> does not exist
> Stacktrace
> java.io.FileNotFoundException: File
> file:/tmp/TestReduceProcessor/output/_temporary/0/task_01_0000_r_000000/part-v001-o000-00000
> does not exist
> at
> org.apache.hadoop.fs.RawLocalFileSystem.deprecatedGetFileStatus(RawLocalFileSystem.java:641)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.getFileLinkStatusInternal(RawLocalFileSystem.java:867)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:631)
> at
> org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:442)
> at
> org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1863)
> at
> org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1886)
> at
> org.apache.tez.mapreduce.processor.reduce.TestReduceProcessor.testReduceProcessor(TestReduceProcessor.java:259)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
