[
https://issues.apache.org/jira/browse/MAPREDUCE-6165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14233944#comment-14233944
]
Akira AJISAKA commented on MAPREDUCE-6165:
------------------------------------------
The test asserted the location of the splits, which is defined by the iteration
order of {{HashMap.entrySet()}}. However, the order of JDK8 is different from
JDK7, so the location of the splits is different.
For example, JDK7 creates two split,
{code}
Made splits(Test1): 2
File split(Test1): Paths:/dir2/file2:0+1024,/dir2/file2:1024+1024
Locations:host2.rack2.com:;
File split(Test1): Paths:/dir1/file1:0+1024 Locations:host1.rack1.com:;
{code}
on the other hand, JDK8 creates one split in the same block placement
{code}
Made splits(Test1): 1
File split(Test1):
Paths:/dir1/file1:0+1024,/dir2/file2:0+1024,/dir2/file2:1024+1024
Locations:host1.rack1.com:;
{code}
The patch does not assert the locations. It asserts the number of the path and
the total length of the splits.
> [JDK8] TestCombineFileInputFormat failed on JDK8
> ------------------------------------------------
>
> Key: MAPREDUCE-6165
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6165
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Reporter: Wei Yan
> Assignee: Akira AJISAKA
> Priority: Minor
> Attachments: MAPREDUCE-6165-001.patch, MAPREDUCE-6165-reproduce.patch
>
>
> The error msg:
> {noformat}
> testSplitPlacementForCompressedFiles(org.apache.hadoop.mapreduce.lib.input.TestCombineFileInputFormat)
> Time elapsed: 2.487 sec <<< FAILURE!
> junit.framework.AssertionFailedError: expected:<2> but was:<1>
> at junit.framework.Assert.fail(Assert.java:57)
> at junit.framework.Assert.failNotEquals(Assert.java:329)
> at junit.framework.Assert.assertEquals(Assert.java:78)
> at junit.framework.Assert.assertEquals(Assert.java:234)
> at junit.framework.Assert.assertEquals(Assert.java:241)
> at junit.framework.TestCase.assertEquals(TestCase.java:409)
> at
> org.apache.hadoop.mapreduce.lib.input.TestCombineFileInputFormat.testSplitPlacementForCompressedFiles(TestCombineFileInputFormat.java:911)
> testSplitPlacement(org.apache.hadoop.mapreduce.lib.input.TestCombineFileInputFormat)
> Time elapsed: 0.985 sec <<< FAILURE!
> junit.framework.AssertionFailedError: expected:<2> but was:<1>
> at junit.framework.Assert.fail(Assert.java:57)
> at junit.framework.Assert.failNotEquals(Assert.java:329)
> at junit.framework.Assert.assertEquals(Assert.java:78)
> at junit.framework.Assert.assertEquals(Assert.java:234)
> at junit.framework.Assert.assertEquals(Assert.java:241)
> at junit.framework.TestCase.assertEquals(TestCase.java:409)
> at
> org.apache.hadoop.mapreduce.lib.input.TestCombineFileInputFormat.testSplitPlacement(TestCombineFileInputFormat.java:368)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
