[
https://issues.apache.org/jira/browse/TEZ-3924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16460534#comment-16460534
]
Rajesh Balamohan commented on TEZ-3924:
---------------------------------------
>> For this scenario, DefaultSorter appends key values and never triggers
>> intermediate spill so sort is delayed until final flush
Right, this is true for both DefaultSorter and PipelinedSorter.
>> Flush is called and a value of "sameKeys=0" is saved for the writer.
This is a bug. It is possible to have non-zero "sameKeys" after sorting is
done. But currently it ends up accounting as "0" based on old assignment. This
needs to be fixed in DefaultSorter. This issue is not present in
pipelinedsorter.
> TestDefaultSorter fails intermittently due random keys and interaction with
> RLE and partition collisions
> ---------------------------------------------------------------------------------------------------------
>
> Key: TEZ-3924
> URL: https://issues.apache.org/jira/browse/TEZ-3924
> Project: Apache Tez
> Issue Type: Bug
> Reporter: Jonathan Eagles
> Assignee: Jonathan Eagles
> Priority: Major
> Attachments: TEZ-3924.001.patch
>
>
> Example of failure stack trace
> {noformat}
> Failed
> org.apache.tez.runtime.library.common.sort.impl.dflt.TestDefaultSorter.testEmptyCaseFileLengths
> Failing for the past 1 build (Since Failed#671 )
> Took 67 ms.
> (no description)
> Error Message
> Number of empty partitions did not match!
> Stacktrace
> java.lang.AssertionError: Number of empty partitions did not match!
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at
> org.apache.tez.runtime.library.common.sort.impl.dflt.TestDefaultSorter.testEmptyCaseFileLengthsHelper(TestDefaultSorter.java:321)
> at
> org.apache.tez.runtime.library.common.sort.impl.dflt.TestDefaultSorter.testEmptyCaseFileLengths(TestDefaultSorter.java:288)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
