[
https://issues.apache.org/jira/browse/DRILL-5325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16027097#comment-16027097
]
ASF GitHub Bot commented on DRILL-5325:
---------------------------------------
Github user Ben-Zvi commented on a diff in the pull request:
https://github.com/apache/drill/pull/808#discussion_r118407555
--- Diff:
exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/xsort/managed/ExternalSortBatch.java
---
@@ -672,50 +350,74 @@ private IterOutcome load() {
// Anything to actually sort?
- if (inputRecordCount == 0) {
+ resultsIterator = sortImpl.startMerge();
+ if (! resultsIterator.next()) {
sortState = SortState.DONE;
return IterOutcome.NONE;
}
- logger.debug("Completed load phase: read {} batches, spilled {} times,
total input bytes: {}",
- inputBatchCount, spilledRuns.size(), totalInputBytes);
- // Do the merge of the loaded batches. The merge can be done entirely
in memory if
- // the results fit; else we have to do a disk-based merge of
- // pre-sorted spilled batches.
+ // sort may have prematurely exited due to should continue returning
false.
--- End diff --
This comment is a little confusing ... due to ? ..... return false or
return STOP ?
> Implement sub-operator unit tests for managed external sort
> -----------------------------------------------------------
>
> Key: DRILL-5325
> URL: https://issues.apache.org/jira/browse/DRILL-5325
> Project: Apache Drill
> Issue Type: Improvement
> Components: Tools, Build & Test
> Affects Versions: 1.11.0
> Reporter: Paul Rogers
> Assignee: Paul Rogers
> Fix For: 1.11.0
>
>
> Validate the proposed sub-operator test framework, by creating low-level unit
> tests for the managed version of the external sort.
> The external sort has a small number of existing tests, but those tests are
> quite superficial; the "managed sort" project found many bugs. The managed
> sort itself was tested with ad-hoc system-level tests created using the new
> "cluster fixture" framework. But, again, such tests could not reach deep
> inside the sort code to exercise very specific conditions.
> As a result, we spent far too much time using QA functional tests to identify
> specific code issues.
> Using the sub-opeator unit test framework, we can instead test each bit of
> functionality at the unit test level.
> If doing so works, and is practical, it can serve as a model for other
> operator testing projects.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
