[
https://issues.apache.org/jira/browse/DRILL-5083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15703762#comment-15703762
]
Paul Rogers edited comment on DRILL-5083 at 11/29/16 1:13 AM:
--------------------------------------------------------------
Stepping further, once the original {{MergeJoinBatch.close( )}} returned, it
was called a second time during {{FragmentExeucutor.closeOutResources()}}. The
whole bounce up & down occurs a second time.
But, the second time, in {{IteratorValidatorBatchIterator.next()}}, the
{{exceptionState}} member is (somehow) no longer set. This lets the operator
proceed as if there is no error. It proceeds to call:
{code}
batchState = incoming.next();
{code}
This eventually hits:
{code}
public final IterOutcome next(final RecordBatch b) {
if(!context.shouldContinue()) {
return IterOutcome.STOP; // <-- Ends up here
}
return next(0, b);
}
{code}
This all seems rather ad-hoc...
was (Author: paul-rogers):
Stepping further, once the original {{MergeJoinBatch.close( )}} returned, it
was called a second time during {{FragmentExeucutor.closeOutResources()}}. The
whole bounce up & down occurs a second time.
> IteratorValidator does not handle RecordIterator cleanup call to next( )
> ------------------------------------------------------------------------
>
> Key: DRILL-5083
> URL: https://issues.apache.org/jira/browse/DRILL-5083
> Project: Apache Drill
> Issue Type: Bug
> Affects Versions: 1.8.0
> Reporter: Paul Rogers
> Priority: Minor
>
> This one is very confusing...
> In a test with a MergeJoin and external sort, operators are stacked something
> like this:
> {code}
> Screen
> - MergeJoin
> - - External Sort
> ...
> {code}
> Using the injector to force a OOM in spill, the external sort threw a
> UserException up the stack. This was handed by:
> {code}
> IteratorValidatorBatchIterator.next( )
> RecordIterator.clearInflightBatches( )
> RecordIterator.close( )
> MergeJoinBatch.close( )
> {code}
> Which does the following:
> {code}
> // Check whether next() should even have been called in current state.
> if (null != exceptionState) {
> throw new IllegalStateException(
> {code}
> But, the exceptionState is set, so we end up throwing an
> IllegalStateException during cleanup.
> Seems the code should agree: if {{next( )}} will be called during cleanup,
> then {{next( )}} should gracefully handle that case.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
