I believe kill() will only stop the upstream fragments from sending batches, but it does nothing about the batches that have already been sent. When kill() is called on the RawBatchBuffer, this will release all of the batches in the queue. But I believe it is still necessary to wait for all remaining batches to arrive, so they can be cleared. It's possible that it's not necessary to do this in LimitRecordBatch, and that we are handling this in the RawBatchBuffer. You would have to examine the code to confirm.
On Thu, Oct 29, 2015 at 11:38 AM, Abdel Hakim Deneche <[email protected] > wrote: > Hey all, > > As part of DRILL-991 when LimitRecordBatch receives enough records it calls > kill() on it's upstream to inform the remaining operators and fragments > that they can stop sending batches. > > But, limit operator will keep calling next() until it gets a NONE. Is there > a specific reason for this behavior ? > > Thanks > > -- > > Abdelhakim Deneche > > Software Engineer > > <http://www.mapr.com/> > > > Now Available - Free Hadoop On-Demand Training > < > http://www.mapr.com/training?utm_source=Email&utm_medium=Signature&utm_campaign=Free%20available > > >
