I'm sorry that I missed this posting. Given our PUSH based architecture, I am not sure LIMIT operator can "finalize/stop" frames from its input side. To remedy this issue, we are working on LIMIT pushdown, which passes LIMIT information to an index-search or external-sort so that only certain number of tuples will be generated from the source.
Best, Taewoo On Wed, Dec 30, 2015 at 11:09 AM, Chen Li <[email protected]> wrote: > Maybe Jianfeng and Taewoo can provide some input? > > On Sun, Dec 27, 2015 at 5:41 AM, Wail Alkowaileet <[email protected]> > wrote: > > Hi all, > > > > I was working on a large external dataset and when I query something with > > LIMIT clause, I see that the job waits for the > > ExternalDataScanOperatorDescriptor to finish scanning the whole file. > > > > When I looked at the plan, I found that the limit operator described > > in StreamLimitRuntimeFactory. > > The implementation only skips the frames when the limit expression > reached > > and it doesn't *finalize/stop* the source operator from pushing frames, > > which halts the overall job. > > > > public void nextFrame(ByteBuffer buffer) throws > > HyracksDataException { > > if (afterLastTuple) { > > return; //Accepting frames without appending it to > the > > parent operator > > } > > > > I think that applied for any source operator. > > > > > > > > -- > > > > *Regards,* > > Wail Alkowaileet >
