LIMIT push down doesn't really LIMIT the source op.

[Wail Alkowaileet]

Hi all,

I was working on a large external dataset and when I query something with
LIMIT clause, I see that the job waits for the
ExternalDataScanOperatorDescriptor to finish scanning the whole file.

When I looked at the plan, I found that the limit operator described
in StreamLimitRuntimeFactory.
The implementation only skips the frames when the limit expression reached
and it doesn't *finalize/stop* the source operator from pushing frames,
which halts the overall job.

            public void nextFrame(ByteBuffer buffer) throws
HyracksDataException {
                if (afterLastTuple) {
                    return; //Accepting frames without appending it to the
parent operator
                }

I think that applied for any source operator.



-- 

*Regards,*
Wail Alkowaileet