Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/22524#discussion_r220040370
--- Diff:
sql/core/src/main/java/org/apache/spark/sql/execution/BufferedRowIterator.java
---
@@ -38,6 +38,11 @@
protected int partitionIndex = -1;
+ // This indicates whether the query execution should be stopped even the
input rows are still
+ // available. This is used in limit operator. When it reaches the given
number of rows to limit,
+ // this flag is set and the execution should be stopped.
+ protected boolean isStopEarly = false;
--- End diff --
I've added a test for 2 limits.
When any of 2 limits sets `isStopEarly`, I think the execution should be
stopped. Is there any case opposite to this?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
