Github user mgaido91 commented on a diff in the pull request:
https://github.com/apache/spark/pull/22630#discussion_r223203167
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/SortExec.scala ---
@@ -132,6 +132,13 @@ case class SortExec(
// a stop check before sorting.
override def needStopCheck: Boolean = false
+ // Sort is a blocking operator. It needs to consume all the inputs
before producing any output.
+ // This means, Limit operator after Sort will never reach its limit
during the execution of Sort's
+ // upstream operators. Here we override this method to return Nil, so
that upstream operators will
+ // not generate useless conditions (which are always evaluated to false)
for the Limit operators
+ // after Sort.
+ override def limitNotReachedChecks: Seq[String] = Nil
--- End diff --
I am fine to do it later, but I'd like to avoid to have other places where
we duplicate this logic in the future in order to avoid possible mistakes.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
