Github user kiszk commented on a diff in the pull request:
https://github.com/apache/spark/pull/17122#discussion_r103857751
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/basicPhysicalOperators.scala
---
@@ -434,6 +434,17 @@ case class RangeExec(range:
org.apache.spark.sql.catalyst.plans.logical.Range)
val input = ctx.freshName("input")
// Right now, Range is only used when there is one upstream.
ctx.addMutableState("scala.collection.Iterator", input, s"$input =
inputs[0];")
+
+ val localIdx = ctx.freshName("localIdx")
+ val localEnd = ctx.freshName("localEnd")
+ val range = ctx.freshName("range")
+ // we need to place consume() before calling isShouldStopRequired
+ val body = consume(ctx, Seq(ev))
+ val shouldStop = if (isShouldStopRequired) {
--- End diff --
I think that `isShouldStopRequired` is simple logic. It just checks whether
`shouldStopRequired` or parents `shouldStopRequired` is true
There are two reasons why `isShouldStopRequired` is necessary.
1. The improvement is largely degraded from 7.6x to 5.5x without
`isShouldStopRequired`
2. We may miss some opportunities to enable compiler optimizations since
the size of loop body would be increased without `isShouldStopRequired`. This
is because a JIT compiler has a threshold of loop body size to apply some loop
optimizations such as loop unrolling.
```
OpenJDK 64-Bit Server VM 1.8.0_111-8u111-b14-2ubuntu0.16.04.2-b14 on Linux
4.4.0-47-generic
Intel(R) Xeon(R) CPU E5-2667 v3 @ 3.20GHz
cnt: Best/Avg Time(ms) Rate(M/s)
Per Row(ns) Relative
------------------------------------------------------------------------------------------------
cnt 247 / 289 4340.6
0.2 1.0X
```
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
