[GitHub] [spark] AngersZhuuuu commented on issue #26437: [SPARK-29800][SQL] Plan non-correlated Exists 's subquery in PlanSubqueries

AngersZh commented on issue #26437: [SPARK-29800][SQL] Plan non-correlated Exists 's subquery in PlanSubqueries URL: https://github.com/apache/spark/pull/26437#issuecomment-570447062 > LGTM, waiting for the updated benchmark result. Great job @AngersZh ! Updated all, thanks

[GitHub] [spark] AngersZhuuuu commented on issue #26437: [SPARK-29800][SQL] Plan non-correlated Exists 's subquery in PlanSubqueries

AngersZh commented on issue #26437: [SPARK-29800][SQL] Plan non-correlated Exists 's subquery in PlanSubqueries URL: https://github.com/apache/spark/pull/26437#issuecomment-570445224 **With current pr** ``` scala> (1 to 1).toDF("id").createOrReplaceTempView("s1")

[GitHub] [spark] AngersZhuuuu commented on issue #26437: [SPARK-29800][SQL] Plan non-correlated Exists 's subquery in PlanSubqueries

AngersZh commented on issue #26437: [SPARK-29800][SQL] Plan non-correlated Exists 's subquery in PlanSubqueries URL: https://github.com/apache/spark/pull/26437#issuecomment-570132270 @cloud-fan Could you help to trigger retest?

[GitHub] [spark] AngersZhuuuu commented on issue #26437: [SPARK-29800][SQL] Plan non-correlated Exists 's subquery in PlanSubqueries

AngersZh commented on issue #26437: [SPARK-29800][SQL] Plan non-correlated Exists 's subquery in PlanSubqueries URL: https://github.com/apache/spark/pull/26437#issuecomment-570127420 > We don't need to add `Project(Nil, ...)` because we convert to `WHERE (SELECT 1 FROM (subquery)

[GitHub] [spark] AngersZhuuuu commented on issue #26437: [SPARK-29800][SQL] Plan non-correlated Exists 's subquery in PlanSubqueries

AngersZh commented on issue #26437: [SPARK-29800][SQL] Plan non-correlated Exists 's subquery in PlanSubqueries URL: https://github.com/apache/spark/pull/26437#issuecomment-570112465 > @AngersZh any updates? If use `ScalaSubquery` we can't add `Project(Nil,..)` because of

[GitHub] [spark] AngersZhuuuu commented on issue #26437: [SPARK-29800][SQL] Plan non-correlated Exists 's subquery in PlanSubqueries

AngersZh commented on issue #26437: [SPARK-29800][SQL] Plan non-correlated Exists 's subquery in PlanSubqueries URL: https://github.com/apache/spark/pull/26437#issuecomment-558476143 > For top-level limit, we use a special physical plan `CollectLimitExec`, which doesn't have a

[GitHub] [spark] AngersZhuuuu commented on issue #26437: [SPARK-29800][SQL] Plan non-correlated Exists 's subquery in PlanSubqueries

AngersZh commented on issue #26437: [SPARK-29800][SQL] Plan non-correlated Exists 's subquery in PlanSubqueries URL: https://github.com/apache/spark/pull/26437#issuecomment-558463297 > @AngersZh after some more thoughts, I think we can still reuse non-correlated scalar subquery.

[GitHub] [spark] AngersZhuuuu commented on issue #26437: [SPARK-29800][SQL] Plan non-correlated Exists 's subquery in PlanSubqueries

AngersZh commented on issue #26437: [SPARK-29800][SQL] Plan non-correlated Exists 's subquery in PlanSubqueries URL: https://github.com/apache/spark/pull/26437#issuecomment-558387275 > @AngersZh, I think there's probably some thing wrong with your latest master merge...

[GitHub] [spark] AngersZhuuuu commented on issue #26437: [SPARK-29800][SQL] Plan non-correlated Exists 's subquery in PlanSubqueries

AngersZh commented on issue #26437: [SPARK-29800][SQL] Plan non-correlated Exists 's subquery in PlanSubqueries URL: https://github.com/apache/spark/pull/26437#issuecomment-558386503 > hmmm why are there so many conflicts? Last merge master branch bring unexpected code,..

[GitHub] [spark] AngersZhuuuu commented on issue #26437: [SPARK-29800][SQL] Plan non-correlated Exists 's subquery in PlanSubqueries

AngersZh commented on issue #26437: [SPARK-29800][SQL] Plan non-correlated Exists 's subquery in PlanSubqueries URL: https://github.com/apache/spark/pull/26437#issuecomment-557636283 cc @cloud-fan Simply look at the calculation process, the calculation of non-correlated exists

[GitHub] [spark] AngersZhuuuu commented on issue #26437: [SPARK-29800][SQL] Plan non-correlated Exists 's subquery in PlanSubqueries

AngersZh commented on issue #26437: [SPARK-29800][SQL] Plan non-correlated Exists 's subquery in PlanSubqueries URL: https://github.com/apache/spark/pull/26437#issuecomment-553415168 > @AngersZh I think it's pretty close now. Can we have some benchmark to show the benefit of this