[GitHub] spark pull request: [SPARK-4226][SQL] Support Correlated Sub-queri...

2016-04-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12306#issuecomment-211583586 **[Test build #56138 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/56138/consoleFull)** for PR 12306 at commit [`471bb93`](https://gi

[GitHub] spark pull request: [SPARK-4226][SQL] Support Correlated Sub-queri...

2016-04-18 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/12306#discussion_r60135788 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -1447,3 +1450,133 @@ object EmbedSerializerInFilter

[GitHub] spark pull request: [SPARK-4226][SQL] Support Correlated Sub-queri...

2016-04-18 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/12306#discussion_r60135763 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/subquery.scala --- @@ -77,3 +79,81 @@ case class ScalarSubquery(

[GitHub] spark pull request: [SPARK-4226][SQL] Support Correlated Sub-queri...

2016-04-18 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/12306#discussion_r60135581 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/CheckAnalysis.scala --- @@ -110,6 +110,31 @@ trait CheckAnalysis {

[GitHub] spark pull request: [SPARK-4226][SQL] Support Correlated Sub-queri...

2016-04-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12306#issuecomment-211580526 **[Test build #56136 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/56136/consoleFull)** for PR 12306 at commit [`e041d07`](https://gi

[GitHub] spark pull request: [SPARK-4226][SQL] Support Correlated Sub-queri...

2016-04-18 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/12306#discussion_r60134928 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/CheckAnalysis.scala --- @@ -110,6 +110,31 @@ trait CheckAnalysis {

[GitHub] spark pull request: [SPARK-4226][SQL] Support Correlated Sub-queri...

2016-04-18 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/12306#discussion_r60128005 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -1447,3 +1450,133 @@ object EmbedSerializerInFilter exte

[GitHub] spark pull request: [SPARK-4226][SQL] Support Correlated Sub-queri...

2016-04-18 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/12306#discussion_r60125483 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -1447,3 +1450,133 @@ object EmbedSerializerInFilter

[GitHub] spark pull request: [SPARK-4226][SQL] Support Correlated Sub-queri...

2016-04-18 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/12306#discussion_r60120133 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -1447,3 +1450,133 @@ object EmbedSerializerInFilter exte

[GitHub] spark pull request: [SPARK-4226][SQL] Support Correlated Sub-queri...

2016-04-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12306#issuecomment-211539603 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-4226][SQL] Support Correlated Sub-queri...

2016-04-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12306#issuecomment-211539601 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-4226][SQL] Support Correlated Sub-queri...

2016-04-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12306#issuecomment-211538928 **[Test build #56086 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/56086/consoleFull)** for PR 12306 at commit [`b473240`](https://g

[GitHub] spark pull request: [SPARK-4226][SQL] Support Correlated Sub-queri...

2016-04-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12306#issuecomment-211487778 **[Test build #56086 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/56086/consoleFull)** for PR 12306 at commit [`b473240`](https://gi

[GitHub] spark pull request: [SPARK-4226][SQL] Support Correlated Sub-queri...

2016-04-18 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/12306#discussion_r60097133 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/subquery.scala --- @@ -77,3 +79,81 @@ case class ScalarSubquery(

[GitHub] spark pull request: [SPARK-4226][SQL] Support Correlated Sub-queri...

2016-04-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12306#issuecomment-211397398 **[Test build #2810 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/2810/consoleFull)** for PR 12306 at commit [`b473240`](https://

[GitHub] spark pull request: [SPARK-4226][SQL] Support Correlated Sub-queri...

2016-04-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12306#issuecomment-211365407 **[Test build #2810 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/2810/consoleFull)** for PR 12306 at commit [`b473240`](https://g

[GitHub] spark pull request: [SPARK-4226][SQL] Support Correlated Sub-queri...

2016-04-18 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/12306#discussion_r60024110 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -1447,3 +1450,133 @@ object EmbedSerializerInFilter

[GitHub] spark pull request: [SPARK-4226][SQL] Support Correlated Sub-queri...

2016-04-18 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/12306#discussion_r60023217 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/subquery.scala --- @@ -77,3 +79,81 @@ case class ScalarSubquery(

[GitHub] spark pull request: [SPARK-4226][SQL] Support Correlated Sub-queri...

2016-04-17 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/12306#discussion_r60010812 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -1447,3 +1450,133 @@ object EmbedSerializerInFilter exte

[GitHub] spark pull request: [SPARK-4226][SQL] Support Correlated Sub-queri...

2016-04-17 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/12306#discussion_r60010483 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -1447,3 +1450,133 @@ object EmbedSerializerInFilter exte

[GitHub] spark pull request: [SPARK-4226][SQL] Support Correlated Sub-queri...

2016-04-17 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/12306#discussion_r60010345 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -1447,3 +1450,133 @@ object EmbedSerializerInFilter exte

[GitHub] spark pull request: [SPARK-4226][SQL] Support Correlated Sub-queri...

2016-04-17 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/12306#discussion_r60010283 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -1447,3 +1450,133 @@ object EmbedSerializerInFilter exte

[GitHub] spark pull request: [SPARK-4226][SQL] Support Correlated Sub-queri...

2016-04-17 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/12306#discussion_r60010078 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/CheckAnalysis.scala --- @@ -110,6 +110,31 @@ trait CheckAnalysis {

[GitHub] spark pull request: [SPARK-4226][SQL] Support Correlated Sub-queri...

2016-04-17 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/12306#discussion_r60009754 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -1447,3 +1450,133 @@ object EmbedSerializerInFilter exte

[GitHub] spark pull request: [SPARK-4226][SQL] Support Correlated Sub-queri...

2016-04-17 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/12306#discussion_r60009505 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/CheckAnalysis.scala --- @@ -110,6 +110,31 @@ trait CheckAnalysis {

[GitHub] spark pull request: [SPARK-4226][SQL] Support Correlated Sub-queri...

2016-04-17 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/12306#discussion_r60009438 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -1447,3 +1450,133 @@ object EmbedSerializerInFilter exte

[GitHub] spark pull request: [SPARK-4226][SQL] Support Correlated Sub-queri...

2016-04-17 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/12306#discussion_r60009316 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/CheckAnalysis.scala --- @@ -110,6 +110,31 @@ trait CheckAnalysis {

[GitHub] spark pull request: [SPARK-4226][SQL] Support Correlated Sub-queri...

2016-04-17 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/12306#discussion_r60008722 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/subquery.scala --- @@ -77,3 +79,81 @@ case class ScalarSubquery(

[GitHub] spark pull request: [SPARK-4226][SQL] Support Correlated Sub-queri...

2016-04-17 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/12306#discussion_r60008703 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/CheckAnalysis.scala --- @@ -110,6 +110,31 @@ trait CheckAnalysis {

[GitHub] spark pull request: [SPARK-4226][SQL] Support Correlated Sub-queri...

2016-04-17 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/12306#discussion_r60008557 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/CheckAnalysis.scala --- @@ -110,6 +110,31 @@ trait CheckAnalysis {

[GitHub] spark pull request: [SPARK-4226][SQL] Support Correlated Sub-queri...

2016-04-17 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/12306#discussion_r60008476 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/CheckAnalysis.scala --- @@ -110,6 +110,31 @@ trait CheckAnalysis {

[GitHub] spark pull request: [SPARK-4226][SQL] Support Correlated Sub-queri...

2016-04-17 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/12306#discussion_r60008271 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/CheckAnalysis.scala --- @@ -110,6 +110,31 @@ trait CheckAnalysis {

[GitHub] spark pull request: [SPARK-4226][SQL] Support Correlated Sub-queri...

2016-04-17 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/12306#discussion_r60008194 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/CheckAnalysis.scala --- @@ -110,6 +110,31 @@ trait CheckAnalysis {

[GitHub] spark pull request: [SPARK-4226][SQL] Support Correlated Sub-queri...

2016-04-17 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/12306#discussion_r60007877 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/CheckAnalysis.scala --- @@ -110,6 +110,31 @@ trait CheckAnalysis {

[GitHub] spark pull request: [SPARK-4226][SQL] Support Correlated Sub-queri...

2016-04-17 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/12306#discussion_r60007659 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/CheckAnalysis.scala --- @@ -110,6 +110,31 @@ trait CheckAnalysis {

[GitHub] spark pull request: [SPARK-4226][SQL] Support Correlated Sub-queri...

2016-04-17 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/12306#discussion_r60007367 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/CheckAnalysis.scala --- @@ -110,6 +110,31 @@ trait CheckAnalysis {

[GitHub] spark pull request: [SPARK-4226][SQL] Support Correlated Sub-queri...

2016-04-17 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/12306#discussion_r60007268 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/CheckAnalysis.scala --- @@ -110,6 +110,31 @@ trait CheckAnalysis {

[GitHub] spark pull request: [SPARK-4226][SQL] Support Correlated Sub-queri...

2016-04-17 Thread hvanhovell
Github user hvanhovell commented on the pull request: https://github.com/apache/spark/pull/12306#issuecomment-211199893 Retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-4226][SQL] Support Correlated Sub-queri...

2016-04-17 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12306#issuecomment-211136525 **[Test build #2802 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/2802/consoleFull)** for PR 12306 at commit [`b473240`](https://

[GitHub] spark pull request: [SPARK-4226][SQL] Support Correlated Sub-queri...

2016-04-17 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12306#issuecomment-211126406 **[Test build #2802 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/2802/consoleFull)** for PR 12306 at commit [`b473240`](https://g

[GitHub] spark pull request: [SPARK-4226][SQL] Support Correlated Sub-queri...

2016-04-17 Thread hvanhovell
Github user hvanhovell commented on the pull request: https://github.com/apache/spark/pull/12306#issuecomment-211122925 @rxin / @davies this one is ready for review. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-4226][SQL] Support Correlated Sub-queri...

2016-04-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12306#issuecomment-210910928 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-4226][SQL] Support Correlated Sub-queri...

2016-04-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12306#issuecomment-210910931 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-4226][SQL] Support Correlated Sub-queri...

2016-04-16 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12306#issuecomment-210910762 **[Test build #56017 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/56017/consoleFull)** for PR 12306 at commit [`b29f741`](https://g

[GitHub] spark pull request: [SPARK-4226][SQL] Support Correlated Sub-queri...

2016-04-16 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12306#issuecomment-210888688 **[Test build #56017 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/56017/consoleFull)** for PR 12306 at commit [`b29f741`](https://gi

[GitHub] spark pull request: [SPARK-4226][SQL] Support Correlated Sub-queri...

2016-04-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12306#issuecomment-210878434 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-4226][SQL] Support Correlated Sub-queri...

2016-04-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12306#issuecomment-210878432 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-4226][SQL] Support Correlated Sub-queri...

2016-04-16 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12306#issuecomment-210878380 **[Test build #56013 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/56013/consoleFull)** for PR 12306 at commit [`64921a2`](https://g

[GitHub] spark pull request: [SPARK-4226][SQL] Support Correlated Sub-queri...

2016-04-16 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12306#issuecomment-210867437 **[Test build #56013 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/56013/consoleFull)** for PR 12306 at commit [`64921a2`](https://gi

[GitHub] spark pull request: [SPARK-4226][SQL] Support Correlated Sub-queri...

2016-04-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12306#issuecomment-210664740 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-4226][SQL] Support Correlated Sub-queri...

2016-04-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12306#issuecomment-210664743 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-4226][SQL] Support Correlated Sub-queri...

2016-04-15 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12306#issuecomment-210664630 **[Test build #55958 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/55958/consoleFull)** for PR 12306 at commit [`f016ace`](https://g

[GitHub] spark pull request: [SPARK-4226][SQL] Support Correlated Sub-queri...

2016-04-15 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/12306#discussion_r59942854 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/subquery.scala --- @@ -77,3 +79,53 @@ case class ScalarSubquery(

[GitHub] spark pull request: [SPARK-4226][SQL] Support Correlated Sub-queri...

2016-04-15 Thread davies
Github user davies commented on the pull request: https://github.com/apache/spark/pull/12306#issuecomment-210648932 @hvanhovell Since you separate resolution and rewrite, it's easy to move the rewrite rule into optimizer, right (those checking could be moved to CheckAnalysis)? By defe

[GitHub] spark pull request: [SPARK-4226][SQL] Support Correlated Sub-queri...

2016-04-15 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/12306#discussion_r59941599 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -852,25 +855,148 @@ class Analyzer( }

[GitHub] spark pull request: [SPARK-4226][SQL] Support Correlated Sub-queri...

2016-04-15 Thread hvanhovell
Github user hvanhovell commented on the pull request: https://github.com/apache/spark/pull/12306#issuecomment-210642462 @davies after all splitting the rule made reasoning about sub-queries easier. So now we have a rule for resolution and one for rewriting. The rewriting is cu

[GitHub] spark pull request: [SPARK-4226][SQL] Support Correlated Sub-queri...

2016-04-15 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12306#issuecomment-210642377 **[Test build #55958 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/55958/consoleFull)** for PR 12306 at commit [`f016ace`](https://gi

[GitHub] spark pull request: [SPARK-4226][SQL] Support Correlated Sub-queri...

2016-04-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12306#issuecomment-210192233 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-4226][SQL] Support Correlated Sub-queri...

2016-04-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12306#issuecomment-210192238 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-4226][SQL] Support Correlated Sub-queri...

2016-04-14 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12306#issuecomment-210192012 **[Test build #55848 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/55848/consoleFull)** for PR 12306 at commit [`c0f6b8d`](https://g

[GitHub] spark pull request: [SPARK-4226][SQL] Support Correlated Sub-queri...

2016-04-14 Thread davies
Github user davies commented on the pull request: https://github.com/apache/spark/pull/12306#issuecomment-210174437 The way to run an uncorrelated EXISTS/IN could be changed (we may also use that for dynamic partition pruning), keep them as predicate enable us to do predicate push dow

[GitHub] spark pull request: [SPARK-4226][SQL] Support Correlated Sub-queri...

2016-04-14 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/12306#discussion_r59799685 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -852,26 +852,142 @@ class Analyzer( }

[GitHub] spark pull request: [SPARK-4226][SQL] Support Correlated Sub-queri...

2016-04-14 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/12306#discussion_r59799251 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -852,26 +852,142 @@ class Analyzer( }

[GitHub] spark pull request: [SPARK-4226][SQL] Support Correlated Sub-queri...

2016-04-14 Thread hvanhovell
Github user hvanhovell commented on the pull request: https://github.com/apache/spark/pull/12306#issuecomment-210168565 added WIP tag because I am still working on a nicer EXISTS rule. --- If your project is set up for it, you can reply to this email and have your reply appear on GitH

[GitHub] spark pull request: [SPARK-4226][SQL] Support Correlated Sub-queri...

2016-04-14 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/12306#discussion_r59798418 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -852,26 +852,142 @@ class Analyzer( }

[GitHub] spark pull request: [SPARK-4226][SQL] Support Correlated Sub-queri...

2016-04-14 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/12306#discussion_r59798253 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -852,26 +852,150 @@ class Analyzer( }

[GitHub] spark pull request: [SPARK-4226][SQL] Support Correlated Sub-queri...

2016-04-14 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/12306#discussion_r59798226 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -852,26 +852,142 @@ class Analyzer( }

[GitHub] spark pull request: [SPARK-4226][SQL] Support Correlated Sub-queri...

2016-04-14 Thread hvanhovell
Github user hvanhovell commented on the pull request: https://github.com/apache/spark/pull/12306#issuecomment-210167691 @davies I do agree that it is better to do the rewriting later on. I don't think SQL generation is too much of a problem, since semi and anti joins are supported by

[GitHub] spark pull request: [SPARK-4226][SQL] Support Correlated Sub-queri...

2016-04-14 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12306#issuecomment-210153552 **[Test build #55848 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/55848/consoleFull)** for PR 12306 at commit [`c0f6b8d`](https://gi

[GitHub] spark pull request: [SPARK-4226][SQL] Support Correlated Sub-queri...

2016-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12306#issuecomment-208594859 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-4226][SQL] Support Correlated Sub-queri...

2016-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12306#issuecomment-208594869 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-4226][SQL] Support Correlated Sub-queri...

2016-04-11 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12306#issuecomment-208594618 **[Test build #55538 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/55538/consoleFull)** for PR 12306 at commit [`082bb86`](https://g

[GitHub] spark pull request: [SPARK-4226][SQL] Support Correlated Sub-queri...

2016-04-11 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/12306#discussion_r59289073 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -665,7 +665,7 @@ class Analyzer( def apply(pl

[GitHub] spark pull request: [SPARK-4226][SQL] Support Correlated Sub-queri...

2016-04-11 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/12306#discussion_r59288445 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -852,26 +852,150 @@ class Analyzer( } /*

[GitHub] spark pull request: [SPARK-4226][SQL] Support Correlated Sub-queri...

2016-04-11 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/12306#discussion_r59288302 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -852,26 +852,150 @@ class Analyzer( } /*

[GitHub] spark pull request: [SPARK-4226][SQL] Support Correlated Sub-queri...

2016-04-11 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/12306#discussion_r59288229 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/SQLQuerySuite.scala --- @@ -2468,4 +2468,72 @@ class SQLQuerySuite extends QueryTest with SharedSQLConte

[GitHub] spark pull request: [SPARK-4226][SQL] Support Correlated Sub-queri...

2016-04-11 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/12306#discussion_r59288145 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/subquery.scala --- @@ -77,3 +77,24 @@ case class ScalarSubquery( o

[GitHub] spark pull request: [SPARK-4226][SQL] Support Correlated Sub-queri...

2016-04-11 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/12306#discussion_r59288093 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/subquery.scala --- @@ -77,3 +77,24 @@ case class ScalarSubquery( o

[GitHub] spark pull request: [SPARK-4226][SQL] Support Correlated Sub-queri...

2016-04-11 Thread davies
Github user davies commented on the pull request: https://github.com/apache/spark/pull/12306#issuecomment-208577343 @hvanhovell Thanks for taking over these! In the first prototype, I did the rewriting in analyzer, later I realize that it may make more sense to do that in Opti

[GitHub] spark pull request: [SPARK-4226][SQL] Support Correlated Sub-queri...

2016-04-11 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/12306#discussion_r59288009 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/subquery.scala --- @@ -77,3 +77,24 @@ case class ScalarSubquery( o

[GitHub] spark pull request: [SPARK-4226][SQL] Support Correlated Sub-queri...

2016-04-11 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/12306#discussion_r59287005 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -665,7 +665,7 @@ class Analyzer( def apply

[GitHub] spark pull request: [SPARK-4226][SQL] Support Correlated Sub-queri...

2016-04-11 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12306#issuecomment-208561088 **[Test build #55538 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/55538/consoleFull)** for PR 12306 at commit [`082bb86`](https://gi

[GitHub] spark pull request: [SPARK-4226][SQL] Support Correlated Sub-queri...

2016-04-11 Thread hvanhovell
GitHub user hvanhovell opened a pull request: https://github.com/apache/spark/pull/12306 [SPARK-4226][SQL] Support Correlated Sub-queries ### What changes were proposed in this pull request? This PR adds support for correlated subqueries to Spark. Correlated sub-queries are typi