[GitHub] spark pull request: [SPARK-4226][SQL]Add subquery (not) in/exists ...

2015-12-31 Thread gatorsmile
Github user gatorsmile commented on the pull request: https://github.com/apache/spark/pull/9055#issuecomment-168152168 Found a related HIVE JIRA to support the left anti join: https://issues.apache.org/jira/browse/HIVE-12519 However, their proposed solution has a hole. Anyway

[GitHub] spark pull request: [SPARK-4226][SQL]Add subquery (not) in/exists ...

2015-12-30 Thread chenghao-intel
Github user chenghao-intel commented on the pull request: https://github.com/apache/spark/pull/9055#issuecomment-168104208 ok, closing it now --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have t

[GitHub] spark pull request: [SPARK-4226][SQL]Add subquery (not) in/exists ...

2015-12-30 Thread chenghao-intel
Github user chenghao-intel closed the pull request at: https://github.com/apache/spark/pull/9055 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] spark pull request: [SPARK-4226][SQL]Add subquery (not) in/exists ...

2015-12-30 Thread yhuai
Github user yhuai commented on the pull request: https://github.com/apache/spark/pull/9055#issuecomment-168101565 @chenghao-intel How about we close this PR for now? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-4226][SQL]Add subquery (not) in/exists ...

2015-12-30 Thread yhuai
Github user yhuai commented on the pull request: https://github.com/apache/spark/pull/9055#issuecomment-168101552 I had a offline discussion with @chenghao-intel. We will split this PR to smaller PRs. The first work will be on the backend operators. Then, we will add parser and analyz

[GitHub] spark pull request: [SPARK-4226][SQL]Add subquery (not) in/exists ...

2015-12-18 Thread roland-mendix
Github user roland-mendix commented on the pull request: https://github.com/apache/spark/pull/9055#issuecomment-165722584 We've added our own In/Exists - plus Subquery in Select - support to a partial fork of Spark SQL Catalyst (which we use in transformations from our own query langu

[GitHub] spark pull request: [SPARK-4226][SQL]Add subquery (not) in/exists ...

2015-12-15 Thread maver1ck
Github user maver1ck commented on the pull request: https://github.com/apache/spark/pull/9055#issuecomment-164912029 So what next ? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this featur

[GitHub] spark pull request: [SPARK-4226][SQL]Add subquery (not) in/exists ...

2015-11-05 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/9055#issuecomment-154154884 Yeah, sorry. It is too late for a patch this large. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark pull request: [SPARK-4226][SQL]Add subquery (not) in/exists ...

2015-11-04 Thread chenghao-intel
Github user chenghao-intel commented on the pull request: https://github.com/apache/spark/pull/9055#issuecomment-153921494 Unfortunately, we probably will miss this in Spark 1.6, as it's almost code freeze for 1.6. @rxin @yhuai --- If your project is set up for it, you can reply to

[GitHub] spark pull request: [SPARK-4226][SQL]Add subquery (not) in/exists ...

2015-11-04 Thread gatorsmile
Github user gatorsmile commented on the pull request: https://github.com/apache/spark/pull/9055#issuecomment-153920042 @jameszhouyi Agree. This is an important feature for any SQL engine. We are also waiting for this feature. So far, using joins is an alternative to bypass it.

[GitHub] spark pull request: [SPARK-4226][SQL]Add subquery (not) in/exists ...

2015-11-04 Thread jameszhouyi
Github user jameszhouyi commented on the pull request: https://github.com/apache/spark/pull/9055#issuecomment-153918005 Thank you @gatorsmile for your suggestion. I think this feature("IN" sub query) is necessary for Spark SQL engine as SQL-on-Hadoop. --- If your project is set

[GitHub] spark pull request: [SPARK-4226][SQL]Add subquery (not) in/exists ...

2015-11-04 Thread gatorsmile
Github user gatorsmile commented on the pull request: https://github.com/apache/spark/pull/9055#issuecomment-153857289 @jameszhouyi We hit the same issue. Now, we bypass it by using joins. --- If your project is set up for it, you can reply to this email and have your reply appe

[GitHub] spark pull request: [SPARK-4226][SQL]Add subquery (not) in/exists ...

2015-11-01 Thread jameszhouyi
Github user jameszhouyi commented on the pull request: https://github.com/apache/spark/pull/9055#issuecomment-152941016 Hi @yhuai , This missing feature("IN" sub query) in Spark SQL blocked our real-world case. Could you please help to review this PR ? Strongly hopefully this PR

[GitHub] spark pull request: [SPARK-4226][SQL]Add subquery (not) in/exists ...

2015-10-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9055#issuecomment-149941345 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44

[GitHub] spark pull request: [SPARK-4226][SQL]Add subquery (not) in/exists ...

2015-10-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9055#issuecomment-149941344 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-4226][SQL]Add subquery (not) in/exists ...

2015-10-21 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9055#issuecomment-149941094 **[Test build #44064 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44064/consoleFull)** for PR 9055 at commit [`cb69166`](https://git

[GitHub] spark pull request: [SPARK-4226][SQL]Add subquery (not) in/exists ...

2015-10-21 Thread chenghao-intel
Github user chenghao-intel commented on the pull request: https://github.com/apache/spark/pull/9055#issuecomment-149908301 BTW: IN / NOT IN definitely supports the uncorrelated, but EXISTS/NOT EXISTS are not in this cases, the same behavior as Hive does. --- If your project is set up

[GitHub] spark pull request: [SPARK-4226][SQL]Add subquery (not) in/exists ...

2015-10-21 Thread chenghao-intel
Github user chenghao-intel commented on the pull request: https://github.com/apache/spark/pull/9055#issuecomment-149907407 Thank you @yhuai for reviewing this. I've added some more docs for this PR, hopefully make more sense. First, I'll agree with you to make a general lo

[GitHub] spark pull request: [SPARK-4226][SQL]Add subquery (not) in/exists ...

2015-10-21 Thread chenghao-intel
Github user chenghao-intel commented on a diff in the pull request: https://github.com/apache/spark/pull/9055#discussion_r42625267 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/BroadcastLeftSemiJoinHash.scala --- @@ -70,7 +77,10 @@ case class BroadcastLeft

[GitHub] spark pull request: [SPARK-4226][SQL]Add subquery (not) in/exists ...

2015-10-21 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9055#issuecomment-149899904 **[Test build #44064 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44064/consoleFull)** for PR 9055 at commit [`cb69166`](https://gith

[GitHub] spark pull request: [SPARK-4226][SQL]Add subquery (not) in/exists ...

2015-10-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9055#issuecomment-149898000 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-4226][SQL]Add subquery (not) in/exists ...

2015-10-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9055#issuecomment-149897980 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-4226][SQL]Add subquery (not) in/exists ...

2015-10-21 Thread chenghao-intel
Github user chenghao-intel commented on a diff in the pull request: https://github.com/apache/spark/pull/9055#discussion_r42620776 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/joinTypes.scala --- @@ -47,4 +48,9 @@ case object RightOuter extends JoinTyp

[GitHub] spark pull request: [SPARK-4226][SQL]Add subquery (not) in/exists ...

2015-10-21 Thread chenghao-intel
Github user chenghao-intel commented on a diff in the pull request: https://github.com/apache/spark/pull/9055#discussion_r42617393 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveSemiJoinSuite.scala --- @@ -0,0 +1,450 @@ +/* + * Licensed to the Ap

[GitHub] spark pull request: [SPARK-4226][SQL]Add subquery (not) in/exists ...

2015-10-21 Thread chenghao-intel
Github user chenghao-intel commented on a diff in the pull request: https://github.com/apache/spark/pull/9055#discussion_r42593628 --- Diff: sql/hive/compatibility/src/test/scala/org/apache/spark/sql/hive/execution/HiveCompatibilitySuite.scala --- @@ -265,6 +265,32 @@ class HiveCo

[GitHub] spark pull request: [SPARK-4226][SQL]Add subquery (not) in/exists ...

2015-10-21 Thread chenghao-intel
Github user chenghao-intel commented on a diff in the pull request: https://github.com/apache/spark/pull/9055#discussion_r42593468 --- Diff: sql/hive/compatibility/src/test/scala/org/apache/spark/sql/hive/execution/HiveCompatibilitySuite.scala --- @@ -265,6 +265,32 @@ class HiveCo

[GitHub] spark pull request: [SPARK-4226][SQL]Add subquery (not) in/exists ...

2015-10-20 Thread chenghao-intel
Github user chenghao-intel commented on a diff in the pull request: https://github.com/apache/spark/pull/9055#discussion_r42586670 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/joinTypes.scala --- @@ -47,4 +48,9 @@ case object RightOuter extends JoinTyp

[GitHub] spark pull request: [SPARK-4226][SQL]Add subquery (not) in/exists ...

2015-10-20 Thread chenghao-intel
Github user chenghao-intel commented on a diff in the pull request: https://github.com/apache/spark/pull/9055#discussion_r42586165 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -408,6 +552,25 @@ class Analyzer( }

[GitHub] spark pull request: [SPARK-4226][SQL]Add subquery (not) in/exists ...

2015-10-20 Thread chenghao-intel
Github user chenghao-intel commented on a diff in the pull request: https://github.com/apache/spark/pull/9055#discussion_r42585365 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -270,6 +273,146 @@ class Analyzer( }

[GitHub] spark pull request: [SPARK-4226][SQL]Add subquery (not) in/exists ...

2015-10-20 Thread chenghao-intel
Github user chenghao-intel commented on a diff in the pull request: https://github.com/apache/spark/pull/9055#discussion_r42584123 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -270,6 +273,146 @@ class Analyzer( }

[GitHub] spark pull request: [SPARK-4226][SQL]Add subquery (not) in/exists ...

2015-10-20 Thread chenghao-intel
Github user chenghao-intel commented on a diff in the pull request: https://github.com/apache/spark/pull/9055#discussion_r42583933 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -270,6 +273,146 @@ class Analyzer( }

[GitHub] spark pull request: [SPARK-4226][SQL]Add subquery (not) in/exists ...

2015-10-20 Thread chenghao-intel
Github user chenghao-intel commented on a diff in the pull request: https://github.com/apache/spark/pull/9055#discussion_r42583623 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -270,6 +273,146 @@ class Analyzer( }

[GitHub] spark pull request: [SPARK-4226][SQL]Add subquery (not) in/exists ...

2015-10-20 Thread chenghao-intel
Github user chenghao-intel commented on a diff in the pull request: https://github.com/apache/spark/pull/9055#discussion_r42583551 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -270,6 +273,146 @@ class Analyzer( }

[GitHub] spark pull request: [SPARK-4226][SQL]Add subquery (not) in/exists ...

2015-10-20 Thread chenghao-intel
Github user chenghao-intel commented on a diff in the pull request: https://github.com/apache/spark/pull/9055#discussion_r42583480 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -270,6 +273,146 @@ class Analyzer( }

[GitHub] spark pull request: [SPARK-4226][SQL]Add subquery (not) in/exists ...

2015-10-20 Thread chenghao-intel
Github user chenghao-intel commented on a diff in the pull request: https://github.com/apache/spark/pull/9055#discussion_r42583439 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -270,6 +273,146 @@ class Analyzer( }

[GitHub] spark pull request: [SPARK-4226][SQL]Add subquery (not) in/exists ...

2015-10-20 Thread chenghao-intel
Github user chenghao-intel commented on a diff in the pull request: https://github.com/apache/spark/pull/9055#discussion_r42583413 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -69,6 +71,7 @@ class Analyzer( WindowsSu

[GitHub] spark pull request: [SPARK-4226][SQL]Add subquery (not) in/exists ...

2015-10-20 Thread chenghao-intel
Github user chenghao-intel commented on a diff in the pull request: https://github.com/apache/spark/pull/9055#discussion_r42583339 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveQl.scala --- @@ -1485,14 +1490,39 @@ https://cwiki.apache.org/confluence/display/Hive/E

[GitHub] spark pull request: [SPARK-4226][SQL]Add subquery (not) in/exists ...

2015-10-20 Thread yhuai
Github user yhuai commented on the pull request: https://github.com/apache/spark/pull/9055#issuecomment-149777477 Two general comments. First, we need to add document to explain how we rewrite a plan when (1) there is a uncorrelated subquery and (2) there is a correlated subquery. Sec

[GitHub] spark pull request: [SPARK-4226][SQL]Add subquery (not) in/exists ...

2015-10-20 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/9055#discussion_r42581865 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveSemiJoinSuite.scala --- @@ -0,0 +1,450 @@ +/* + * Licensed to the Apache Soft

[GitHub] spark pull request: [SPARK-4226][SQL]Add subquery (not) in/exists ...

2015-10-20 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/9055#discussion_r42581807 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/unresolved.scala --- @@ -263,3 +263,50 @@ case class UnresolvedAlias(child: Expres

[GitHub] spark pull request: [SPARK-4226][SQL]Add subquery (not) in/exists ...

2015-10-20 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/9055#discussion_r42581802 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -408,6 +552,25 @@ class Analyzer( } }

[GitHub] spark pull request: [SPARK-4226][SQL]Add subquery (not) in/exists ...

2015-10-20 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/9055#discussion_r42581792 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -270,6 +273,146 @@ class Analyzer( } /**

[GitHub] spark pull request: [SPARK-4226][SQL]Add subquery (not) in/exists ...

2015-10-20 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/9055#discussion_r42581804 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/unresolved.scala --- @@ -263,3 +263,50 @@ case class UnresolvedAlias(child: Expres

[GitHub] spark pull request: [SPARK-4226][SQL]Add subquery (not) in/exists ...

2015-10-20 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/9055#discussion_r42581799 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -270,6 +273,146 @@ class Analyzer( } /**

[GitHub] spark pull request: [SPARK-4226][SQL]Add subquery (not) in/exists ...

2015-10-20 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/9055#discussion_r42581797 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -270,6 +273,146 @@ class Analyzer( } /**

[GitHub] spark pull request: [SPARK-4226][SQL]Add subquery (not) in/exists ...

2015-10-20 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/9055#discussion_r42581809 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/joinTypes.scala --- @@ -47,4 +48,9 @@ case object RightOuter extends JoinType

[GitHub] spark pull request: [SPARK-4226][SQL]Add subquery (not) in/exists ...

2015-10-20 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/9055#discussion_r42581817 --- Diff: sql/hive/compatibility/src/test/scala/org/apache/spark/sql/hive/execution/HiveCompatibilitySuite.scala --- @@ -265,6 +265,32 @@ class HiveCompatibili

[GitHub] spark pull request: [SPARK-4226][SQL]Add subquery (not) in/exists ...

2015-10-20 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/9055#discussion_r42581783 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -270,6 +273,146 @@ class Analyzer( } /**

[GitHub] spark pull request: [SPARK-4226][SQL]Add subquery (not) in/exists ...

2015-10-20 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/9055#discussion_r42581795 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -270,6 +273,146 @@ class Analyzer( } /**

[GitHub] spark pull request: [SPARK-4226][SQL]Add subquery (not) in/exists ...

2015-10-20 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/9055#discussion_r42581793 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -270,6 +273,146 @@ class Analyzer( } /**

[GitHub] spark pull request: [SPARK-4226][SQL]Add subquery (not) in/exists ...

2015-10-20 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/9055#discussion_r42581805 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/unresolved.scala --- @@ -263,3 +263,50 @@ case class UnresolvedAlias(child: Expres

[GitHub] spark pull request: [SPARK-4226][SQL]Add subquery (not) in/exists ...

2015-10-20 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/9055#discussion_r42581787 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -270,6 +273,146 @@ class Analyzer( } /**

[GitHub] spark pull request: [SPARK-4226][SQL]Add subquery (not) in/exists ...

2015-10-20 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/9055#discussion_r42581800 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -270,6 +273,146 @@ class Analyzer( } /**

[GitHub] spark pull request: [SPARK-4226][SQL]Add subquery (not) in/exists ...

2015-10-20 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/9055#discussion_r42581789 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -270,6 +273,146 @@ class Analyzer( } /**

[GitHub] spark pull request: [SPARK-4226][SQL]Add subquery (not) in/exists ...

2015-10-20 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/9055#discussion_r42581813 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/BroadcastLeftSemiJoinHash.scala --- @@ -70,7 +77,10 @@ case class BroadcastLeftSemiJoinH

[GitHub] spark pull request: [SPARK-4226][SQL]Add subquery (not) in/exists ...

2015-10-20 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/9055#discussion_r42581785 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -270,6 +273,146 @@ class Analyzer( } /**

[GitHub] spark pull request: [SPARK-4226][SQL]Add subquery (not) in/exists ...

2015-10-20 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/9055#discussion_r42581788 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -270,6 +273,146 @@ class Analyzer( } /**

[GitHub] spark pull request: [SPARK-4226][SQL]Add subquery (not) in/exists ...

2015-10-20 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/9055#discussion_r42581790 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -270,6 +273,146 @@ class Analyzer( } /**

[GitHub] spark pull request: [SPARK-4226][SQL]Add subquery (not) in/exists ...

2015-10-20 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/9055#discussion_r42581780 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -69,6 +71,7 @@ class Analyzer( WindowsSubstitutio

[GitHub] spark pull request: [SPARK-4226][SQL]Add subquery (not) in/exists ...

2015-10-20 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/9055#discussion_r42581781 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -270,6 +273,146 @@ class Analyzer( } /**

[GitHub] spark pull request: [SPARK-4226][SQL]Add subquery (not) in/exists ...

2015-10-20 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/9055#discussion_r42581782 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -270,6 +273,146 @@ class Analyzer( } /**

[GitHub] spark pull request: [SPARK-4226][SQL]Add subquery (not) in/exists ...

2015-10-20 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/9055#discussion_r42581775 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveQl.scala --- @@ -1485,14 +1490,39 @@ https://cwiki.apache.org/confluence/display/Hive/Enhanced+A

[GitHub] spark pull request: [SPARK-4226][SQL]Add subquery (not) in/exists ...

2015-10-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9055#issuecomment-148592465 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-4226][SQL]Add subquery (not) in/exists ...

2015-10-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9055#issuecomment-148592469 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43

[GitHub] spark pull request: [SPARK-4226][SQL]Add subquery (not) in/exists ...

2015-10-15 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9055#issuecomment-148592066 [Test build #43826 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43826/console) for PR 9055 at commit [`7511f47`](https://github.

[GitHub] spark pull request: [SPARK-4226][SQL]Add subquery (not) in/exists ...

2015-10-15 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9055#issuecomment-148571269 [Test build #43826 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43826/consoleFull) for PR 9055 at commit [`7511f47`](https://gith

[GitHub] spark pull request: [SPARK-4226][SQL]Add subquery (not) in/exists ...

2015-10-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9055#issuecomment-148570278 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-4226][SQL]Add subquery (not) in/exists ...

2015-10-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9055#issuecomment-148570292 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-4226][SQL]Add subquery (not) in/exists ...

2015-10-15 Thread chenghao-intel
Github user chenghao-intel commented on the pull request: https://github.com/apache/spark/pull/9055#issuecomment-148570097 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have t

[GitHub] spark pull request: [SPARK-4226][SQL]Add subquery (not) in/exists ...

2015-10-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9055#issuecomment-148563286 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43

[GitHub] spark pull request: [SPARK-4226][SQL]Add subquery (not) in/exists ...

2015-10-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9055#issuecomment-148563285 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-4226][SQL]Add subquery (not) in/exists ...

2015-10-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9055#issuecomment-148562591 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-4226][SQL]Add subquery (not) in/exists ...

2015-10-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9055#issuecomment-148562593 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43

[GitHub] spark pull request: [SPARK-4226][SQL]Add subquery (not) in/exists ...

2015-10-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9055#issuecomment-148561456 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-4226][SQL]Add subquery (not) in/exists ...

2015-10-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9055#issuecomment-148561446 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-4226][SQL]Add subquery (not) in/exists ...

2015-10-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9055#issuecomment-148560924 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-4226][SQL]Add subquery (not) in/exists ...

2015-10-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9055#issuecomment-148560908 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-4226][SQL]Add subquery (not) in/exists ...

2015-10-15 Thread chenghao-intel
Github user chenghao-intel commented on the pull request: https://github.com/apache/spark/pull/9055#issuecomment-148560878 Seems not related. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have t

[GitHub] spark pull request: [SPARK-4226][SQL]Add subquery (not) in/exists ...

2015-10-15 Thread chenghao-intel
Github user chenghao-intel commented on the pull request: https://github.com/apache/spark/pull/9055#issuecomment-148560889 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have t

[GitHub] spark pull request: [SPARK-4226][SQL]Add subquery (not) in/exists ...

2015-10-15 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9055#issuecomment-148326579 [Test build #43782 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43782/console) for PR 9055 at commit [`7511f47`](https://github.

[GitHub] spark pull request: [SPARK-4226][SQL]Add subquery (not) in/exists ...

2015-10-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9055#issuecomment-148326617 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-4226][SQL]Add subquery (not) in/exists ...

2015-10-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9055#issuecomment-148326620 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43

[GitHub] spark pull request: [SPARK-4226][SQL]Add subquery (not) in/exists ...

2015-10-15 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9055#issuecomment-148322379 [Test build #43782 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43782/consoleFull) for PR 9055 at commit [`7511f47`](https://gith

[GitHub] spark pull request: [SPARK-4226][SQL]Add subquery (not) in/exists ...

2015-10-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9055#issuecomment-148320889 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-4226][SQL]Add subquery (not) in/exists ...

2015-10-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9055#issuecomment-148320902 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-4226][SQL]Add subquery (not) in/exists ...

2015-10-12 Thread chenghao-intel
Github user chenghao-intel commented on the pull request: https://github.com/apache/spark/pull/9055#issuecomment-147558163 cc @rxin as well, this is required by many of our customers, and most of the code change is about the unit test, should not be hard to follow. --- If your projec

[GitHub] spark pull request: [SPARK-4226][SQL]Add subquery (not) in/exists ...

2015-10-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9055#issuecomment-147278139 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-4226][SQL]Add subquery (not) in/exists ...

2015-10-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9055#issuecomment-147278144 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43

[GitHub] spark pull request: [SPARK-4226][SQL]Add subquery (not) in/exists ...

2015-10-11 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9055#issuecomment-147277801 [Test build #43552 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43552/console) for PR 9055 at commit [`ab22171`](https://github.

[GitHub] spark pull request: [SPARK-4226][SQL]Add subquery (not) in/exists ...

2015-10-11 Thread chenghao-intel
Github user chenghao-intel commented on the pull request: https://github.com/apache/spark/pull/9055#issuecomment-147273772 No, we don't support that in this PR, but should be very easy to support once this PR merged. I can plan the work if you feel that's very critical to your custome

[GitHub] spark pull request: [SPARK-4226][SQL]Add subquery (not) in/exists ...

2015-10-11 Thread scwf
Github user scwf commented on the pull request: https://github.com/apache/spark/pull/9055#issuecomment-147273499 ok, does this support multi exists and in in where clause? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. I

[GitHub] spark pull request: [SPARK-4226][SQL]Add subquery (not) in/exists ...

2015-10-11 Thread chenghao-intel
Github user chenghao-intel commented on the pull request: https://github.com/apache/spark/pull/9055#issuecomment-147272870 This is much simpler than #4812, by using the `SubQueryExpression`, particularly in processing the case `key IN (subquery) AND other_condition` case. #4812 doe

[GitHub] spark pull request: [SPARK-4226][SQL]Add subquery (not) in/exists ...

2015-10-11 Thread scwf
Github user scwf commented on the pull request: https://github.com/apache/spark/pull/9055#issuecomment-147272550 what's the difference with #4812? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-4226][SQL]Add subquery (not) in/exists ...

2015-10-11 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9055#issuecomment-147267270 [Test build #43552 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43552/consoleFull) for PR 9055 at commit [`ab22171`](https://gith

[GitHub] spark pull request: [SPARK-4226][SQL]Add subquery (not) in/exists ...

2015-10-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9055#issuecomment-147266659 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-4226][SQL]Add subquery (not) in/exists ...

2015-10-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9055#issuecomment-14725 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-4226][SQL]Add subquery (not) in/exists ...

2015-10-11 Thread chenghao-intel
Github user chenghao-intel commented on the pull request: https://github.com/apache/spark/pull/9055#issuecomment-147266579 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have t

[GitHub] spark pull request: [SPARK-4226][SQL]Add subquery (not) in/exists ...

2015-10-11 Thread chenghao-intel
Github user chenghao-intel commented on the pull request: https://github.com/apache/spark/pull/9055#issuecomment-147263349 Seems the failure is not related. retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark pull request: [SPARK-4226][SQL]Add subquery (not) in/exists ...

2015-10-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9055#issuecomment-147074069 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43

[GitHub] spark pull request: [SPARK-4226][SQL]Add subquery (not) in/exists ...

2015-10-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9055#issuecomment-147074068 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

  1   2   >