[GitHub] spark pull request: [SPARK-12503] [SQL] Pushing Limit Through Unio...

2016-01-11 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/10451#issuecomment-170757463 Let's close the limit push down pull requests. We will need to design this more properly because it is expensive to push down large limits. --- If your project is set up

[GitHub] spark pull request: [SPARK-12503] [SQL] Pushing Limit Through Unio...

2016-01-11 Thread gatorsmile
Github user gatorsmile commented on the pull request: https://github.com/apache/spark/pull/10451#issuecomment-170757733 Sure, let me close it. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-12503] [SQL] Pushing Limit Through Unio...

2016-01-11 Thread gatorsmile
Github user gatorsmile closed the pull request at: https://github.com/apache/spark/pull/10451 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] spark pull request: [SPARK-12503] [SQL] Pushing Limit Through Unio...

2015-12-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10451#issuecomment-167954886 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-12503] [SQL] Pushing Limit Through Unio...

2015-12-30 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10451#issuecomment-167954987 **[Test build #48461 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48461/consoleFull)** for PR 10451 at commit

[GitHub] spark pull request: [SPARK-12503] [SQL] Pushing Limit Through Unio...

2015-12-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10451#issuecomment-167954885 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-12503] [SQL] Pushing Limit Through Unio...

2015-12-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10451#issuecomment-167972021 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-12503] [SQL] Pushing Limit Through Unio...

2015-12-30 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10451#issuecomment-167971953 **[Test build #48465 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48465/consoleFull)** for PR 10451 at commit

[GitHub] spark pull request: [SPARK-12503] [SQL] Pushing Limit Through Unio...

2015-12-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10451#issuecomment-167972019 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-12503] [SQL] Pushing Limit Through Unio...

2015-12-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10451#issuecomment-167955427 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-12503] [SQL] Pushing Limit Through Unio...

2015-12-30 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10451#issuecomment-167955424 **[Test build #48461 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48461/consoleFull)** for PR 10451 at commit

[GitHub] spark pull request: [SPARK-12503] [SQL] Pushing Limit Through Unio...

2015-12-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10451#issuecomment-167955428 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-12503] [SQL] Pushing Limit Through Unio...

2015-12-30 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10451#issuecomment-167957490 **[Test build #48465 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48465/consoleFull)** for PR 10451 at commit

[GitHub] spark pull request: [SPARK-12503] [SQL] Pushing Limit Through Unio...

2015-12-29 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/10451#discussion_r48558035 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -153,6 +153,17 @@ object SetOperationPushDown extends

[GitHub] spark pull request: [SPARK-12503] [SQL] Pushing Limit Through Unio...

2015-12-29 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/10451#issuecomment-167845210 Thanks for working on this. I think its getting pretty close. A few minor cleanups that might be nice: - I think we should consider pulling all the Limit rules

[GitHub] spark pull request: [SPARK-12503] [SQL] Pushing Limit Through Unio...

2015-12-29 Thread gatorsmile
Github user gatorsmile commented on the pull request: https://github.com/apache/spark/pull/10451#issuecomment-167902518 After rethinking the `Limit` push-down rules, we are unable to push Limit through any operator that could change the values or the number of rows. Thus, so far, the

[GitHub] spark pull request: [SPARK-12503] [SQL] Pushing Limit Through Unio...

2015-12-29 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/10451#discussion_r48581020 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -80,6 +81,33 @@ abstract class Optimizer extends

[GitHub] spark pull request: [SPARK-12503] [SQL] Pushing Limit Through Unio...

2015-12-29 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/10451#discussion_r48578529 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -69,6 +73,33 @@ object DefaultOptimizer extends

[GitHub] spark pull request: [SPARK-12503] [SQL] Pushing Limit Through Unio...

2015-12-29 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/10451#discussion_r48581249 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/LogicalPlan.scala --- @@ -91,6 +91,11 @@ abstract class LogicalPlan

[GitHub] spark pull request: [SPARK-12503] [SQL] Pushing Limit Through Unio...

2015-12-29 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/10451#discussion_r48582039 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -80,6 +81,33 @@ abstract class Optimizer extends

[GitHub] spark pull request: [SPARK-12503] [SQL] Pushing Limit Through Unio...

2015-12-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10451#issuecomment-167916894 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-12503] [SQL] Pushing Limit Through Unio...

2015-12-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10451#issuecomment-167916892 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-12503] [SQL] Pushing Limit Through Unio...

2015-12-29 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10451#issuecomment-167899814 **[Test build #48431 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48431/consoleFull)** for PR 10451 at commit

[GitHub] spark pull request: [SPARK-12503] [SQL] Pushing Limit Through Unio...

2015-12-29 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/10451#discussion_r48579121 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -69,6 +73,33 @@ object DefaultOptimizer extends

[GitHub] spark pull request: [SPARK-12503] [SQL] Pushing Limit Through Unio...

2015-12-29 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/10451#discussion_r48580661 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -80,6 +81,33 @@ abstract class Optimizer extends

[GitHub] spark pull request: [SPARK-12503] [SQL] Pushing Limit Through Unio...

2015-12-29 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10451#issuecomment-167916591 **[Test build #48435 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48435/consoleFull)** for PR 10451 at commit

[GitHub] spark pull request: [SPARK-12503] [SQL] Pushing Limit Through Unio...

2015-12-29 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10451#issuecomment-167900829 **[Test build #48433 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48433/consoleFull)** for PR 10451 at commit

[GitHub] spark pull request: [SPARK-12503] [SQL] Pushing Limit Through Unio...

2015-12-29 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/10451#discussion_r48579360 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -153,6 +153,17 @@ object SetOperationPushDown

[GitHub] spark pull request: [SPARK-12503] [SQL] Pushing Limit Through Unio...

2015-12-29 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10451#issuecomment-167903020 **[Test build #48435 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48435/consoleFull)** for PR 10451 at commit

[GitHub] spark pull request: [SPARK-12503] [SQL] Pushing Limit Through Unio...

2015-12-29 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/10451#discussion_r48581133 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -80,6 +81,33 @@ abstract class Optimizer extends

[GitHub] spark pull request: [SPARK-12503] [SQL] Pushing Limit Through Unio...

2015-12-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10451#issuecomment-167932525 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-12503] [SQL] Pushing Limit Through Unio...

2015-12-29 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10451#issuecomment-167932503 **[Test build #48433 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48433/consoleFull)** for PR 10451 at commit

[GitHub] spark pull request: [SPARK-12503] [SQL] Pushing Limit Through Unio...

2015-12-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10451#issuecomment-167932524 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-12503] [SQL] Pushing Limit Through Unio...

2015-12-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10451#issuecomment-167931936 Build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: [SPARK-12503] [SQL] Pushing Limit Through Unio...

2015-12-29 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10451#issuecomment-167931892 **[Test build #48431 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48431/consoleFull)** for PR 10451 at commit

[GitHub] spark pull request: [SPARK-12503] [SQL] Pushing Limit Through Unio...

2015-12-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10451#issuecomment-167931938 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-12503] [SQL] Pushing Limit Through Unio...

2015-12-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10451#issuecomment-167954060 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-12503] [SQL] Pushing Limit Through Unio...

2015-12-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10451#issuecomment-167954059 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-12503] [SQL] Pushing Limit Through Unio...

2015-12-29 Thread gatorsmile
Github user gatorsmile commented on the pull request: https://github.com/apache/spark/pull/10451#issuecomment-167952117 The latest version covers both cases: 1) If the children' `maxRows` has a smaller number, add the extra limit. 2) If the children' `maxRows` is `None`, add

[GitHub] spark pull request: [SPARK-12503] [SQL] Pushing Limit Through Unio...

2015-12-29 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/10451#discussion_r48590465 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/LogicalPlan.scala --- @@ -91,6 +91,11 @@ abstract class LogicalPlan

[GitHub] spark pull request: [SPARK-12503] [SQL] Pushing Limit Through Unio...

2015-12-29 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10451#issuecomment-167742397 **[Test build #48400 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48400/consoleFull)** for PR 10451 at commit

[GitHub] spark pull request: [SPARK-12503] [SQL] Pushing Limit Through Unio...

2015-12-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10451#issuecomment-167742459 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-12503] [SQL] Pushing Limit Through Unio...

2015-12-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10451#issuecomment-167742457 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-12503] [SQL] Pushing Limit Through Unio...

2015-12-28 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/10451#discussion_r48517911 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -153,6 +153,15 @@ object SetOperationPushDown

[GitHub] spark pull request: [SPARK-12503] [SQL] Pushing Limit Through Unio...

2015-12-28 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/10451#discussion_r48518043 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -153,6 +153,15 @@ object SetOperationPushDown

[GitHub] spark pull request: [SPARK-12503] [SQL] Pushing Limit Through Unio...

2015-12-28 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/10451#discussion_r48518201 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -153,6 +153,15 @@ object SetOperationPushDown extends

[GitHub] spark pull request: [SPARK-12503] [SQL] Pushing Limit Through Unio...

2015-12-28 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/10451#discussion_r48518190 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -153,6 +153,15 @@ object SetOperationPushDown extends

[GitHub] spark pull request: [SPARK-12503] [SQL] Pushing Limit Through Unio...

2015-12-28 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/10451#discussion_r48518228 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/LogicalPlan.scala --- @@ -91,6 +91,11 @@ abstract class LogicalPlan

[GitHub] spark pull request: [SPARK-12503] [SQL] Pushing Limit Through Unio...

2015-12-28 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/10451#issuecomment-167700694 > add a comment and explain the current solution. In the future, if we add such an operator, we can change the current way and fix the issue? (Already added a comment

[GitHub] spark pull request: [SPARK-12503] [SQL] Pushing Limit Through Unio...

2015-12-28 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/10451#discussion_r48518442 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/LogicalPlan.scala --- @@ -91,6 +91,11 @@ abstract class LogicalPlan

[GitHub] spark pull request: [SPARK-12503] [SQL] Pushing Limit Through Unio...

2015-12-28 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/10451#discussion_r48501449 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/basicOperators.scala --- @@ -448,7 +448,21 @@ case class Pivot( }

[GitHub] spark pull request: [SPARK-12503] [SQL] Pushing Limit Through Unio...

2015-12-28 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/10451#discussion_r48501690 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/basicOperators.scala --- @@ -448,7 +448,21 @@ case class Pivot(

[GitHub] spark pull request: [SPARK-12503] [SQL] Pushing Limit Through Unio...

2015-12-28 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/10451#discussion_r48523071 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/LogicalPlan.scala --- @@ -91,6 +91,11 @@ abstract class LogicalPlan

[GitHub] spark pull request: [SPARK-12503] [SQL] Pushing Limit Through Unio...

2015-12-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10451#issuecomment-167677723 **[Test build #48379 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48379/consoleFull)** for PR 10451 at commit

[GitHub] spark pull request: [SPARK-12503] [SQL] Pushing Limit Through Unio...

2015-12-28 Thread gatorsmile
Github user gatorsmile commented on the pull request: https://github.com/apache/spark/pull/10451#issuecomment-167678849 A few updates are done, but I am not sure if the changes are appropriate. 1. `limit` is already used in

[GitHub] spark pull request: [SPARK-12503] [SQL] Pushing Limit Through Unio...

2015-12-28 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/10451#discussion_r48512926 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/LogicalPlan.scala --- @@ -91,6 +91,11 @@ abstract class LogicalPlan

[GitHub] spark pull request: [SPARK-12503] [SQL] Pushing Limit Through Unio...

2015-12-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10451#issuecomment-167679998 **[Test build #48380 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48380/consoleFull)** for PR 10451 at commit

[GitHub] spark pull request: [SPARK-12503] [SQL] Pushing Limit Through Unio...

2015-12-28 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/10451#discussion_r48513000 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/LogicalPlan.scala --- @@ -91,6 +91,11 @@ abstract class LogicalPlan

[GitHub] spark pull request: [SPARK-12503] [SQL] Pushing Limit Through Unio...

2015-12-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10451#issuecomment-167688592 **[Test build #48379 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48379/consoleFull)** for PR 10451 at commit

[GitHub] spark pull request: [SPARK-12503] [SQL] Pushing Limit Through Unio...

2015-12-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10451#issuecomment-167688631 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-12503] [SQL] Pushing Limit Through Unio...

2015-12-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10451#issuecomment-167688630 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-12503] [SQL] Pushing Limit Through Unio...

2015-12-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10451#issuecomment-167689297 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-12503] [SQL] Pushing Limit Through Unio...

2015-12-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10451#issuecomment-167689296 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-12503] [SQL] Pushing Limit Through Unio...

2015-12-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10451#issuecomment-167689240 **[Test build #48380 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48380/consoleFull)** for PR 10451 at commit

[GitHub] spark pull request: [SPARK-12503] [SQL] Pushing Limit Through Unio...

2015-12-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10451#issuecomment-167729202 **[Test build #48400 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48400/consoleFull)** for PR 10451 at commit

[GitHub] spark pull request: [SPARK-12503] [SQL] Pushing Limit Through Unio...

2015-12-25 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10451#issuecomment-167213191 **[Test build #48322 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48322/consoleFull)** for PR 10451 at commit

[GitHub] spark pull request: [SPARK-12503] [SQL] Pushing Limit Through Unio...

2015-12-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10451#issuecomment-167213359 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-12503] [SQL] Pushing Limit Through Unio...

2015-12-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10451#issuecomment-167213357 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-12503] [SQL] Pushing Limit Through Unio...

2015-12-25 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/10451#discussion_r48441937 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -153,6 +153,16 @@ object SetOperationPushDown

[GitHub] spark pull request: [SPARK-12503] [SQL] Pushing Limit Through Unio...

2015-12-25 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/10451#discussion_r48441867 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -153,6 +153,16 @@ object SetOperationPushDown

[GitHub] spark pull request: [SPARK-12503] [SQL] Pushing Limit Through Unio...

2015-12-24 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/10451#discussion_r48432615 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -153,6 +153,15 @@ object SetOperationPushDown

[GitHub] spark pull request: [SPARK-12503] [SQL] Pushing Limit Through Unio...

2015-12-24 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/10451#discussion_r48432925 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -153,6 +153,15 @@ object SetOperationPushDown

[GitHub] spark pull request: [SPARK-12503] [SQL] Pushing Limit Through Unio...

2015-12-24 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10451#issuecomment-167203391 **[Test build #48320 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48320/consoleFull)** for PR 10451 at commit

[GitHub] spark pull request: [SPARK-12503] [SQL] Pushing Limit Through Unio...

2015-12-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10451#issuecomment-167203394 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-12503] [SQL] Pushing Limit Through Unio...

2015-12-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10451#issuecomment-167203393 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-12503] [SQL] Pushing Limit Through Unio...

2015-12-24 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10451#issuecomment-167203168 **[Test build #48320 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48320/consoleFull)** for PR 10451 at commit

[GitHub] spark pull request: [SPARK-12503] [SQL] Pushing Limit Through Unio...

2015-12-24 Thread gatorsmile
Github user gatorsmile commented on the pull request: https://github.com/apache/spark/pull/10451#issuecomment-167202389 @cloud-fan Just added a quick fix by introducing a stop flag to avoid adding extra `Limit` in the subsequent iteration, but I am not sure if my way is good.

[GitHub] spark pull request: [SPARK-12503] [SQL] Pushing Limit Through Unio...

2015-12-24 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10451#issuecomment-167205352 **[Test build #48322 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48322/consoleFull)** for PR 10451 at commit

[GitHub] spark pull request: [SPARK-12503] [SQL] Pushing Limit Through Unio...

2015-12-24 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10451#issuecomment-167067050 **[Test build #48299 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48299/consoleFull)** for PR 10451 at commit

[GitHub] spark pull request: [SPARK-12503] [SQL] Pushing Limit Through Unio...

2015-12-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10451#issuecomment-167079554 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-12503] [SQL] Pushing Limit Through Unio...

2015-12-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10451#issuecomment-167079556 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-12503] [SQL] Pushing Limit Through Unio...

2015-12-24 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10451#issuecomment-167079472 **[Test build #48299 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48299/consoleFull)** for PR 10451 at commit

[GitHub] spark pull request: [SPARK-12503] [SQL] Pushing Limit Through Unio...

2015-12-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10451#issuecomment-166963932 **[Test build #48248 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48248/consoleFull)** for PR 10451 at commit

[GitHub] spark pull request: [SPARK-12503] [SQL] Pushing Limit Through Unio...

2015-12-23 Thread gatorsmile
GitHub user gatorsmile opened a pull request: https://github.com/apache/spark/pull/10451 [SPARK-12503] [SQL] Pushing Limit Through Union "Rule that applies to a Limit on top of a Union. The original Limit won't go away after applying this rule, but additional Limit nodes will be

[GitHub] spark pull request: [SPARK-12503] [SQL] Pushing Limit Through Unio...

2015-12-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10451#issuecomment-167013142 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-12503] [SQL] Pushing Limit Through Unio...

2015-12-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10451#issuecomment-167013085 **[Test build #48261 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48261/consoleFull)** for PR 10451 at commit

[GitHub] spark pull request: [SPARK-12503] [SQL] Pushing Limit Through Unio...

2015-12-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10451#issuecomment-167013143 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-12503] [SQL] Pushing Limit Through Unio...

2015-12-23 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/10451#discussion_r48391800 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -153,6 +153,15 @@ object SetOperationPushDown

[GitHub] spark pull request: [SPARK-12503] [SQL] Pushing Limit Through Unio...

2015-12-23 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/10451#discussion_r48392104 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -153,6 +153,15 @@ object SetOperationPushDown

[GitHub] spark pull request: [SPARK-12503] [SQL] Pushing Limit Through Unio...

2015-12-23 Thread gatorsmile
Github user gatorsmile commented on the pull request: https://github.com/apache/spark/pull/10451#issuecomment-166980946 @marmbrus @rxin Could you check if this is an appropriate improvement for Spark too? Thanks! --- If your project is set up for it, you can reply to this

[GitHub] spark pull request: [SPARK-12503] [SQL] Pushing Limit Through Unio...

2015-12-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10451#issuecomment-166980561 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-12503] [SQL] Pushing Limit Through Unio...

2015-12-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10451#issuecomment-166980563 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-12503] [SQL] Pushing Limit Through Unio...

2015-12-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10451#issuecomment-167005403 **[Test build #48261 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48261/consoleFull)** for PR 10451 at commit

[GitHub] spark pull request: [SPARK-12503] [SQL] Pushing Limit Through Unio...

2015-12-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10451#issuecomment-166980458 **[Test build #48248 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48248/consoleFull)** for PR 10451 at commit

[GitHub] spark pull request: [SPARK-12503] [SQL] Pushing Limit Through Unio...

2015-12-23 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/10451#discussion_r48377323 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -153,6 +153,15 @@ object SetOperationPushDown