[GitHub] spark pull request: [SPARK-14939][SQL] Add FoldablePropagation opt...

2016-05-19 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the pull request: https://github.com/apache/spark/pull/12719#issuecomment-220256852 Thank you so much, @cloud-fan ! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project do

[GitHub] spark pull request: [SPARK-14939][SQL] Add FoldablePropagation opt...

2016-05-19 Thread cloud-fan
Github user cloud-fan commented on the pull request: https://github.com/apache/spark/pull/12719#issuecomment-220254151 thanks, merging to master and 2.0! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project doe

[GitHub] spark pull request: [SPARK-14939][SQL] Add FoldablePropagation opt...

2016-05-19 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/12719 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark pull request: [SPARK-14939][SQL] Add FoldablePropagation opt...

2016-05-18 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the pull request: https://github.com/apache/spark/pull/12719#issuecomment-220233909 @cloud-fan . Now, it's ready again. Could you merge this PR. --- If your project is set up for it, you can reply to this email and have your reply appear

[GitHub] spark pull request: [SPARK-14939][SQL] Add FoldablePropagation opt...

2016-05-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12719#issuecomment-220225586 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-14939][SQL] Add FoldablePropagation opt...

2016-05-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12719#issuecomment-220225588 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-14939][SQL] Add FoldablePropagation opt...

2016-05-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12719#issuecomment-220225490 **[Test build #58837 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58837/consoleFull)** for PR 12719 at commit [`0cb1136`](https://g

[GitHub] spark pull request: [SPARK-14939][SQL] Add FoldablePropagation opt...

2016-05-18 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the pull request: https://github.com/apache/spark/pull/12719#issuecomment-220217482 Oh, amazing. According to the last Jenkins results. The seven test failures in `catalyst` are all of them. ``` [info] *** 7 TESTS FAILED *** [error] Fa

[GitHub] spark pull request: [SPARK-14939][SQL] Add FoldablePropagation opt...

2016-05-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12719#issuecomment-220217398 **[Test build #58837 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58837/consoleFull)** for PR 12719 at commit [`0cb1136`](https://gi

[GitHub] spark pull request: [SPARK-14939][SQL] Add FoldablePropagation opt...

2016-05-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12719#issuecomment-220217295 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-14939][SQL] Add FoldablePropagation opt...

2016-05-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12719#issuecomment-220217294 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-14939][SQL] Add FoldablePropagation opt...

2016-05-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12719#issuecomment-220217222 **[Test build #58834 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58834/consoleFull)** for PR 12719 at commit [`d8257ee`](https://g

[GitHub] spark pull request: [SPARK-14939][SQL] Add FoldablePropagation opt...

2016-05-18 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the pull request: https://github.com/apache/spark/pull/12719#issuecomment-220217246 I removed the last test commit. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project do

[GitHub] spark pull request: [SPARK-14939][SQL] Add FoldablePropagation opt...

2016-05-18 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the pull request: https://github.com/apache/spark/pull/12719#issuecomment-220216995 Thank you for understanding. I'll try to handle those test issues in another PR. --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark pull request: [SPARK-14939][SQL] Add FoldablePropagation opt...

2016-05-18 Thread cloud-fan
Github user cloud-fan commented on the pull request: https://github.com/apache/spark/pull/12719#issuecomment-220216050 I'm fine to leave the `resolved` checking in this PR, because the test issue is kind of unrelated. But it will be good if we can send another PR to fix the test issue

[GitHub] spark pull request: [SPARK-14939][SQL] Add FoldablePropagation opt...

2016-05-18 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the pull request: https://github.com/apache/spark/pull/12719#issuecomment-220213423 Hmm. @cloud-fan . What about just using `resolved` checking simply? IMHO, it provides just robustness. And, in fact, I'm reluctant to change testsuite when adding

[GitHub] spark pull request: [SPARK-14939][SQL] Add FoldablePropagation opt...

2016-05-18 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the pull request: https://github.com/apache/spark/pull/12719#issuecomment-220212842 For the second suggestion, `the optimizer is not tested but skipped`, you mean skipping `FoldablePropagation` optimizer? --- If your project is set up for it, yo

[GitHub] spark pull request: [SPARK-14939][SQL] Add FoldablePropagation opt...

2016-05-18 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the pull request: https://github.com/apache/spark/pull/12719#issuecomment-220212554 Oops. Analyzing by modifying `checkEvaluation` is not helpful in this case. For example, `CastSuite`, ``` { val ret = cast(array_notNull, Arr

[GitHub] spark pull request: [SPARK-14939][SQL] Add FoldablePropagation opt...

2016-05-18 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the pull request: https://github.com/apache/spark/pull/12719#issuecomment-220211380 Okay, I will proceed with modifying `checkEvaluation`. Thank you for fast decision, @cloud-fan . --- If your project is set up for it, you can reply to this emai

[GitHub] spark pull request: [SPARK-14939][SQL] Add FoldablePropagation opt...

2016-05-18 Thread cloud-fan
Github user cloud-fan commented on the pull request: https://github.com/apache/spark/pull/12719#issuecomment-220211147 Ok, so there are a lot of places in our tests that use unresolved expression to test optimizer. I think we should analyze the expression first in `checkEvaluation`, o

[GitHub] spark pull request: [SPARK-14939][SQL] Add FoldablePropagation opt...

2016-05-18 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the pull request: https://github.com/apache/spark/pull/12719#issuecomment-220210913 Should I touch those too? I agree with you that this situation will not occur in **real** testcases. Maybe, only `catalyst`-related problems? --- If your pro

[GitHub] spark pull request: [SPARK-14939][SQL] Add FoldablePropagation opt...

2016-05-18 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the pull request: https://github.com/apache/spark/pull/12719#issuecomment-220210664 @cloud-fan . Here is the result of `catalyst` first. - **catalyst**: 7 failures - DateExpressionsSuite: 3 failures - CastSuite: 4 failures --- If

[GitHub] spark pull request: [SPARK-14939][SQL] Add FoldablePropagation opt...

2016-05-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12719#issuecomment-220208917 **[Test build #58834 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58834/consoleFull)** for PR 12719 at commit [`d8257ee`](https://gi

[GitHub] spark pull request: [SPARK-14939][SQL] Add FoldablePropagation opt...

2016-05-18 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the pull request: https://github.com/apache/spark/pull/12719#issuecomment-220208494 Here is another test run (without resolved checking and the test case `checkEvaluation(FormatNumber(Literal.create(null, NullType), Literal(3)), null)`. --- If

[GitHub] spark pull request: [SPARK-14939][SQL] Add FoldablePropagation opt...

2016-05-18 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the pull request: https://github.com/apache/spark/pull/12719#issuecomment-220207983 May I rollback the last commit? Let's see the Jenkins result. I think It's worth of doing that. --- If your project is set up for it, you can reply to this email

[GitHub] spark pull request: [SPARK-14939][SQL] Add FoldablePropagation opt...

2016-05-18 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the pull request: https://github.com/apache/spark/pull/12719#issuecomment-220207782 Actually, I made two separate Jenkins run to show you the comparison. Those fails on MiMa errors. --- If your project is set up for it, you can reply to this ema

[GitHub] spark pull request: [SPARK-14939][SQL] Add FoldablePropagation opt...

2016-05-18 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the pull request: https://github.com/apache/spark/pull/12719#issuecomment-220207591 No, there are more test suite failures. That is just one example. --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark pull request: [SPARK-14939][SQL] Add FoldablePropagation opt...

2016-05-18 Thread cloud-fan
Github user cloud-fan commented on the pull request: https://github.com/apache/spark/pull/12719#issuecomment-220202352 If this is the only case, I prefer to remove this line: `checkEvaluation(FormatNumber(Literal.create(null, NullType), Literal(3)), null)` In real world, a

[GitHub] spark pull request: [SPARK-14939][SQL] Add FoldablePropagation opt...

2016-05-18 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the pull request: https://github.com/apache/spark/pull/12719#issuecomment-220134254 In addition, sorry for that I wrote a wrong example before `checkEvaluation(FormatNumber(Literal(4.asInstanceOf[Byte]), Literal(3)), "4.000")`. --- If your pro

[GitHub] spark pull request: [SPARK-14939][SQL] Add FoldablePropagation opt...

2016-05-18 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the pull request: https://github.com/apache/spark/pull/12719#issuecomment-220128276 Hi, @cloud-fan . I found the root cause. There is an exceptional case for **Literal.create(null, NullType)**. ```scala scala> import org.apache.spark.sql.t

[GitHub] spark pull request: [SPARK-14939][SQL] Add FoldablePropagation opt...

2016-05-18 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the pull request: https://github.com/apache/spark/pull/12719#issuecomment-220100881 Sure! I'll investigate more. Let you know. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-14939][SQL] Add FoldablePropagation opt...

2016-05-18 Thread cloud-fan
Github user cloud-fan commented on the pull request: https://github.com/apache/spark/pull/12719#issuecomment-219986401 I'm ok to keep the `resolved` checking, if we can figure out why the test failed and there is no easy workaround. Can you dig into it to see what happened? Basically

[GitHub] spark pull request: [SPARK-14939][SQL] Add FoldablePropagation opt...

2016-05-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12719#issuecomment-219895673 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-14939][SQL] Add FoldablePropagation opt...

2016-05-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12719#issuecomment-219895669 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-14939][SQL] Add FoldablePropagation opt...

2016-05-17 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12719#issuecomment-219895536 **[Test build #58726 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58726/consoleFull)** for PR 12719 at commit [`3206bd0`](https://g

[GitHub] spark pull request: [SPARK-14939][SQL] Add FoldablePropagation opt...

2016-05-17 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12719#issuecomment-219882335 **[Test build #58726 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58726/consoleFull)** for PR 12719 at commit [`3206bd0`](https://gi

[GitHub] spark pull request: [SPARK-14939][SQL] Add FoldablePropagation opt...

2016-05-17 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the pull request: https://github.com/apache/spark/pull/12719#issuecomment-219881850 retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-14939][SQL] Add FoldablePropagation opt...

2016-05-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12719#issuecomment-219866626 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-14939][SQL] Add FoldablePropagation opt...

2016-05-17 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12719#issuecomment-219866605 **[Test build #58722 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58722/consoleFull)** for PR 12719 at commit [`3206bd0`](https://g

[GitHub] spark pull request: [SPARK-14939][SQL] Add FoldablePropagation opt...

2016-05-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12719#issuecomment-219866627 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-14939][SQL] Add FoldablePropagation opt...

2016-05-17 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12719#issuecomment-219864486 **[Test build #58722 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58722/consoleFull)** for PR 12719 at commit [`3206bd0`](https://gi

[GitHub] spark pull request: [SPARK-14939][SQL] Add FoldablePropagation opt...

2016-05-17 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the pull request: https://github.com/apache/spark/pull/12719#issuecomment-219864350 Rebased to resolve conflicts. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: [SPARK-14939][SQL] Add FoldablePropagation opt...

2016-05-17 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the pull request: https://github.com/apache/spark/pull/12719#issuecomment-219782448 Ya. I was really flustrated, too. When you read the body of `checkEvaluationWithOptimization` function above, it has the following lines. It adds `Alias`. ```

[GitHub] spark pull request: [SPARK-14939][SQL] Add FoldablePropagation opt...

2016-05-17 Thread cloud-fan
Github user cloud-fan commented on the pull request: https://github.com/apache/spark/pull/12719#issuecomment-219708181 Why is that `Alias` not analyzed? Isn't `FormatNumber(Literal(4.asInstanceOf[Byte]), Literal(3))` a resolved expression? --- If your project is set up for it, you ca

[GitHub] spark pull request: [SPARK-14939][SQL] Add FoldablePropagation opt...

2016-05-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12719#issuecomment-219700590 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-14939][SQL] Add FoldablePropagation opt...

2016-05-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12719#issuecomment-219700589 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-14939][SQL] Add FoldablePropagation opt...

2016-05-17 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12719#issuecomment-219700387 **[Test build #58687 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58687/consoleFull)** for PR 12719 at commit [`d69f521`](https://g

[GitHub] spark pull request: [SPARK-14939][SQL] Add FoldablePropagation opt...

2016-05-17 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the pull request: https://github.com/apache/spark/pull/12719#issuecomment-219688169 At this time, MiMa test passed. Let's wait and see the result. By the way, please note that I only replaced `transformAllExpressions` in this final commit. -

[GitHub] spark pull request: [SPARK-14939][SQL] Add FoldablePropagation opt...

2016-05-17 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12719#issuecomment-219684101 **[Test build #58687 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58687/consoleFull)** for PR 12719 at commit [`d69f521`](https://gi

[GitHub] spark pull request: [SPARK-14939][SQL] Add FoldablePropagation opt...

2016-05-17 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the pull request: https://github.com/apache/spark/pull/12719#issuecomment-219683646 retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-14939][SQL] Add FoldablePropagation opt...

2016-05-17 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the pull request: https://github.com/apache/spark/pull/12719#issuecomment-219683326 Hmm. @cloud-fan . Unfortunately, for `resolved`, we can not pass the tests without that. For example, `StringExpressionsSuite.format_number` has the fo

[GitHub] spark pull request: [SPARK-14939][SQL] Add FoldablePropagation opt...

2016-05-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12719#issuecomment-219682736 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-14939][SQL] Add FoldablePropagation opt...

2016-05-17 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12719#issuecomment-219682710 **[Test build #58686 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58686/consoleFull)** for PR 12719 at commit [`d69f521`](https://g

[GitHub] spark pull request: [SPARK-14939][SQL] Add FoldablePropagation opt...

2016-05-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12719#issuecomment-219682734 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-14939][SQL] Add FoldablePropagation opt...

2016-05-17 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the pull request: https://github.com/apache/spark/pull/12719#issuecomment-219682307 Oh, I think MiMa failure is irrelevant to this PR. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well.

[GitHub] spark pull request: [SPARK-14939][SQL] Add FoldablePropagation opt...

2016-05-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12719#issuecomment-219681775 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-14939][SQL] Add FoldablePropagation opt...

2016-05-17 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12719#issuecomment-219681748 **[Test build #58685 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58685/consoleFull)** for PR 12719 at commit [`97afe31`](https://g

[GitHub] spark pull request: [SPARK-14939][SQL] Add FoldablePropagation opt...

2016-05-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12719#issuecomment-219681780 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-14939][SQL] Add FoldablePropagation opt...

2016-05-17 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12719#issuecomment-219681064 **[Test build #58686 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58686/consoleFull)** for PR 12719 at commit [`d69f521`](https://gi

[GitHub] spark pull request: [SPARK-14939][SQL] Add FoldablePropagation opt...

2016-05-17 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12719#issuecomment-219680042 **[Test build #58685 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58685/consoleFull)** for PR 12719 at commit [`97afe31`](https://gi

[GitHub] spark pull request: [SPARK-14939][SQL] Add FoldablePropagation opt...

2016-05-17 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the pull request: https://github.com/apache/spark/pull/12719#issuecomment-219670004 Thank YOU, @cloud-fan . I fix them soon. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your p

[GitHub] spark pull request: [SPARK-14939][SQL] Add FoldablePropagation opt...

2016-05-17 Thread cloud-fan
Github user cloud-fan commented on the pull request: https://github.com/apache/spark/pull/12719#issuecomment-219668576 Mostly LGTM except some minor comments, thanks for working on it! --- If your project is set up for it, you can reply to this email and have your reply appear on GitH

[GitHub] spark pull request: [SPARK-14939][SQL] Add FoldablePropagation opt...

2016-05-17 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/12719#discussion_r63491459 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -659,6 +660,45 @@ object NullPropagation extends Rule

[GitHub] spark pull request: [SPARK-14939][SQL] Add FoldablePropagation opt...

2016-05-17 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/12719#discussion_r63491368 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -659,6 +660,45 @@ object NullPropagation extends Rule

[GitHub] spark pull request: [SPARK-14939][SQL] Add FoldablePropagation opt...

2016-05-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12719#issuecomment-219502838 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-14939][SQL] Add FoldablePropagation opt...

2016-05-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12719#issuecomment-219502841 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-14939][SQL] Add FoldablePropagation opt...

2016-05-16 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12719#issuecomment-219502550 **[Test build #58650 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58650/consoleFull)** for PR 12719 at commit [`631cea6`](https://g

[GitHub] spark pull request: [SPARK-14939][SQL] Add FoldablePropagation opt...

2016-05-16 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12719#issuecomment-219479060 **[Test build #58650 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58650/consoleFull)** for PR 12719 at commit [`631cea6`](https://gi

[GitHub] spark pull request: [SPARK-14939][SQL] Add FoldablePropagation opt...

2016-05-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12719#issuecomment-218959631 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-14939][SQL] Add FoldablePropagation opt...

2016-05-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12719#issuecomment-218959627 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-14939][SQL] Add FoldablePropagation opt...

2016-05-12 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12719#issuecomment-218959501 **[Test build #58540 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58540/consoleFull)** for PR 12719 at commit [`2d56bc2`](https://g

[GitHub] spark pull request: [SPARK-14939][SQL] Add FoldablePropagation opt...

2016-05-12 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12719#issuecomment-218950690 **[Test build #58540 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58540/consoleFull)** for PR 12719 at commit [`2d56bc2`](https://gi

[GitHub] spark pull request: [SPARK-14939][SQL] Add FoldablePropagation opt...

2016-05-12 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the pull request: https://github.com/apache/spark/pull/12719#issuecomment-218950477 Rebase to resolve conflicts. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: [SPARK-14939][SQL] Add FoldablePropagation opt...

2016-05-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12719#issuecomment-218922635 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-14939][SQL] Add FoldablePropagation opt...

2016-05-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12719#issuecomment-218922631 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-14939][SQL] Add FoldablePropagation opt...

2016-05-12 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12719#issuecomment-218922499 **[Test build #58518 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58518/consoleFull)** for PR 12719 at commit [`61f26f2`](https://g

[GitHub] spark pull request: [SPARK-14939][SQL] Add FoldablePropagation opt...

2016-05-12 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12719#issuecomment-218909190 **[Test build #58518 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58518/consoleFull)** for PR 12719 at commit [`61f26f2`](https://gi

[GitHub] spark pull request: [SPARK-14939][SQL] Add FoldablePropagation opt...

2016-05-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12719#issuecomment-218856347 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-14939][SQL] Add FoldablePropagation opt...

2016-05-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12719#issuecomment-218856345 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-14939][SQL] Add FoldablePropagation opt...

2016-05-12 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12719#issuecomment-218856024 **[Test build #58508 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58508/consoleFull)** for PR 12719 at commit [`18e9e05`](https://g

[GitHub] spark pull request: [SPARK-14939][SQL] Add FoldablePropagation opt...

2016-05-12 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12719#issuecomment-218833125 **[Test build #58508 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58508/consoleFull)** for PR 12719 at commit [`18e9e05`](https://gi

[GitHub] spark pull request: [SPARK-14939][SQL] Add FoldablePropagation opt...

2016-05-12 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/12719#discussion_r63063384 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -618,6 +619,48 @@ object NullPropagation extends

[GitHub] spark pull request: [SPARK-14939][SQL] Add FoldablePropagation opt...

2016-05-12 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/12719#discussion_r63060729 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/FoldablePropagationSuite.scala --- @@ -0,0 +1,138 @@ +/* + * Lic

[GitHub] spark pull request: [SPARK-14939][SQL] Add FoldablePropagation opt...

2016-05-12 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/12719#discussion_r63060451 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -618,6 +619,48 @@ object NullPropagation extends

[GitHub] spark pull request: [SPARK-14939][SQL] Add FoldablePropagation opt...

2016-05-12 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the pull request: https://github.com/apache/spark/pull/12719#issuecomment-218825375 Hi, @cloud-fan . For `AggregateOptimizeSuite.scala`, we need to use `caseSensitiveAnalysis`. Especially, `test("remove repetition in grouping expression")`.

[GitHub] spark pull request: [SPARK-14939][SQL] Add FoldablePropagation opt...

2016-05-11 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/12719#discussion_r62957143 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -618,6 +619,48 @@ object NullPropagation extends Rule

[GitHub] spark pull request: [SPARK-14939][SQL] Add FoldablePropagation opt...

2016-05-11 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/12719#discussion_r62956465 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -618,6 +619,48 @@ object NullPropagation extends Rule

[GitHub] spark pull request: [SPARK-14939][SQL] Add FoldablePropagation opt...

2016-05-11 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/12719#discussion_r62956411 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/FoldablePropagationSuite.scala --- @@ -0,0 +1,138 @@ +/* + * License

[GitHub] spark pull request: [SPARK-14939][SQL] Add FoldablePropagation opt...

2016-05-11 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/12719#discussion_r62956372 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/AggregateOptimizeSuite.scala --- @@ -34,29 +34,34 @@ class AggregateOptimize

[GitHub] spark pull request: [SPARK-14939][SQL] Add FoldablePropagation opt...

2016-05-11 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/12719#discussion_r62956390 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/EliminateSortsSuite.scala --- @@ -69,4 +70,16 @@ class EliminateSortsSuite e

[GitHub] spark pull request: [SPARK-14939][SQL] Add FoldablePropagation opt...

2016-05-11 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/12719#discussion_r62956339 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/AggregateOptimizeSuite.scala --- @@ -34,29 +34,34 @@ class AggregateOptimize

[GitHub] spark pull request: [SPARK-14939][SQL] Add FoldablePropagation opt...

2016-05-11 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the pull request: https://github.com/apache/spark/pull/12719#issuecomment-218560535 Hi, @cloud-fan . Now, it passed again. For the remaining comments, I can do that but the reason is not clear to me. > I prefer to handle one Project each ro

[GitHub] spark pull request: [SPARK-14939][SQL] Add FoldablePropagation opt...

2016-05-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12719#issuecomment-218550436 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-14939][SQL] Add FoldablePropagation opt...

2016-05-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12719#issuecomment-218550435 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-14939][SQL] Add FoldablePropagation opt...

2016-05-11 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12719#issuecomment-218550113 **[Test build #58385 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58385/consoleFull)** for PR 12719 at commit [`f379a09`](https://g

[GitHub] spark pull request: [SPARK-14939][SQL] Add FoldablePropagation opt...

2016-05-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12719#issuecomment-218525102 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-14939][SQL] Add FoldablePropagation opt...

2016-05-11 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12719#issuecomment-218526294 **[Test build #58385 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58385/consoleFull)** for PR 12719 at commit [`f379a09`](https://gi

[GitHub] spark pull request: [SPARK-14939][SQL] Add FoldablePropagation opt...

2016-05-11 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12719#issuecomment-218525058 **[Test build #58382 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58382/consoleFull)** for PR 12719 at commit [`7ca8b2a`](https://g

[GitHub] spark pull request: [SPARK-14939][SQL] Add FoldablePropagation opt...

2016-05-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12719#issuecomment-218525099 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-14939][SQL] Add FoldablePropagation opt...

2016-05-11 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the pull request: https://github.com/apache/spark/pull/12719#issuecomment-218518949 There was build break on master branched. I made a hotfix for that. After merging that, I'll retry this. --- If your project is set up for it, you can reply to t

  1   2   3   >