[GitHub] spark issue #15048: [SPARK-17409] [SQL] Do Not Optimize Query in CTAS More T...

2016-10-12 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/15048 Yeah, based on my understanding, it should cover the hive serde table. I will submit a PR to make sure it and also include the test case you provided above. Thank you! --- If your project is se

[GitHub] spark issue #15048: [SPARK-17409] [SQL] Do Not Optimize Query in CTAS More T...

2016-10-12 Thread yhuai
Github user yhuai commented on the issue: https://github.com/apache/spark/pull/15048 Also, can we add a test for hive tables? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enab

[GitHub] spark issue #15048: [SPARK-17409] [SQL] Do Not Optimize Query in CTAS More T...

2016-10-12 Thread yhuai
Github user yhuai commented on the issue: https://github.com/apache/spark/pull/15048 Also, another good test for this is ``` val df = sql("select 0 as id") df.registerTempTable("foo") val df2 = sql("""select * from foo group by id""") df2.write.mode("overwrite").sav

[GitHub] spark issue #15048: [SPARK-17409] [SQL] Do Not Optimize Query in CTAS More T...

2016-10-12 Thread yhuai
Github user yhuai commented on the issue: https://github.com/apache/spark/pull/15048 Thanks! btw, does this patch cover hive tables? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this featu

[GitHub] spark issue #15048: [SPARK-17409] [SQL] Do Not Optimize Query in CTAS More T...

2016-10-12 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/15048 Yeah. We should backport it to 2.0 Yeah. It affects both data source tables and hive serde tables. To fix it in Spark 2.0, we need to rewrite the fix since Spark 2.0 does not have a unif

[GitHub] spark issue #15048: [SPARK-17409] [SQL] Do Not Optimize Query in CTAS More T...

2016-10-12 Thread yhuai
Github user yhuai commented on the issue: https://github.com/apache/spark/pull/15048 @gatorsmile Also, does it affect `CTAS` for creating a hive serde table? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark issue #15048: [SPARK-17409] [SQL] Do Not Optimize Query in CTAS More T...

2016-10-12 Thread yhuai
Github user yhuai commented on the issue: https://github.com/apache/spark/pull/15048 @gatorsmile We should also backport this to branch 2.0, right? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark issue #15048: [SPARK-17409] [SQL] Do Not Optimize Query in CTAS More T...

2016-09-14 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/15048 thanks, merging to master! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wi

[GitHub] spark issue #15048: [SPARK-17409] [SQL] Do Not Optimize Query in CTAS More T...

2016-09-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15048 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65283/ Test PASSed. ---

[GitHub] spark issue #15048: [SPARK-17409] [SQL] Do Not Optimize Query in CTAS More T...

2016-09-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15048 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15048: [SPARK-17409] [SQL] Do Not Optimize Query in CTAS More T...

2016-09-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15048 **[Test build #65283 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65283/consoleFull)** for PR 15048 at commit [`4c3c955`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15048: [SPARK-17409] [SQL] Do Not Optimize Query in CTAS More T...

2016-09-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15048 **[Test build #65283 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65283/consoleFull)** for PR 15048 at commit [`4c3c955`](https://github.com/apache/spark/commit/4

[GitHub] spark issue #15048: [SPARK-17409] [SQL] Do Not Optimize Query in CTAS More T...

2016-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15048 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15048: [SPARK-17409] [SQL] Do Not Optimize Query in CTAS More T...

2016-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15048 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65233/ Test PASSed. ---

[GitHub] spark issue #15048: [SPARK-17409] [SQL] Do Not Optimize Query in CTAS More T...

2016-09-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15048 **[Test build #65233 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65233/consoleFull)** for PR 15048 at commit [`ae335ae`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15048: [SPARK-17409] [SQL] Do Not Optimize Query in CTAS More T...

2016-09-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15048 **[Test build #65233 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65233/consoleFull)** for PR 15048 at commit [`ae335ae`](https://github.com/apache/spark/commit/a

[GitHub] spark issue #15048: [SPARK-17409] [SQL] Do Not Optimize Query in CTAS More T...

2016-09-11 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/15048 @hvanhovell Sure, will do it. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15048: [SPARK-17409] [SQL] Do Not Optimize Query in CTAS More T...

2016-09-11 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/15048 @gatorsmile so should we check all commands? It might also be an idea to have `Command` extend `LeafNode` (and make children `final`). I think @davies did something similar for https://github.com

[GitHub] spark issue #15048: [SPARK-17409] [SQL] Do Not Optimize Query in CTAS More T...

2016-09-10 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/15048 cc @cloud-fan @yhuai @davies --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled an

[GitHub] spark issue #15048: [SPARK-17409] [SQL] Do Not Optimize Query in CTAS More T...

2016-09-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15048 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15048: [SPARK-17409] [SQL] Do Not Optimize Query in CTAS More T...

2016-09-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15048 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65217/ Test PASSed. ---

[GitHub] spark issue #15048: [SPARK-17409] [SQL] Do Not Optimize Query in CTAS More T...

2016-09-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15048 **[Test build #65217 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65217/consoleFull)** for PR 15048 at commit [`da7deed`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15048: [SPARK-17409] [SQL] Do Not Optimize Query in CTAS More T...

2016-09-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15048 **[Test build #65217 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65217/consoleFull)** for PR 15048 at commit [`da7deed`](https://github.com/apache/spark/commit/d