[GitHub] spark issue #21764: [SPARK-24802] Optimization Rule Exclusion

2018-07-22 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21764 Also, can you update the title? You need to at least add `[SQL]` in the title: https://github.com/apache/spark/pull/21764#issuecomment-405832763 ---

[GitHub] spark issue #21764: [SPARK-24802] Optimization Rule Exclusion

2018-07-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21764 **[Test build #93419 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93419/testReport)** for PR 21764 at commit

[GitHub] spark issue #21764: [SPARK-24802] Optimization Rule Exclusion

2018-07-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21764 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21764: [SPARK-24802] Optimization Rule Exclusion

2018-07-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21764 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #21764: [SPARK-24802] Optimization Rule Exclusion

2018-07-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21764 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93400/ Test PASSed. ---

[GitHub] spark issue #21764: [SPARK-24802] Optimization Rule Exclusion

2018-07-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21764 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21764: [SPARK-24802] Optimization Rule Exclusion

2018-07-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21764 **[Test build #93400 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93400/testReport)** for PR 21764 at commit

[GitHub] spark issue #21764: [SPARK-24802] Optimization Rule Exclusion

2018-07-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21764 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21764: [SPARK-24802] Optimization Rule Exclusion

2018-07-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21764 **[Test build #93400 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93400/testReport)** for PR 21764 at commit

[GitHub] spark issue #21764: [SPARK-24802] Optimization Rule Exclusion

2018-07-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21764 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #21764: [SPARK-24802] Optimization Rule Exclusion

2018-07-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21764 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93220/ Test PASSed. ---

[GitHub] spark issue #21764: [SPARK-24802] Optimization Rule Exclusion

2018-07-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21764 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21764: [SPARK-24802] Optimization Rule Exclusion

2018-07-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21764 **[Test build #93220 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93220/testReport)** for PR 21764 at commit

[GitHub] spark issue #21764: [SPARK-24802] Optimization Rule Exclusion

2018-07-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21764 **[Test build #93220 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93220/testReport)** for PR 21764 at commit

[GitHub] spark issue #21764: [SPARK-24802] Optimization Rule Exclusion

2018-07-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21764 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #21764: [SPARK-24802] Optimization Rule Exclusion

2018-07-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21764 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21764: [SPARK-24802] Optimization Rule Exclusion

2018-07-18 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21764 btw, I feel the title is a little obscure and how about `[SPARK-24802][SQL] Add a new config for Optimization Rule Exclusion`? ---

[GitHub] spark issue #21764: [SPARK-24802] Optimization Rule Exclusion

2018-07-18 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21764 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21764: [SPARK-24802] Optimization Rule Exclusion

2018-07-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21764 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93217/ Test FAILed. ---

[GitHub] spark issue #21764: [SPARK-24802] Optimization Rule Exclusion

2018-07-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21764 **[Test build #93217 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93217/testReport)** for PR 21764 at commit

[GitHub] spark issue #21764: [SPARK-24802] Optimization Rule Exclusion

2018-07-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21764 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21764: [SPARK-24802] Optimization Rule Exclusion

2018-07-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21764 **[Test build #93217 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93217/testReport)** for PR 21764 at commit

[GitHub] spark issue #21764: [SPARK-24802] Optimization Rule Exclusion

2018-07-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21764 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21764: [SPARK-24802] Optimization Rule Exclusion

2018-07-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21764 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #21764: [SPARK-24802] Optimization Rule Exclusion

2018-07-14 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21764 ok, thx for the kind explanation. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #21764: [SPARK-24802] Optimization Rule Exclusion

2018-07-14 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/21764 @maropu This is for advanced end users or Spark developers. External conf looks fine, but I have to admit this might be rarely used. BTW, after having this conf, we can deprecate a few internal

[GitHub] spark issue #21764: [SPARK-24802] Optimization Rule Exclusion

2018-07-14 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21764 @gatorsmile aha, ok. We need to make this option not `internal` but `external`? BTW, the interfaces to add/delete optimizer rules (addition via `ExperimentalMethods` and deletion via

[GitHub] spark issue #21764: [SPARK-24802] Optimization Rule Exclusion

2018-07-14 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/21764 Let me give an example. The ticket https://issues.apache.org/jira/browse/SPARK-24624 shows a common issue in which our optimizer does not work well in all the cases. It is a bug but our users

[GitHub] spark issue #21764: [SPARK-24802] Optimization Rule Exclusion

2018-07-14 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21764 Do you have concrete usecases in your business? Basically, I think the optimizer is a black-box for most users and they don't easily understand how it works correctly when excluding some rules. Are

[GitHub] spark issue #21764: [SPARK-24802] Optimization Rule Exclusion

2018-07-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21764 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21764: [SPARK-24802] Optimization Rule Exclusion

2018-07-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21764 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92992/ Test PASSed. ---

[GitHub] spark issue #21764: [SPARK-24802] Optimization Rule Exclusion

2018-07-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21764 **[Test build #92992 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92992/testReport)** for PR 21764 at commit

[GitHub] spark issue #21764: [SPARK-24802] Optimization Rule Exclusion

2018-07-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21764 **[Test build #92992 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92992/testReport)** for PR 21764 at commit

[GitHub] spark issue #21764: [SPARK-24802] Optimization Rule Exclusion

2018-07-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21764 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/942/

[GitHub] spark issue #21764: [SPARK-24802] Optimization Rule Exclusion

2018-07-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21764 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional