[GitHub] spark issue #16245: [SPARK-18824][SQL] Add optimizer rule to reorder Filter ...
Github user viirya commented on the issue: https://github.com/apache/spark/pull/16245 Close this for now since no further feedback. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16245: [SPARK-18824][SQL] Add optimizer rule to reorder Filter ...
Github user viirya commented on the issue: https://github.com/apache/spark/pull/16245 It is true of course you can construct a combination of complex string operations and compare it with a simple Scala UDF. But as you said, the previous claim is true in most of time. I also think Scala UDF is usually used to write complex logic which can't be achieved by built-in expressions. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16245: [SPARK-18824][SQL] Add optimizer rule to reorder Filter ...
Github user chenghao-intel commented on the issue: https://github.com/apache/spark/pull/16245 I think that's true in most of time for`Scala UDF needs extra conversion between internal format and external format on input and out`, not all of the time, for example, some built-in string based operations and its combinations are also quite heavy in evaluation, and most likely, this probably causes concern for an experienced SQL developers, to write an optimal(business related short-cutting logic) SQL expressions. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16245: [SPARK-18824][SQL] Add optimizer rule to reorder Filter ...
Github user viirya commented on the issue: https://github.com/apache/spark/pull/16245 I think most of time it should be as Scala UDF needs extra conversion between internal format and external format on input and out. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16245: [SPARK-18824][SQL] Add optimizer rule to reorder Filter ...
Github user chenghao-intel commented on the issue: https://github.com/apache/spark/pull/16245 Actually I doubt this is really an optimization, as the assumption of Scala UDF is slower than the non-SCALA UDF probably not always true. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16245: [SPARK-18824][SQL] Add optimizer rule to reorder Filter ...
Github user viirya commented on the issue: https://github.com/apache/spark/pull/16245 ping @hvanhovell @rxin again. Please let me know if you have more comments on this. Thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16245: [SPARK-18824][SQL] Add optimizer rule to reorder Filter ...
Github user viirya commented on the issue: https://github.com/apache/spark/pull/16245 ping @hvanhovell @rxin Any thing I need to address more? Thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16245: [SPARK-18824][SQL] Add optimizer rule to reorder Filter ...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16245 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/70011/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16245: [SPARK-18824][SQL] Add optimizer rule to reorder Filter ...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16245 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16245: [SPARK-18824][SQL] Add optimizer rule to reorder Filter ...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16245 **[Test build #70011 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70011/consoleFull)** for PR 16245 at commit [`63c50b8`](https://github.com/apache/spark/commit/63c50b8066a77506c6751710d5b5b5edb77ca933). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16245: [SPARK-18824][SQL] Add optimizer rule to reorder Filter ...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16245 **[Test build #70011 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70011/consoleFull)** for PR 16245 at commit [`63c50b8`](https://github.com/apache/spark/commit/63c50b8066a77506c6751710d5b5b5edb77ca933). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16245: [SPARK-18824][SQL] Add optimizer rule to reorder Filter ...
Github user viirya commented on the issue: https://github.com/apache/spark/pull/16245 retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16245: [SPARK-18824][SQL] Add optimizer rule to reorder Filter ...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16245 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/70005/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16245: [SPARK-18824][SQL] Add optimizer rule to reorder Filter ...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16245 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16245: [SPARK-18824][SQL] Add optimizer rule to reorder Filter ...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16245 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16245: [SPARK-18824][SQL] Add optimizer rule to reorder Filter ...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16245 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/6/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16245: [SPARK-18824][SQL] Add optimizer rule to reorder Filter ...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16245 **[Test build #6 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/6/consoleFull)** for PR 16245 at commit [`66a3d98`](https://github.com/apache/spark/commit/66a3d983d978b902858a34dde992640a489f5351). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16245: [SPARK-18824][SQL] Add optimizer rule to reorder Filter ...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16245 **[Test build #70005 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70005/consoleFull)** for PR 16245 at commit [`63c50b8`](https://github.com/apache/spark/commit/63c50b8066a77506c6751710d5b5b5edb77ca933). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
