[GitHub] spark issue #19021: [SPARK-21603][SQL][FOLLOW-UP] Change the default value o...
Github user viirya commented on the issue: https://github.com/apache/spark/pull/19021 @maropu Should be a good idea. Especially the number of lines of code may not be intuitive to set for this purpose. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19021: [SPARK-21603][SQL][FOLLOW-UP] Change the default value o...
Github user maropu commented on the issue: https://github.com/apache/spark/pull/19021 Just for your info, again, I looked into this issue in TPC-DS quries; I added [some code](https://github.com/apache/spark/compare/master...maropu:SPARK-21603-FOLLOWUP-3) to check the actual bytecode size of these quries and I found the gen'd function in Q17/Q66 only had too-long bytecode over `8000`: ``` = TPCDS QUERY BENCHMARK OUTPUT FOR q17 = 17/08/23 14:45:02 WARN CodeGenerator: GeneratedClass.agg_doAggregateWithKeys is too large to do JIT compilation on HotSpot; the size of agg_doAggregateWithKeys is 17665; the limit is 8000 = TPCDS QUERY BENCHMARK OUTPUT FOR q66 = 17/08/23 14:55:39 WARN CodeGenerator: GeneratedClass.agg_doAggregateWithKeys is too large to do JIT compilation on HotSpot; the size of agg_doAggregateWithKeys is 11012; the limit is 8000 17/08/23 14:55:39 WARN CodeGenerator: GeneratedClass.agg_doAggregateWithKeys is too large to do JIT compilation on HotSpot; the size of agg_doAggregateWithKeys is 13420; the limit is 8000 17/08/23 14:55:39 WARN CodeGenerator: GeneratedClass.agg_doAggregateWithKeys is too large to do JIT compilation on HotSpot; the size of agg_doAggregateWithKeys is 16641; the limit is 8000 ``` BTW, why we don't check if gen'd bytecode size is over `8000` directly instead of code line num. in #18810? cc: @gatorsmile @viirya @kiszk --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19021: [SPARK-21603][SQL][FOLLOW-UP] Change the default value o...
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19021 Thanks! Merging to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19021: [SPARK-21603][SQL][FOLLOW-UP] Change the default value o...
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19021 cc @rednaxelafx --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19021: [SPARK-21603][SQL][FOLLOW-UP] Change the default value o...
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19021 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19021: [SPARK-21603][SQL][FOLLOW-UP] Change the default value o...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19021 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19021: [SPARK-21603][SQL][FOLLOW-UP] Change the default value o...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19021 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80976/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19021: [SPARK-21603][SQL][FOLLOW-UP] Change the default value o...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19021 **[Test build #80976 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80976/testReport)** for PR 19021 at commit [`d0bc6a6`](https://github.com/apache/spark/commit/d0bc6a62cd58147c9809d9c99fd4452cf7e40aff). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19021: [SPARK-21603][SQL][FOLLOW-UP] Change the default value o...
Github user maropu commented on the issue: https://github.com/apache/spark/pull/19021 Also, I talked with @viirya [here](https://github.com/apache/spark/pull/18931#issuecomment-323697425) and `-1` seems to be more natural to disable this too-long-function option. If we don't strongly disagree with this, I could make [another trivial pr](https://github.com/apache/spark/compare/master...maropu:SPARK-21603-FOLLOWUP-2) to fix, too. @gatorsmile --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19021: [SPARK-21603][SQL][FOLLOW-UP] Change the default value o...
Github user maropu commented on the issue: https://github.com/apache/spark/pull/19021 Got the numbers and see them [here](https://docs.google.com/spreadsheets/d/10-tmFdYcCQpYwhBV4BFdWWCa8LqXdEqAe2WoP4Abfqk/edit?usp=sharing). The numbers seems to be the almost same with the master just before #18810 merged. @gatorsmile @viirya --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19021: [SPARK-21603][SQL][FOLLOW-UP] Change the default value o...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19021 **[Test build #80976 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80976/testReport)** for PR 19021 at commit [`d0bc6a6`](https://github.com/apache/spark/commit/d0bc6a62cd58147c9809d9c99fd4452cf7e40aff). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19021: [SPARK-21603][SQL][FOLLOW-UP] Change the default value o...
Github user maropu commented on the issue: https://github.com/apache/spark/pull/19021 I'll check performance changes in TPC-DS. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
