[GitHub] spark pull request: [SPARK-13185][SQL] Improve the performance of ...

2016-02-13 Thread adrian-wang
Github user adrian-wang commented on the pull request: https://github.com/apache/spark/pull/11071#issuecomment-183791219 @srowen you are right. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-13185][SQL] Improve the performance of ...

2016-02-06 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/11071#issuecomment-180793931 @adrian-wang but the `@transient` was applied to the whole reference to the `Map`. This doesn't make sense. The way to avoid a big cache is with weak keys, which has not

[GitHub] spark pull request: [SPARK-13185][SQL] Improve the performance of ...

2016-02-04 Thread carsonwang
Github user carsonwang closed the pull request at: https://github.com/apache/spark/pull/11071 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-13185][SQL] Improve the performance of ...

2016-02-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11071#issuecomment-180223605 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-13185][SQL] Improve the performance of ...

2016-02-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11071#issuecomment-180223604 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-13185][SQL] Improve the performance of ...

2016-02-04 Thread carsonwang
Github user carsonwang commented on the pull request: https://github.com/apache/spark/pull/11071#issuecomment-180207538 I have a sub query like this `SELECT a, b, c FROM table UV WHERE (datediff(UV.visitDate, '1997-01-01')>=0 AND datediff(UV.visitDate, '2015-01-01')<=0)) ` When p

[GitHub] spark pull request: [SPARK-13185][SQL] Improve the performance of ...

2016-02-04 Thread adrian-wang
Github user adrian-wang commented on the pull request: https://github.com/apache/spark/pull/11071#issuecomment-180201265 The map key could be like "UTC+01:00". "American/Los Angeles", "PST", etc., they are already cached in `getTimeZone`, but the method itself is a synchronized one.

[GitHub] spark pull request: [SPARK-13185][SQL] Improve the performance of ...

2016-02-04 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/11071#issuecomment-180200750 transient is wrong for a static member. What are you trying to do, avoid a large cache? There are ways to do this but its not this. but how many timezones could there be

[GitHub] spark pull request: [SPARK-13185][SQL] Improve the performance of ...

2016-02-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11071#issuecomment-180199774 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-13185][SQL] Improve the performance of ...

2016-02-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11071#issuecomment-180199780 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-13185][SQL] Improve the performance of ...

2016-02-04 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11071#issuecomment-180199036 **[Test build #50797 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/50797/consoleFull)** for PR 11071 at commit [`72b31c8`](https://g

[GitHub] spark pull request: [SPARK-13185][SQL] Improve the performance of ...

2016-02-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11071#issuecomment-180192861 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-13185][SQL] Improve the performance of ...

2016-02-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11071#issuecomment-180192869 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-13185][SQL] Improve the performance of ...

2016-02-04 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11071#issuecomment-180191627 **[Test build #50794 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/50794/consoleFull)** for PR 11071 at commit [`72b31c8`](https://g

[GitHub] spark pull request: [SPARK-13185][SQL] Improve the performance of ...

2016-02-04 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11071#issuecomment-180175835 **[Test build #50797 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/50797/consoleFull)** for PR 11071 at commit [`72b31c8`](https://gi

[GitHub] spark pull request: [SPARK-13185][SQL] Improve the performance of ...

2016-02-04 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11071#issuecomment-180172844 **[Test build #50794 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/50794/consoleFull)** for PR 11071 at commit [`72b31c8`](https://gi

[GitHub] spark pull request: [SPARK-13185][SQL] Improve the performance of ...

2016-02-04 Thread carsonwang
Github user carsonwang commented on the pull request: https://github.com/apache/spark/pull/11071#issuecomment-180172808 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-13185][SQL] Improve the performance of ...

2016-02-04 Thread carsonwang
Github user carsonwang commented on a diff in the pull request: https://github.com/apache/spark/pull/11071#discussion_r51972422 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/DateTimeUtils.scala --- @@ -55,10 +56,19 @@ object DateTimeUtils { // thi

[GitHub] spark pull request: [SPARK-13185][SQL] Improve the performance of ...

2016-02-04 Thread adrian-wang
Github user adrian-wang commented on a diff in the pull request: https://github.com/apache/spark/pull/11071#discussion_r51853631 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/DateTimeUtils.scala --- @@ -55,10 +56,19 @@ object DateTimeUtils { // th

[GitHub] spark pull request: [SPARK-13185][SQL] Improve the performance of ...

2016-02-04 Thread adrian-wang
Github user adrian-wang commented on a diff in the pull request: https://github.com/apache/spark/pull/11071#discussion_r51852765 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/DateTimeUtils.scala --- @@ -55,10 +56,19 @@ object DateTimeUtils { // th

[GitHub] spark pull request: [SPARK-13185][SQL] Improve the performance of ...

2016-02-04 Thread adrian-wang
Github user adrian-wang commented on a diff in the pull request: https://github.com/apache/spark/pull/11071#discussion_r51851099 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/DateTimeUtils.scala --- @@ -55,10 +56,19 @@ object DateTimeUtils { // th

[GitHub] spark pull request: [SPARK-13185][SQL] Improve the performance of ...

2016-02-04 Thread adrian-wang
Github user adrian-wang commented on a diff in the pull request: https://github.com/apache/spark/pull/11071#discussion_r51851087 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/DateTimeUtils.scala --- @@ -55,10 +56,19 @@ object DateTimeUtils { // th

[GitHub] spark pull request: [SPARK-13185][SQL] Improve the performance of ...

2016-02-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11071#issuecomment-179741770 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-13185][SQL] Improve the performance of ...

2016-02-04 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11071#issuecomment-179741404 **[Test build #50746 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/50746/consoleFull)** for PR 11071 at commit [`0ab90ed`](https://g

[GitHub] spark pull request: [SPARK-13185][SQL] Improve the performance of ...

2016-02-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11071#issuecomment-179741773 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-13185][SQL] Improve the performance of ...

2016-02-04 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11071#issuecomment-179713645 **[Test build #50746 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/50746/consoleFull)** for PR 11071 at commit [`0ab90ed`](https://gi

[GitHub] spark pull request: [SPARK-13185][SQL] Improve the performance of ...

2016-02-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11071#issuecomment-179711573 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-13185][SQL] Improve the performance of ...

2016-02-04 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11071#issuecomment-179711566 **[Test build #50744 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/50744/consoleFull)** for PR 11071 at commit [`cb9b157`](https://g

[GitHub] spark pull request: [SPARK-13185][SQL] Improve the performance of ...

2016-02-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11071#issuecomment-179711574 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-13185][SQL] Improve the performance of ...

2016-02-04 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11071#issuecomment-179711327 **[Test build #50744 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/50744/consoleFull)** for PR 11071 at commit [`cb9b157`](https://gi

[GitHub] spark pull request: [SPARK-13185][SQL] Improve the performance of ...

2016-02-04 Thread carsonwang
GitHub user carsonwang opened a pull request: https://github.com/apache/spark/pull/11071 [SPARK-13185][SQL] Improve the performance of DateTimeUtils by reusing TimeZone and Calendar objects It is expensive to create java TimeZone and Calendar objects in each method of DateTimeUtils