[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-11-03 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/9383#issuecomment-153506241 Yep, confirmed and reproduced. If we just wanted to address that bug, we could do something like #9447, but I think the broader set of changes here are a good idea.

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-11-03 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/9383#issuecomment-153504624 There was a problem in my test-selection regex... trying again. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-11-03 Thread davies
Github user davies commented on the pull request: https://github.com/apache/spark/pull/9383#issuecomment-153503203 I tried with master, it did failed with the sam exception: ``` [info] *** 1 TEST FAILED *** [error] Failed: Total 6, Failed 1, Errors 0, Passed 5 [erro

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-11-03 Thread yhuai
Github user yhuai commented on the pull request: https://github.com/apache/spark/pull/9383#issuecomment-153501959 So, you tried `TungstenAggregationQueryWithControlledFallbackSuite` and it did not fail? --- If your project is set up for it, you can reply to this email and have your r

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-11-03 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/9383#issuecomment-153500615 @davies, @yhuai: I tried taking two sample regression test queries that you listed here and putting them into AggregationQuerySuite, which should automatically trigger

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-11-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9383#issuecomment-153496764 **[Test build #44959 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44959/consoleFull)** for PR 9383 at commit [`6f3bb15`](https://gith

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9383#issuecomment-153496072 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9383#issuecomment-153496037 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-11-03 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/9383#issuecomment-153492838 I filed https://issues.apache.org/jira/browse/SPARK-11486 to describe the String aggregation bug. --- If your project is set up for it, you can reply to this email an

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-11-03 Thread davies
Github user davies commented on the pull request: https://github.com/apache/spark/pull/9383#issuecomment-153490648 Will add that into this PR, working on it. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-11-03 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/9383#issuecomment-153489816 What do you think about adding a dedicated regression test for that one case as part of a separate PR? I'll try that now. --- If your project is set up for it, you ca

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-11-03 Thread yhuai
Github user yhuai commented on the pull request: https://github.com/apache/spark/pull/9383#issuecomment-153485327 Maybe `sqlContext.range(10).selectExpr("id", "repeat(id, 20) as s").groupBy("s").agg($"s")` is a better case. --- If your project is set up for it, you can reply to t

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-11-03 Thread yhuai
Github user yhuai commented on the pull request: https://github.com/apache/spark/pull/9383#issuecomment-153482419 @JoshRosen The cause is that if the buffer is an empty one (e.g. we do not have agg function or all agg functions do not use any buffer slot), the initialInputBufferOffset

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-11-03 Thread davies
Github user davies commented on the pull request: https://github.com/apache/spark/pull/9383#issuecomment-153466959 This failure could be reproduce by ``` sqlContext.range(N).selectExpr("id", "repeat(id, 20) as s").groupBy("s").max("id").count() ``` Maybe related t

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-11-03 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/9383#issuecomment-153451560 I _do_ think that the change in this patch makes sense, but I'd like to really get to the bottom of why the old code malfunctioned. --- If your project is set up for

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-11-03 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/9383#issuecomment-153451108 @davies: in the stacktrace that you posted, it looks like it's failing when trying to update a string column when extracting a partial aggregate result from an input r

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-11-03 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/9383#discussion_r43787218 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/UnsafeFixedWidthAggregationMapSuite.scala --- @@ -291,28 +278,27 @@ class UnsafeFixedWidth

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-11-03 Thread yhuai
Github user yhuai commented on the pull request: https://github.com/apache/spark/pull/9383#issuecomment-153448265 @JoshRosen @davies has a case to reproduce that problem --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-11-03 Thread davies
Github user davies commented on the pull request: https://github.com/apache/spark/pull/9383#issuecomment-153447782 @JoshRosen When the grouping key is StringType, but no column in buffer, then got: ``` java.lang.UnsupportedOperationException at org.apache.spark.sql.cat

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-11-03 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/9383#issuecomment-153447195 Do you remember what the symptoms were? Just curious. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. I

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-11-03 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/9383#discussion_r43786489 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/UnsafeFixedWidthAggregationMapSuite.scala --- @@ -291,28 +278,27 @@ class UnsafeFixedWidthAgg

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-11-03 Thread davies
Github user davies commented on the pull request: https://github.com/apache/spark/pull/9383#issuecomment-153447090 @JoshRosen I meant the old implementation of switching, it's broken in master (not 1.5 branch), had not figured out the root cause. --- If your project is set up for it,

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-11-03 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/9383#discussion_r43786437 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/UnsafeFixedWidthAggregationMapSuite.scala --- @@ -291,28 +278,27 @@ class UnsafeFixedWidthAgg

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-11-03 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/9383#discussion_r43786236 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/TungstenAggregationIterator.scala --- @@ -762,15 +679,7 @@ class TungstenAggregatio

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-11-03 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/9383#discussion_r43786160 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/UnsafeFixedWidthAggregationMapSuite.scala --- @@ -291,28 +278,27 @@ class UnsafeFixedWidth

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-11-03 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/9383#discussion_r43786023 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/UnsafeFixedWidthAggregationMapSuite.scala --- @@ -291,28 +278,27 @@ class UnsafeFixedWidth

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-11-03 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/9383#discussion_r43785559 --- Diff: sql/core/src/main/java/org/apache/spark/sql/execution/UnsafeKVExternalSorter.java --- @@ -142,6 +142,10 @@ public void insertKV(UnsafeRow key, Un

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-11-03 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/9383#discussion_r43785456 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/TungstenAggregationIterator.scala --- @@ -473,24 +473,28 @@ class TungstenAggreg

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-11-03 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/9383#discussion_r43785059 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/TungstenAggregationIterator.scala --- @@ -762,15 +679,7 @@ class TungstenAggrega

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-11-03 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/9383#discussion_r43784097 --- Diff: core/src/main/java/org/apache/spark/util/collection/unsafe/sort/UnsafeExternalSorter.java --- @@ -386,6 +386,18 @@ public void insertKVRecord(Obj

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-11-03 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/9383#issuecomment-153440260 @davies, the block comment at the top of `TungstenAggregationIterator` is now out-of-date; do you mind updating it to reflect the new behavior? --- If your project is

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-11-03 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/9383#issuecomment-153434983 > Currently, the old one is broken, I'd to remove that one. @davies, are you referring to the old Aggregate1 interface or the old implementation of sort fallba

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9383#issuecomment-153205634 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9383#issuecomment-153205632 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-11-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9383#issuecomment-153205520 **[Test build #44834 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44834/consoleFull)** for PR 9383 at commit [`df44fc6`](https://git

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-11-02 Thread davies
Github user davies commented on the pull request: https://github.com/apache/spark/pull/9383#issuecomment-153201802 ping @yhuai @JoshRosen --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-11-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9383#issuecomment-153201546 **[Test build #1970 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/1970/consoleFull)** for PR 9383 at commit [`df44fc6`](https://gi

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-11-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9383#issuecomment-153189652 **[Test build #44830 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44830/consoleFull)** for PR 9383 at commit [`3864095`](https://git

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9383#issuecomment-153189858 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9383#issuecomment-153189866 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-11-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9383#issuecomment-153167316 **[Test build #44834 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44834/consoleFull)** for PR 9383 at commit [`df44fc6`](https://gith

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9383#issuecomment-153164295 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-11-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9383#issuecomment-153165332 **[Test build #1970 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/1970/consoleFull)** for PR 9383 at commit [`df44fc6`](https://git

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9383#issuecomment-153164966 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9383#issuecomment-153164925 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-11-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9383#issuecomment-153164208 **[Test build #44823 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44823/consoleFull)** for PR 9383 at commit [`2e341f5`](https://git

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9383#issuecomment-153164298 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-11-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9383#issuecomment-153156635 **[Test build #44830 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44830/consoleFull)** for PR 9383 at commit [`3864095`](https://gith

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9383#issuecomment-153155439 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9383#issuecomment-153155474 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-11-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9383#issuecomment-153127602 **[Test build #44823 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44823/consoleFull)** for PR 9383 at commit [`2e341f5`](https://gith

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9383#issuecomment-153125548 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9383#issuecomment-153125505 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9383#issuecomment-153121664 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-11-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9383#issuecomment-153121656 **[Test build #44820 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44820/consoleFull)** for PR 9383 at commit [`53dbdf2`](https://git

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9383#issuecomment-153121661 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-11-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9383#issuecomment-153121113 **[Test build #44820 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44820/consoleFull)** for PR 9383 at commit [`53dbdf2`](https://gith

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9383#issuecomment-153119226 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9383#issuecomment-153119257 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-11-02 Thread davies
Github user davies commented on the pull request: https://github.com/apache/spark/pull/9383#issuecomment-153118810 After some benchmark, realized that using hashcode as prefix in timsort will cause regression in timsort and snappy compression (especially for aggregation after join, th

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-10-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9383#issuecomment-152711765 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-10-31 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9383#issuecomment-152711698 **[Test build #44730 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44730/consoleFull)** for PR 9383 at commit [`6fde4d5`](https://git

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-10-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9383#issuecomment-152711759 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-10-30 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9383#issuecomment-152697605 **[Test build #44730 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44730/consoleFull)** for PR 9383 at commit [`6fde4d5`](https://gith

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-10-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9383#issuecomment-152697511 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-10-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9383#issuecomment-152697504 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-10-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9383#issuecomment-152697243 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-10-30 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9383#issuecomment-152697240 **[Test build #44728 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44728/consoleFull)** for PR 9383 at commit [`28f84e1`](https://git

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-10-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9383#issuecomment-152697242 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-10-30 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9383#issuecomment-152697137 **[Test build #44728 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44728/consoleFull)** for PR 9383 at commit [`28f84e1`](https://gith

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-10-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9383#issuecomment-152696868 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-10-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9383#issuecomment-152696861 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-10-30 Thread davies
Github user davies commented on the pull request: https://github.com/apache/spark/pull/9383#issuecomment-152694417 Currently, the old one is broken, I'd to remove that one. The new should be as fast as the old one in worst case, I think we don't need a configuration for this. --- If

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-10-30 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/9383#discussion_r43567962 --- Diff: core/src/main/java/org/apache/spark/unsafe/map/BytesToBytesMap.java --- @@ -860,45 +844,48 @@ void growAndRehash() { resizeStartTime = Sys

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-10-30 Thread viirya
Github user viirya commented on the pull request: https://github.com/apache/spark/pull/9383#issuecomment-152691155 Besides, as we discussed in #9067, should we add a configuration for turning on/off this feature? This feature may not always have performance gain. --- If your project

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-10-30 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/9383#discussion_r43567875 --- Diff: core/src/main/java/org/apache/spark/unsafe/map/BytesToBytesMap.java --- @@ -860,45 +844,48 @@ void growAndRehash() { resizeStartTime = Sys

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-10-30 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/9383#discussion_r43567867 --- Diff: core/src/main/java/org/apache/spark/unsafe/map/BytesToBytesMap.java --- @@ -860,45 +844,48 @@ void growAndRehash() { resizeStartTime = Sys

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-10-30 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/9383#discussion_r43567640 --- Diff: core/src/main/java/org/apache/spark/unsafe/map/BytesToBytesMap.java --- @@ -860,45 +844,48 @@ void growAndRehash() { resizeStartTime = Sys

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-10-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9383#issuecomment-152683164 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-10-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9383#issuecomment-152683162 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-10-30 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9383#issuecomment-152683137 **[Test build #44714 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44714/consoleFull)** for PR 9383 at commit [`0512f1e`](https://git

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-10-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9383#issuecomment-152671953 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-10-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9383#issuecomment-152671951 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-10-30 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9383#issuecomment-152670962 **[Test build #44714 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44714/consoleFull)** for PR 9383 at commit [`0512f1e`](https://gith

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-10-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9383#issuecomment-152670753 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-10-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9383#issuecomment-152670739 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-10-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9383#issuecomment-152670207 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-10-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9383#issuecomment-152670180 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-10-30 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9383#issuecomment-152669486 **[Test build #44710 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44710/consoleFull)** for PR 9383 at commit [`752c8e7`](https://git

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-10-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9383#issuecomment-152669491 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-10-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9383#issuecomment-152669489 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-10-30 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9383#issuecomment-152669192 **[Test build #44710 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44710/consoleFull)** for PR 9383 at commit [`752c8e7`](https://gith

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-10-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9383#issuecomment-152668952 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-10-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9383#issuecomment-152668968 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-10-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9383#issuecomment-152663282 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-10-30 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9383#issuecomment-152663276 **[Test build #44708 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44708/consoleFull)** for PR 9383 at commit [`55c47ed`](https://git

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-10-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9383#issuecomment-152663280 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-10-30 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9383#issuecomment-152662959 **[Test build #44708 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44708/consoleFull)** for PR 9383 at commit [`55c47ed`](https://gith

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-10-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9383#issuecomment-152661708 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-10-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9383#issuecomment-152661686 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

  1   2   >