GitHub user lamuguo opened a pull request:
https://github.com/apache/spark/pull/867
Use Aggregator for Spark SQL
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/lamuguo/spark master
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/867.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #867
----
commit 6507d1596de1fc9e1f73aea76bc6e596ef59e578
Author: Lamu Guo <[email protected]>
Date: 2014-05-02T06:04:50Z
Applying Aggregator in Aggregate execution
First try.
Some test errors, but matches with another baseline result. Will double
check whether all test cases can be passed currently later.
Error info below:
[info] Passed: Total 197, Failed 0, Errors 0, Passed 195, Skipped 2
[error] (repl/test:test) sbt.TestsFailedException: Tests unsuccessful
[error] (streaming/test:test) sbt.TestsFailedException: Tests
unsuccessful
[error] (core/test:test) sbt.TestsFailedException: Tests unsuccessful
[error] Total time: 1755 s, completed May 1, 2014 10:49:27 PM
commit 358874ef5b01acf4d60bbce68756f94223881726
Author: Lamu Guo <[email protected]>
Date: 2014-05-02T16:35:11Z
Support SparkSqlSerializer for Aggregate
And fixed some style problems.
commit f941a50d73304b5716f3e9da8e89436213dba045
Author: Lamu Guo <[email protected]>
Date: 2014-05-02T16:54:53Z
Added spark conf for SparkSqlSerializer
commit e5bc329b1a1be3c563a9024a94e7f641855903ab
Author: Lamu Guo <[email protected]>
Date: 2014-05-23T02:07:27Z
Support AggregateFunction.merge()
To eliminate saving of rows in interim data.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
