GitHub user yhuai opened a pull request:
https://github.com/apache/spark/pull/7954
[SPARK-9630] [SQL] [WIP] Clean up new aggregate operators (SPARK-9240
follow up)
This is the followup of https://github.com/apache/spark/pull/7813. It
renames `HybridUnsafeAggregationIterator` to `TungstenAggregationIterator` and
makes it only work with `UnsafeRow`. Also, I add a `TungstenAggregate` that
uses `TungstenAggregationIterator` and make `SortBasedAggregate` (renamed from
`SortBasedAggregate`) only works with `SafeRow`.
Right now, it contains the fix of
https://github.com/apache/spark/pull/7948. Once #7948 is in, I will rebase this
one.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/yhuai/spark agg-followUp
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/7954.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #7954
----
commit fab284cb125bada54763dadba1dd2f6aac937ecf
Author: Yin Huai <[email protected]>
Date: 2015-08-04T22:39:15Z
1. UnsafeKVExternalSorter does not use 0 as the initialSize to create an
UnsafeInMemorySorter if its BytesToBytesMap is empty.
2. Do not spill a InMemorySorter if it is empty.
3. Do not add spill to SpillMerger if this spill is empty.
commit 2cba09f78e0ffe34fe53cd64adcb2f13d271a11f
Author: Yin Huai <[email protected]>
Date: 2015-08-05T06:53:37Z
Rename UnsafeHybridAggregationIterator to TungstenAggregateIteraotr and
make it only work with UnsafeRow.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
