Jacques Nadeau created DRILL-3910:
-------------------------------------
Summary: Leverage Calcite's Clustered Collation
Key: DRILL-3910
URL: https://issues.apache.org/jira/browse/DRILL-3910
Project: Apache Drill
Issue Type: Improvement
Components: Query Planning & Optimization
Reporter: Jacques Nadeau
Right now streaming aggregate requires full collation. I was just talking to
[~julianhyde] and he pointed out that Calcite has a version of Collation that
is Clustered (similar to what MSSQL calls Segment). Realistically, Streaming
aggregate only requires a clustered collation and we should switch to requiring
this. We should also go through existing operators and make sure we manage
whether or not the operators maintain a clustered collation. We should then be
able to have flatten produce a clustered output against the carry-through
fields. This will allow us to do a better job taking advantage of the
clustered-ness of data for doing additional operations. Flatten should also
produce data which exposes the distribution trait on the carry-through fields.
This means that a query like this:
select a, count(b) from (
select a, flatten(x) as b from t
)x
group by a
Should be executed without redistribution of data.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
