[
https://issues.apache.org/jira/browse/FLINK-1176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14176406#comment-14176406
]
Gyula Fora commented on FLINK-1176:
-----------------------------------
You are right, the local reducer here does not necessarily make a combiner
because we dont always want to globally aggregate (however when we choose to
global aggregate the local reducer works as a combiner).
This issue only considers key-less reducers yes. For grouped reducers another
issue would be to introduce pre-aggregators as we have discussed.
> Add operator for global reduce/aggregations
> -------------------------------------------
>
> Key: FLINK-1176
> URL: https://issues.apache.org/jira/browse/FLINK-1176
> Project: Flink
> Issue Type: Improvement
> Components: Streaming
> Reporter: Gyula Fora
> Priority: Minor
>
> Currently streaming reduce operators (like .reduce or .window(..).reduce,
> sum, etc.) only allow local reduces/aggregations. While one can achieve
> global aggregation by setting the parallelism of these operators to 1 it can
> cause a serious bottleneck in the program.
> For window/batch reduces and aggregations a global reducer can be added, with
> parallelism 1, to further reduce partial results. This global reduce option
> could be introduced as an optional boolean parameter for the intended
> operators.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
