I assume you are using the DataSet API.
There, you can do a combinable group reduce:

The combine() method will be executed on the sender side, reducing the
amount of data to spill on disk. This only works if your data allows such
early aggregations.
This is similar to a combiner in Hadoop:

On Thu, Feb 13, 2020 at 8:01 PM Richard Moorhead <richard.moorh...@gmail.com>

> In batch mode, if input is sorted prior to a group by operation; does
> flink forward the aggregate data early? Is there a way to prevent grouping
> operations from buffering all data in a GBK operation in batch mode?

Reply via email to