GitHub user pnowojski opened a pull request:
https://github.com/apache/flink/pull/4723
[FLINK-7684] Avoid data copies in MergingWindowSet
## What is the purpose of the change
Previously MergingWindowSet uses ListState of tuples to persists it's
mapping. This is inefficient because this ListState of tuples must be converted
to a HashMap on each access.
Furthermore, for some cases it might be inefficient to check whether
mapping has changed before saving it on state.
Fixing those two issues improve session windows
[benchmarks](https://github.com/dataArtisans/flink-benchmarks) results by 10 -
20%
First commit comes from different PR #4722
## Verifying this change
This change is already covered by existing tests, such as *(please describe
tests)*.
## Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): no
- The public API, i.e., is any changed class annotated with
`@Public(Evolving)`: no
- The serializers: **yes** (it changes how WindowOperator is being
serialized)
- The runtime per-record code paths (performance sensitive): **yes**
- Anything that affects deployment or recovery: JobManager (and its
components), Checkpointing, Yarn/Mesos, ZooKeeper: no
## Documentation
- Does this pull request introduce a new feature? yes
- If yes, how is the feature documented? JavaDocs
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/pnowojski/flink window
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/4723.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #4723
----
commit ac41b26cdb5c2341bb25e520c4d449ec4e956a8f
Author: Piotr Nowojski <piotr.nowoj...@gmail.com>
Date: 2017-09-14T10:39:30Z
[FLINK-7683] Iterate over keys in KeyedStateBackend
commit 20ed7f51399752d620696d9502b61233d47f6bcb
Author: Piotr Nowojski <piotr.nowoj...@gmail.com>
Date: 2017-09-25T12:18:52Z
[FLINK-7684] Add OptimizationTarget to the ExecutionConfig
commit c5505e1252ea5b70cff9cf76ff89d7dc52f45057
Author: Piotr Nowojski <piotr.nowoj...@gmail.com>
Date: 2017-07-06T11:56:13Z
[FLINK-7684] Serialize MergingWindowSet to ValueState<Map<W,W>>
This avoids an unnecessary data copy
commit 00c044c30e95087b65abe65da67a77841a3c7740
Author: Piotr Nowojski <piotr.nowoj...@gmail.com>
Date: 2017-09-25T13:28:01Z
[FLINK-7684] Add always persist flag to MegringWindowSet
----
---
