GitHub user je-ik opened a pull request:
https://github.com/apache/flink/pull/5185
[FLINK-8297] [flink-rocksdb] optionally use RocksDBMapState internally for
storing lists
## What is the purpose of the change
Enable storing lists not fitting to memory per single key.
## Brief change log
## Verifying this change
This change added tests and can be verified as follows:
passes additional tests for RocksDBStateBackend.enableLargeListsPerKey()
## Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): no
- The public API, i.e., is any changed class annotated with
`@Public(Evolving)`: yes
- The serializers: no
- The runtime per-record code paths (performance sensitive): no, backward
compatible
- Anything that affects deployment or recovery: JobManager (and its
components), Checkpointing, Yarn/Mesos, ZooKeeper: no
- The S3 file system connector: no
## Documentation
- Does this pull request introduce a new feature? yes
- If yes, how is the feature documented? JavaDocs
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/datadrivencz/flink
rocksdb-backend-memory-optimization
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/5185.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #5185
----
commit f1bbaa30901ba8a54b02908fd3eb3615301b4400
Author: Jan Lukavsky <[email protected]>
Date: 2017-12-14T20:42:06Z
[FLINK-8297] [flink-rocksdb] optionally use RocksDBMapState internally for
storing lists
----
---
