using the ValueState and RocksDB to store a map inside the value state means
that you will have a different map for each key, which is automatically swapped
on a per record basis, depending on the record’s key. If you are using a map
and Checkpointed, there is only one map and your code is responsible for
dispatching state between different keys.
If you use a map and Checkpointed, the map will be on the heap and the
checkpoint will go directly against the filesystem; this is independent of the
chosen backend, so no RocksDB is involved.
On a further note, we are working on an alternative to ValueState that is like
a MapState. In contrast to ValueState, MapState does not deserialize the whole
map on each access, but can access individual key/value pairs. This might be
what you are looking for.
> Am 01.12.2016 um 09:35 schrieb Anirudh Mallem <anirudh.mal...@247-inc.com>:
> Hi Everyone,
> I am trying to understand the Working With State feature page of the Flink
> My question is in case I am using a ValueState in my CustomMap class to
> store my states with the RocksDb as my state backend then it is clear that
> every state value is stored in RocksDb.
> Now instead of a ValueState if I just use a normal Java Hashmap to store my
> states and implement the Checkpointed interface then will the entire HashMap
> reside on the RocksDb backend or will the HashMap be in memory and just the
> snapshots sent to RocksDb? I am trying to see what will I lose/gain if I have
> my own data structure to do state maintenance. Thanks.