Github user StefanRRichter commented on a diff in the pull request:
https://github.com/apache/flink/pull/4963#discussion_r159388932
--- Diff:
flink-contrib/flink-statebackend-rocksdb/src/main/java/org/apache/flink/contrib/streaming/state/RocksDBListState.java
---
@@ -158,4 +158,28 @@ public void mergeNamespaces(N target, Collection<N>
sources) throws Exception {
throw new Exception("Error while merging state in
RocksDB", e);
}
}
+
+ @Override
+ public void update(List<V> values) throws Exception {
+ clear();
+
+ if (values != null && !values.isEmpty()) {
+ try {
+ writeCurrentKeyWithGroupAndNamespace();
+ byte[] key =
keySerializationStream.toByteArray();
+ DataOutputViewStreamWrapper out = new
DataOutputViewStreamWrapper(keySerializationStream);
+
+ List<byte[]> bytes = new
ArrayList<>(values.size());
+ for (V value : values) {
+ keySerializationStream.reset();
+ valueSerializer.serialize(value, out);
+
bytes.add(keySerializationStream.toByteArray());
+ }
+
+ backend.db.put(columnFamily, writeOptions, key,
MergeUtils.merge(bytes));
--- End diff --
Do we have some numbers that this custom merging is better than multiple
calls to RocksDB merging state? I can see that it is potentially a lot better,
but just making sure it is worth the additional complexity.
---
