StefanRRichter commented on a change in pull request #6438: [FLINK-9981] Tune
performance of RocksDB implementation
URL: https://github.com/apache/flink/pull/6438#discussion_r205985966
##########
File path:
flink-state-backends/flink-statebackend-rocksdb/src/main/java/org/apache/flink/contrib/streaming/state/RocksDBOrderedSetStore.java
##########
@@ -116,7 +127,14 @@ public RocksDBOrderedSetStore(
@Override
public void add(@Nonnull T element) {
+
byte[] elementBytes = serializeElement(element);
+
+ if (LEXICOGRAPIC_BYTE_COMPARATOR.compare(elementBytes,
lowerBoundSeekKey) < 0) {
+ // a smaller element means a new lower bound.
+ lowerBoundSeekKey = elementBytes;
Review comment:
Yes, please notice that the lower bound is also changed from requesting an
iterator. Then it is set to the actual current low key. Afterwards, smaller
keys can be inserted again, and that is why this check is required. Maybe keys
get deleted again, that is why it can only be a lower bound without getting to
expensive, but from my experiments small change gave a huge improvement because
it helps ignoring tombstones pre-compactions, and there are often many ahead of
the first existing element because of the typical access pattern of this state.
We might chose another initial value for the lower bound than the prefix, but I
would not introduce `null` and additional branching. Thinking about it, the
prefix of the next key-group might be actually a sensible initial value, but in
the end it also doesn't matter to much because it is properly "calibrated" when
an iterator is requested for the first time
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
