liming30 commented on code in PR #20405:
URL: https://github.com/apache/flink/pull/20405#discussion_r940454587
##########
flink-state-backends/flink-statebackend-rocksdb/src/main/java/org/apache/flink/contrib/streaming/state/AbstractRocksDBState.java:
##########
@@ -223,4 +223,27 @@ public StateIncrementalVisitor<K, N, V>
getStateIncrementalVisitor(
throw new UnsupportedOperationException(
"Global state entry iterator is unsupported for RocksDb
backend");
}
+
+ /**
+ * Similar to decimal addition, add 1 to the last digit to calculate the
upper bound.
+ *
+ * @param prefix the starting prefix for seek.
+ * @return end prefix for seek.
+ */
+ protected final byte[] calculateUpperBound(byte[] prefix) {
+ byte[] upperBound = new byte[prefix.length];
+ System.arraycopy(prefix, 0, upperBound, 0, prefix.length);
+ boolean overFlow = true;
+ for (int i = prefix.length - 1; i >= 0; i--) {
+ int unsignedValue = prefix[i] & 0xff;
+ int result = unsignedValue + 1;
+ upperBound[i] = (byte) (result & 0xff);
+ if (result >> 8 == 0) {
+ overFlow = false;
+ break;
+ }
+ }
+ Preconditions.checkArgument(!overFlow, "The upper boundary
overflows.");
Review Comment:
I ran this code and it returned `-128` as upperBound and no exception was
thrown.
In my opinion, `RocksDB` just does seek matching by prefix, so here we
should calculate as `unsigned` numbers, not as `signed` numbers.
In fact, we always treat the serialized result value as an `unsigned`
number, not as a `signed` number. In
[clipDBWithKeyGroupRange](https://github.com/apache/flink/blob/master/flink-state-backends/flink-statebackend-rocksdb/src/main/java/org/apache/flink/contrib/streaming/state/RocksDBIncrementalCheckpointUtils.java#L128),
when `currentKeyGroupRange.getEndKeyGroup()` is 127,
The result of
`CompositeKeySerializationUtils.serializeKeyGroup(currentKeyGroupRange.getEndKeyGroup()
+ 1, endKeyGroupBytes)` is also -128.
The overflow in this part of the code means that an overflow occurs in the
calculation of `unsigned` numbers, which should not exist.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
