HeartSaVioR commented on code in PR #54083:
URL: https://github.com/apache/spark/pull/54083#discussion_r2767440629
##########
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/state/StateStoreRowChecksum.scala:
##########
@@ -276,14 +278,14 @@ object KeyValueChecksumEncoder {
// skip the checksum (first 4 bytes)
currentPosition += java.lang.Integer.BYTES
val valueRowSize = Platform.getInt(valueBytes, currentPosition)
- // move to the next value and skip the delimiter character used for
rocksdb merge
- currentPosition += java.lang.Integer.BYTES + valueRowSize + 1
+ // move to the next value and skip the delimiter bytes used for rocksdb
merge
+ currentPosition += java.lang.Integer.BYTES + valueRowSize + delimiterSize
resultSize += valueRowSize
numValues += 1
}
// include the number of delimiters used for merge
- resultSize += numValues - 1
+ resultSize += (numValues - 1) * delimiterSize
Review Comment:
It's just that we do not add the size for delimiters. resultSize being
calculated in above logic retains the same.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
