JingsongLi commented on code in PR #8108:
URL: https://github.com/apache/paimon/pull/8108#discussion_r3348877346
##########
paimon-common/src/main/java/org/apache/paimon/globalindex/GlobalIndexResultSerializer.java:
##########
@@ -116,4 +117,20 @@ public GlobalIndexResult deserialize(DataInputView
dataInput) throws IOException
return ScoredGlobalIndexResult.create(roaringNavigableMap64,
scoreMap::get);
}
+
+ public byte[] serialize(GlobalIndexResult globalIndexResult) throws
IOException {
+ DataOutputSerializer dataOutputSerializer = new
DataOutputSerializer(1024);
+ serialize(globalIndexResult, dataOutputSerializer);
+ return dataOutputSerializer.getCopyOfBuffer();
+ }
+
+ public ScoredGlobalIndexResult deserialize(byte[] data) throws IOException
{
Review Comment:
This helper cannot round-trip an empty `ScoredGlobalIndexResult`.
`serialize()` writes only `scoreSize=0` for scored results whose bitmap is
empty, and the existing deserializer interprets `scoreSize == 0` as a plain
`GlobalIndexResult`; this `deserialize(byte[])` method then fails the
`instanceof ScoredGlobalIndexResult` check. In the distributed reader, a split
group can legitimately produce an empty scored result when the scalar
pre-filter excludes all rows in that group, so this can make filtered
distributed searches fail even though the local path handles empty optionals.
We probably need an explicit scored/non-scored marker in the serialization
format, or avoid serializing empty scored results as successful task results.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
