JingsongLi commented on code in PR #8857:
URL: https://github.com/apache/paimon/pull/8857#discussion_r3655750291
##########
paimon-common/src/main/java/org/apache/paimon/lookup/sort/db/LocalKvDb.java:
##########
@@ -549,7 +607,8 @@ private SstFileMetadata
findFileForKey(List<SstFileMetadata> sortedFiles, Memory
private SstFileMetadata writeMemTableToSst(TreeMap<MemorySlice, byte[]>
data)
throws IOException {
File sstFile = newSstFile();
- SortLookupStoreWriter writer = storeFactory.createWriter(sstFile,
null);
+ SortLookupStoreWriter writer =
+ storeFactory.createWriter(sstFile,
bloomFilterBuilderFactory.apply(data.size()));
Review Comment:
`LocalTableQuery` already has the same limitation: it creates the sort
lookup store with `RowCompactedSerializer.createSliceComparator()`, while its
Bloom filter is also built and queried from the raw serialized key bytes
(`lookup.cache.bloom.filter.enabled` is enabled by default). Therefore
comparator-equal FLOAT/DOUBLE NaN payloads can likewise differ at the
Bloom-filter level there. This is a broader existing limitation of
custom-comparator equality versus raw-byte Bloom-filter hashing, rather than
something specific to `LocalKvDb`. We will keep the behavior consistent in this
PR and leave a general solution out of scope for now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
