JingsongLi opened a new issue, #1243: URL: https://github.com/apache/paimon-rust/issues/1243
Global full-text search currently requests each archive's full row-range candidate count and merges all hits before final Top-K. Common-term queries can therefore retain candidates proportional to matched documents even when the caller requests only a few rows. Add a conservative per-shard Top-K path when global post-processing cannot delete, replace, or deduplicate hits across shards. Request K+1 native hits and require a strict score boundary plus distinct selected row IDs before truncating. Preserve the existing exhaustive path for boundary ties, raw sources, deletion exclusions, overlapping shard ranges, and unsupported search modes. Keep local and scheduled execution equivalent. Add a reproducible production-path benchmark comparing the optimized and exhaustive modes on the same data/index snapshot. Check exact row IDs and scores on every iteration, alternate execution order, and include both distinct-score and tied-score corpora. This benchmark is a synthetic local warm-cache comparison, not a Lance or object-store performance claim. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
