XiaoHongbo-Hope commented on PR #672: URL: https://github.com/apache/paimon-rust/pull/672#issuecomment-5178038960
Addressed the mixed-format P1 in a4098c8. Exactness no longer depends on the current `file.format` writer option. Mosaic now widens its physical projection with predicate columns, applies the shared exact residual evaluator per batch, and projects back to the requested output. Added an end-to-end regression with current `file.format=parquet`, an old `.mosaic` split, an unprojected LIKE column, and content assertions (`id = [1, 3]`). Validation: Mosaic 33/33; Paimon table 994 passed / 1 ignored; DataFusion 329 passed / same 7 missing-fixture tests; clippy passed. Also reran the production-scale Shanghai query on this exact head: EXPLAIN 5.625 s, 1 split / 3 files, 1.41 GiB max RSS; SELECT 6.605 s, 1000 rows, all 1000 LIKE-validated, 1.42 GiB max RSS. On the earlier P2 suggestions: the limited scan remains sequential to avoid speculative shard I/O and multiple decoded bitmaps. Preferred-probe results are cleared before capped full fallback because rediscovered duplicate row IDs could otherwise consume the per-shard remaining cap before enough new unique matches are found; comments now document both constraints. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
