JingsongLi commented on PR #10146: URL: https://github.com/apache/paimon/pull/10146#issuecomment-5805617178
Requirement fit: SUPPORTED. The linked issue describes a real Flink session-cluster workload where evaluating roughly one billion global-index matches on the shared JobManager can exhaust memory. Deferring supported index evaluation to data readers addresses that boundary while retaining the existing path for unsupported cases. Implementation: CLEAN in the changed paths after reviewing planning and predicate fallback, split-local row-ID clipping, BTree/Bitmap handling, unindexed rows, deletion vectors, serialization/recovery, and downstream split consumers. I found no evidence-backed new regression. Verification: an isolated Flink 1 reactor package build passed; the focused BTree/index metadata tests passed (139 tests). Other local core/Flink tests were blocked by Mockito inline Byte Buddy agent attachment on this macOS host, including on JDK 17. CI passed core and Flink common jobs; the overall workflow is red because an unrelated MySqlSyncDatabaseActionITCase.testSchemaEvolution timed out at 60 seconds in Flink CDC. Please get that CI gate green before merging. I did not independently reproduce the 100B-row benchmark. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
