JingsongLi commented on PR #10146: URL: https://github.com/apache/paimon/pull/10146#issuecomment-5809225818
Thanks, this clarifies the Flink and Spark production scenarios. The shared problem is the coordinator/driver materializing a very large index result, and it is reasonable to keep this PR scoped to Flink while handling Spark separately. I rechecked the discussion and current head (`3be47c5`): there has been no code revision since my review, so the deferred-index-file lifetime issue (P1), bounded-range planning issue (P2), and checkpoint rollback requirement in my earlier review comment remain open. Please address those before production use, then I can re-review the changed code and tests. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
