CodingCat commented on PR #3569: URL: https://github.com/apache/celeborn/pull/3569#issuecomment-3698574334
> I dropped a comment on the earlier draft PR, please [see here](https://github.com/apache/celeborn/pull/3564#pullrequestreview-3617376884). @mridulm as i said in earlier comments with @zuston , this feature is not supposed to be rollout to every job, but only for those jobs with large shuffle and no shuffle reuse, and this implementation also provides functionality for jobs to recover from the miss-rollout ... (for AQE case , it is not a big problem, in cases I observed, the parallel downstream stages can always capture the lineage since deletion is an async process) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
