XiaoHongbo-Hope commented on PR #8826: URL: https://github.com/apache/paimon/pull/8826#issuecomment-5126281580
> > Incremental commits are not atomic across the whole operation. Ordinary exceptions > > raised while applying a group are reported as results so other groups can finish; > > completed groups are flushed before the error is raised. > > Ray task retry options in `ray_remote_args` do not retry group exceptions reported > > as results, so a transient group failure can leave a partial update. Worker loss and > > other task-level failures which bypass Python exception handling may still lose > > results buffered by that task. > > If this is indeed the case, as stated in your documentation, then the implications are significant—far beyond what a simple description like “max_groups_per_commit” can capture. This is a major semantic change that has been overlooked. Updated, can you help check whether this looks good. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
