xuzifu666 commented on PR #10898:
URL: https://github.com/apache/hudi/pull/10898#issuecomment-2081435032
> > > we also meet the issue in our tests, the case is that we just use
simple bucket index with mor without partition, and when we restart the job, it
will write success once, and but then the bucket id conflict.
00000074-3413-4e9e-b4dd-4676e4eeccb4-0_74-8-3151_20240426173957249.parquet
(bulkinsert generate it)
00000074-50bc-4b34-82c5-08c210d82d33-0_74-26-37004_20240428145515030.parquet
(deltacommit generate it after a restart) according to the driver log, the
restart job read a rollback commit, and seems the timeline not load all the
bucket completely 24/04/28 14:57:32 INFO HoodieBucketIndex: Get
BucketIndexLocationMapper for partitions: [] 24/04/28 14:57:32 INFO
HoodieActiveTimeline: Loaded instants upto :
Option{val=[20240428145515193__rollback__COMPLETED__20240428145515760]}
> >
> >
> > Yes，it is a serious problem which would block user's business. would
rise it to find out error instant firstly which can help user to continue their
business. @danny0405
>
> actually in my use case, no multi writer, just a restart triggered a
rollback, then met this issue.
At lease get the error instant can help you to continue your job.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
