ad1happy2go commented on issue #10468: URL: https://github.com/apache/hudi/issues/10468#issuecomment-1885334906
You can probably try Non Blocking concurrency control with hudi 1.0.0-beta. On Wed, Jan 10, 2024 at 8:30 PM Sam ***@***.***> wrote: > @SamarthRaval <https://github.com/SamarthRaval> OCC is not recommend to > run lot of multi writer jobs together if they write to same file group. You > may be seeing failures which will be resulting in rollbacks. > > The files will keep on failing and retried in case of conflict , so that's > the reason you feel it may be running forever. > > I think that is what exactly happening. > > In future releases of hudi, are we addressing this by any chance ? > Or something is in pipeline already for above scenario ? > > Asking this so in our future development I can keep this in consideration. > Thanks > > — > Reply to this email directly, view it on GitHub > <https://github.com/apache/hudi/issues/10468#issuecomment-1885013120>, or > unsubscribe > <https://github.com/notifications/unsubscribe-auth/APD55YWGBG2ZJXOIKJHM6RDYN2UJ7AVCNFSM6AAAAABBTWUJ56VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQOBVGAYTGMJSGA> > . > You are receiving this because you were mentioned.Message ID: > ***@***.***> > -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
