prashantwason commented on pull request #3427:
URL: https://github.com/apache/hudi/pull/3427#issuecomment-897222618
To summarize:
1. Async [clean, compact, cluster] - these are in the same process but run
parallel in thread
- We can take a lock in syncMetadata()
- We can sync only as part of commit
- We can depend on transaction support
2. Parallel [clean, compact, cluster] - these are in different processes
- We can use the proposed setting to disable sync in all but one pipeline
- We can depend on transaction support
Only transaction support fixed both the cases but this may not come by this
weekend ([I have a patch for you to
review](https://github.com/apache/hudi/commit/f417390aed56c05dbaedeac6ecf47294996bb591)
though). In the short run, we need to:
1. Enable the setting in this PR
2. Handle the async operations as @vinothchandar is proposing.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
