cshuo commented on issue #17700:
URL: https://github.com/apache/hudi/issues/17700#issuecomment-3744064812
Instead of introducing fs based instant mapping, bucket assign operator can
request the mapping from the coordinator, which contains the source of truth
about the inflight instants mapping.
basic idea:
* Use `TaskOperatorEventGateway` to support coordination between bucket
assign operator and coordinator.
* Bucket assign operator keeps the list of inflight instants `List<String>
inflightInstants` used to access MDT.
* On task failover/restart: request the inflight instants from coordinator
event buffers, initialize `inflightInstants`
* On `notifyCheckpointComplete`: request inflight instants from coordinator
* update `inflightInstants`
* clean the inflight RLI data cache
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
