gardenia opened a new pull request, #10689:
URL: https://github.com/apache/ozone/pull/10689
Please describe your PR in detail:
Persist the checkpoint within the OM RocksDB metadata table using a
distributed Ratis update message so that another OM can pick up if the leader
changes.
Technical Solution:
* Update the RocksDB metadata table
(OMSetEventNotificationCheckpointRequest) and distribute this to other OMs via
Ratis (SetEventNotificationCheckpointRequest)
Checkpoint Throttling:
* We update our local progress in-memory on every transaction batch, but
throttle the persistent Ratis writes (configurable via
OZONE_OM_PLUGIN_CHECKPOINT_SAVE_INTERVAL, defaulting to every 100 transactions).
* The intent is to prevent saturating the Ratis log or causing write storms
on RocksDB while maintain our at-least-once delivery guarantee.
* If a Ratis write fails, throttling is bypassed to ensure the next update
cycle retries immediately.
Error Handling:
* Both loading and saving failures are non-blocking for background polling
progress
* On load failures, we log a warning and fallback to starting from
transaction 0/null.
* On save failures, we log a warning but always advance progress in-memory
to ensure progress but conform to at-least-once event delivery.
## What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-15562
## How was this patch tested?
Unit tests
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
