ChenSammi commented on PR #10515: URL: https://github.com/apache/ozone/pull/10515#issuecomment-4874076396
@gardenia , the progress indicator is the txID, key is "notification-checkpoint", value is txID. How about use the OM metadata table to persist this value? metadata is defined in OMDBDefinition.java, currently it's used to persist data like OM layout version, ranger ozone service version, it can perfectly save the "notification-checkpoint" -> txID data, Or we can introduce a new table to save it, just like the statefulServiceConfig table in SCM, which can be used by any feature of SCM. Second, this indicator tries to prevent OM to handle events which have already been handled before OM start or OM leader transfer, but since the event model is at least once, the receiver need to have built-in capability to handle resent event notification. So the checkpoint loading or saving fails, we should still go ahead and send events out. Checkpoint is nice to have, not mandatory, it's failure should lead to the event process task function fail. Third, the event framework supports multiple plugins, each plugin runs independently. The current checkpoint and #10516 are specific to Kafka. If there is another plugin loaded, this plugin may also has it's checkpoint, and it needs to coordinate with Kafka about the purge, in case one plugin moving fast, while another one moving relatively slow. It looks like the current framework doesn't support this if we really want to support multiple plugins running together. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
