josefk31 opened a new pull request, #19468: URL: https://github.com/apache/kafka/pull/19468
Basic architecture for system to manage UncleanRecovery by fetching log information from brokers. The idea is to improve durability by electing the leader with longest during unclean recovery instead of just picking a random unfenced broker. There will be one additional thread - RecoverySendThread which handles NIO between controller and broker - sending requests. LogInfoResponseReceived events will be written to the controllers queue. These events don't write anything until the last response is received - meaning that all needed information is collected to make accurate determinations of log length. Responses to requests are processed in the controllers event queue by a RecoveryManager class. It keeps track of the # of outstanding requests and decides whether and how to retry failed requests. RecoveryManager builds up a "LogInfoStore" object with information about log length of various replicas. Either after all expected requests are received or a timeout; the RecoveryManager will begin to run leadership elections for batches of TopicPartitions (respecting the max batch size configuration). These elections will have access to the store which contains enriched log information to assist them in making leadership decisions. Most of the tracking is performed in a class called "StateMachine" within RecoveryManager. This is deliberate so that eventually the RecoveryManager can supervise multiple unclean election requests at the same time. - Created GetReplicaLogInfoRequest/GetReplicaLogInfoResponse - Created RecoveryManager class to handle sending and receiving requests - Added configs to switch on unclean recovery - Added ability to elect with longest logs in PartitionChangeBuilder - Added RecoverySenderThread for sending -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
