gardenia opened a new pull request, #9336: URL: https://github.com/apache/ozone/pull/9336
## Please describe your PR in detail: A rough draft implementation of this outline in the design doc: https://github.com/apache/ozone/pull/8871#issuecomment-3177110525 NOTE: This PR is not intended to be ready for merging but is being shared as a way to get early feedback on the gist of the approach and help drive the discussion of the design (https://github.com/apache/ozone/pull/8871) There are many TODOs, rough edges and things that need to be fleshed out. There are 2 logical parts: 1. in OzoneManagerStateMachine when certain write requests complete successfully a summary of that request is written to a new rocksdb "ledger" table named CompletedRequestInfo The current CompletedRequestInfo schema is minimal: ``` /** * CompletedRequestInfo table entry */ message CompletedRequestInfo { optional int64 trxLogIndex = 1; required Type cmdType = 2; // Type of the command optional string volumeName = 3; optional string bucketName = 4; optional string keyName = 5; optional uint64 creationTime = 6; optional CreateKeyOperationArgs createKeyArgs = 7; optional RenameKeyOperationArgs renameKeyArgs = 8; optional DeleteKeyOperationArgs deleteKeyArgs = 9; optional CommitKeyOperationArgs commitKeyArgs = 10; optional CreateDirectoryOperationArgs createDirectoryArgs = 11; optional CreateFileOperationArgs createFileArgs = 12; } ``` The use of "optional" for the arguments was based on feedback from the community call where @errose28 made the point that the previous sketch of a schema (using a freeform Map<String, String>) did not jibe well with schema management and this optional pattern had been used in other places to get around that. 2. there is then a concept of an "event listener" plugin which can consume the records in the CompletedRequestInfo table. * plugins implement a new interface: OMEventListener * a helper class OMEventListenerLedgerPoller is provided which plugin implementations can use to periodically poll for newly written CompletedRequestInfo records table and passes them them to a callback. * OMEventListenerKafkaPublisher is a concrete implementation of OMEventListener which consumes the latest CompletedRequestInfo records, serializes them to appropriate S3 style event notification and sends them to a configured kafka broker. * there is a draft implementation of a persistence strategy for the seek position (i.e. the latest CompletedRequestInfo consumed and processed by the plugin) called LocalFileCheckpointStrategy. This is an ultimately flawed approach in the case of leader changes but still meets the minimal criteria of "at least once" until we implement something better. * additionally we also have an implementation of the checkpoint strategy which stores the file on the ozone filesystem OzoneFileCheckpointStrategy (rather than a local file) * there is a crude strawman/draft implementation of a strategy to rotate out old records for the ledger based on number of rows. This needs fleshed out and made robust/efficient. * plugins can be loaded/configured dynamically similarly to ranger plugins, e.g.: ``` ozone.om.plugin.destination.kafka=true ozone.om.plugin.destination.kafka.classname=org.apache.hadoop.ozone.om.eventlistener.OMEventListenerKafkaPublisher ozone.notify.kafka.topic=test123 ozone.notify.kafka.bootstrap.servers=kafka-3:29092,kafka-1:29092,kafka-2:29092 ``` TODO: - [X] move the plugin implementations out of ozone-manager into some separate mvn package - [X] implement a better persistence strategy for the seek position (e.g. write to a file on the ozone filesystem instead of a file local to the OM) - [X] scrub of terminology / renaming. (original terminology OperationInfo now renamed to CompletedRequestInfo / OmCompletedRequestInfo) - [X] flesh out the event mappings - the existing mappings of operations -> events are not an authoritative take - [ ] flesh out/redo the strawman implementation of the strategy to rotate out old records from the ledger table (CompletedRequestInfoCleanupService) - [ ] move the pluginManager creation/lifecycle out of KeyManagerImpl to some better place - [ ] need to work out how to fit ACLs into the ledger schema - [ ] metrics - [ ] more unit tests ## What is the link to the Apache JIRA http://issues.apache.org/jira/browse/HDDS-5984 ## How was this patch tested? unit tests, manual tests (docker compose) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
