gosonzhang opened a new pull request, #11395: URL: https://github.com/apache/inlong/pull/11395
Fixes #11386  Optimize content: 1. Broker regularly refreshes the consumer group offset information in memory to the file Offset Cache according to the configured offsetStgCacheFlushMs value; 2. The consumer group offset information stored in the file Offset Cache is regularly written to the offsetStgFilePath/offsetDir directory according to the configured offsetStgFileSyncMs value. The file name successfully saved is offsets.meta. The persistence logic is as follows: - Persistently save the consumer group offset information to the file offsets.tmp. If the write fails, output the failure log, alarm and exit the persistence process; - Read the offsets.tmp file and compare it with the previously written information to see if it is consistent. If it is inconsistent, output the failure log and exit the persistence process; - Migrate the file from offsets.tmp to offsets.mid. If the offsets.meta file currently exists, change offsets.meta to offsets.old; - Migrate the file from offsets.mid to offsets.meta, and delete the offsets.old file at the same time. 3. Broker supports saving the current consumer group's Offset collection information to the backupPath path specified in the API interface through the HTTP API admin_backup_group_offsets. The saved file name is offsets.meta. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
