XiaoYou201 opened a new pull request, #10440: URL: https://github.com/apache/inlong/pull/10440
### [INLONG-10311][Sort] Implement TubeMQ Source report audit information exactly once Fixes #10311 ### Motivation Now，every connectors report audit information only support at least once. So，This pr base it to implements report audit information exactly once. The audit information will not be wrong in much exception situation. ### Modifications Using the checkpoint principle in Flink to modify the process, the modified flow chart is shown in Figure 1 and Figure 2. In Figure 1, the callback method of notifyCompleteCheckpoint is used to upload audit information instead of scheduled upload. Each Source/Sink will save the checkpointId of the currently ongoing checkpoint, which is nowCheckpointId in the figure. When the audit information is written to the Buffer, nowCheckpointId will be attached, indicating that the audit information is written during this ongoing checkpoint. The audit information and checkpointId are in a many-to-one relationship.  When a snapshot request is received, the current operator's nowCheckpointId is updated to (snapshot) checkpointId + 1. When all operators in a task complete the snapshot, the notifyCompleteCheckpoint method is called back. At this time, AuditBuffer uploads audit information less than or equal to checkpointId (parameters in the notifyCompleteCheckpoint method).  -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
