complone commented on issue #4933: URL: https://github.com/apache/rocketmq/issues/4933#issuecomment-1246367514
Hello everyone, the problems we need to solve at this stage are: In order to improve the accuracy of data and ensure the consistency between the source and the target in the end-to-end transmission process, it is necessary to do idempotent at the producer or consumer However, if consumers cannot connect to the nameserver to obtain partition information temporarily due to network jitter, they need to implement idempotent mechanism in the producer For this reason, our goal is to align the produerId and transactionId of kafka. There are three points to note at this stage by following the code - First, before sending data, you need to manually create a transaction coordinator to control the one to many relationship between PID and transcationId. To this end, you need to ensure that the consumption progress of offset is notified to the tc in a timely manner and that the messages of submitted but unsettled transactions will not be deleted by mistake during the simultaneous process of the scheduled deletion task - Whether it is necessary to judge the progress of the current offset and the global PID visible transaction offset in PullLiteProductor in order to enable PID ->transactionid to achieve cross partition transaction consistency - When downstream consumers consume, they need to ensure that messages marked as transactionId in the upstream market are perceived by downstream consumers -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
