congbobo184 opened a new pull request #8719: URL: https://github.com/apache/pulsar/pull/8719
## Motivation now we transaction abort or commit transaction depend on client send all messageId, but TC time out can abort this TXN, because it won't carry the messageId, so we should defend the <txn, List<MessagId>> in transaction buffer to handle the tc send abort. if we implement it, client will not need to send commit with messageId. ## implement 1. When we send the first txn message, the topic will load the transaction buffer and then the buffer will replay this message 2. we will open a no durable cursor. 3. the no durable cursor start position from managedLedger properties, when the txn message send every 10000(can config) it will modify the position in managedLedger. 3. if the managedLedger properties get this is null, we will start in Position.earliest. 4. we defend a struct ``` ConcurrentOpenHashMap<TxnID, ConcurrentOpenHashSet<PositionImpl>> txnBufferCache ``` This is for positions which txn send. when we append the txn message to the topic log complete, we will add the position into it. commit and abort will remove it 5. we defend a struct ``` private final ConcurrentSkipListSet<PositionImpl> positionsSort = new ConcurrentSkipListSet<>(); ``` This is for replay index when every 10000 txn message send finish, we can pollFirst, it is the minimum position on going transaction. when we append the txn message to the topic log complete, we will add the position into it. commit and abort will remove it 6. we add the state for transaction buffer, when the buffer haven't replay finish, client cant send txn message to broker. ### Verifying this change Add the tests for it Does this pull request potentially affect one of the following parts: If yes was chosen, please highlight the changes Dependencies (does it add or upgrade a dependency): (no) The public API: (no) The schema: (no) The default values of configurations: (no) The wire protocol: (no) The rest endpoints: (no) The admin cli options: (no) Anything that affects deployment: (no) ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
