BewareMyPower commented on PR #16061: URL: https://github.com/apache/pulsar/pull/16061#issuecomment-1158567970
Close this PR. I'll do it in another PR. The main problem is this PR's solution will add all incomplete chunks into the replay queue (`redeliveryMessages`). However, the `readMoreEntries` method either replays the positions from the replay queue or reads new entries. It's why I call `readMoreEntries` before adding messages to the replay queue. But there is still a chance when multiple chunked messages were sent. For example, there are 2 chunked messages M0 and M1, both have 3 chunks. 1. Read M0-C0, M0-C1, M0-C2, M1-C0. 2. M0 will be dispatched to a proper consumer, M1-C0 will be cached. Then there are two possibilities: - [OK] Read M1-C1 and M1-C2, since they are incomplete chunks, they will be added to the replay queue. Next time `readMoreEntries` will replay these 3 chunks and send them to the consumer. - [Failed] M1-C0 is replayed by `readMoreEntries` before reading M1-C1 and M1-C2, then M1-C0 will be added to the replay queue again. The previous steps will be repeated forever. Another flaw is it cannot handle the limitations of the shared dispatcher, like `dispatcherMaxReadSizeBytes` and `dispatcherMaxReadBatchSize`. If the total chunk message size was greater than the `dispatcherMaxReadSizeBytes`, there would be no chance to gather all chunks in the `readEntriesComplete` method and the incomplete chunks would be always replayed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
