sijie commented on a change in pull request #6052: [PIP-54] Acknowledgement for
batch message local index
URL: https://github.com/apache/pulsar/pull/6052#discussion_r367978978
##########
File path: managed-ledger/src/main/proto/MLDataFormats.proto
##########
@@ -66,6 +66,9 @@ message PositionInfo {
// Additional custom properties associated with
// the current cursor position
repeated LongProperty properties = 4;
+
+ // Store which index in the batch message has been deleted
+ repeated BatchDeletedIndexInfo batchDeletedIndexes = 5;
Review comment:
Instead of introducing a new structure for storing index information, why
not just store a bitset in NestedPositionInfo?
```
message NestedPositionInfo {
required int64 ledgerId = 1;
required int64 entryId = 2;
// using one integer for storing the acknowledgement set for the indexes
within a entry
optional uint32 ackset = 0;
}
```
The `ackset` in `NestedPositionInfo` is used for indicating whether an
*entry* is fully acknowledged or *partially* acknowledged. Hence you don't need
to introduce more data structure. You can still use `individualDeletedMessages`
for representing a delete range. In a message range, all *entries* between
`lowerEndpoint` and `higherEndpoint` are *fully* acknowledged. The ackSets in
`lowerEndpoint` and `upperEndpoint` are used for indicating whether these two
entries are *fully* or *partially* acked.
Using this format, you don't need to introduce redundant structure and it is
a very efficient storage format. Also, more importantly, it provides a
consistent view for people to realize the acknowledgment behavior without
looking into multiple different places.
In the implementation, it should be straightforward as well. As you just
need to introduce ackSet to current acknowledgment logic and make sure the
range merging logic handle ackSet correctly. And when exposing the
acknowledgment information to topic stats, you can just simply add the `ackSet`
in the individual delete set.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
