AnonHxy commented on issue #16680:
URL: https://github.com/apache/pulsar/issues/16680#issuecomment-1195274822
> What I am suggesting is to turn this idea a bit: Instead of batching per
the same values of keys, batch all messages, and for each property save a list
of values in the metadata. This way you are not affecting the latency on the
client side or batch size, yet you still have the ability to filter batches
based on metadata without deserializing it.
Thanks for your suggestion @asafm . If I understand correctly, it means that
all the properties, which need to be filted, will be put into the entry
header meta. So the protocol will be like below:
| | batch meta properties | single meta properties | payload | single meta
properties | payload|
| -- | -- | -- | -- | --|---|
| batch | <region: eu,us; version:1> | <region: eu> | msg1 | <region: us;
version:1> | msg2|
But this will not work will for entry filter I think, the reason is that:
For example, if our filter's condition is that :
```
if region==eu
return reject
else
return accept
```
so the expected result is that msg1 will be rejected and msg2 will be
accepted. However because msg1 and msg2 are in the same entry, so the entry
filter will not work well. In other words, entry filter doesn't support
batched message now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
