AnonHxy commented on issue #16680:
URL: https://github.com/apache/pulsar/issues/16680#issuecomment-1193317816
Hi Asaf @asafm
> Is this true now or do you plan to enforce it?
This is what I plan to enforce it. The batch entry metadata properties is
empty now
> What does this contain? Property names to extract from the messages to be
placed in the batch entry metadata properties?
It contains the properties keys that will be put into the batch entry
metadata properties.
For example:
- Let's set `batchedFilterProperties`=`<region, version>`
This means only key named `region` and `version`will be extracted to the
batch meta properties
- Then we have a producer that sends the messges below in order:
- `msg1` with properties: `<region: eu>`
- `msg2` with properties: `<region: eu>`
- `msg3` with properties: `<region: eu, version:1, tag:a>`
- `msg4` with properties: `<region: eu, version:1>`
- `msg5` with properties: `<region: us, version:1>`
- `msg6` with properties: `<region: us, version:2>`
- The process of properties extraction will be:
- msg1 and msg2 have the same properties: <region: eu>, so they will put
into the same batch
- msg3 and msg4 have the same properties: <region: eu, version:1>. tag:a
in msg3 will be ignored because the `batchedFilterProperties` doesn't contains
'tag'. So msg3 and msg4 will put into the same batch.
- msg5 and msg6 have different properties, because the value of version
is different. So we publish msg5 and msg6 with different batch.
- Just to summarize, the result will be:
| |batch meta properties | single meta properties
| payload | single meta properties | payload |
|------- |---------------------- |------------------------------- |------
|-------------------------|------ |
|batch1 | <region: eu> | <region: eu>
| msg1 | <region: eu> | msg2 |
|batch2 | <region: eu, version:1> | <region: eu, version:1, tag:a> | msg3
| <region: eu, version:1> | msg4 |
|batch3 | <region: us, version:1> | <region: us, version:1> |
msg5 | | |
|batch4 | <region: us, version:2> | <region: us, version:2> |
msg6 | | |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
