BewareMyPower commented on issue #11962:
URL: https://github.com/apache/pulsar/issues/11962#issuecomment-915721441
@codelipenghui
> In my opinion,
>
> 1. if users want to achieve high performance, they should avoid the
broker-side data format conversion.
> 2. If users do not care about the performance, they can use
entry.format=pulsar, no need to add more dependencies at the client-side.
It's right. But this proposal is mainly for **the first case**, not the
second case. What if the user switched to Pulsar client later? Without the
converter, they have to discard old messages. If it's not acceptable, they
would never consider switching to Pulsar client.
> but if you have 100 subscriptions to read the data from the topic by
Pulsar client, you might want to choose the entry.format=pulsar.
It's right because in this case users prefer to store messages as Pulsar
format. However, as I've said, the message converter is mainly for the
`entry.format=kafka` case, i.e. users prefer to store messages as Kafka format.
But in some cases, they also need Pulsar consumer to read these messages.
Both `kafka` and `pulsar` entry format would bring burdens on broker, but we
must ensure that the interaction among Kafka clients and Pulsar clients work
for both cases.
> And If we are try to introduce the data format processor at the
client-side again, users will have 3 options, entry.format for KoP,
messageConverter for the broker, data format processor for the client-side.
This will make it very complicated to use.
1. `entry.format=pulsar` without any care for performance.
2. `entry.format=kafka` for better performance among Kafka clients.
If you still want to consume these messages somehow with Pulsar client,
configure the message converter
1. at client level if you can upgrade your Pulsar client >= 2.9.0
2. at broker level if you have some older Pulsar clients and cannot
upgrade for some reasons.
We should cover all use cases.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
