@merlimat : > I would prefer to handle most of logic for the DLQ in > the client library rather than the broker, just leaving the > minimum support required on server side.
I think this PR has already made things pretty simple. it keeps redelivery count in memory and doesn't persistent. the major difference I can see between the approach here and your approach is "who" publishes messages to DLQ: the approach here in favor of broker doing publishes, your approach prefer client doing publishes to DLQ. I am not very convinced on doing DLQ logic in client, for multiple reasons: a) doing DLQ in client actually make things a bit more complicated. because the redelivery count is tracked at broker, but the publishing happens at the client. Things like configurations/policies are separated in different places, which can result in inconsistent behaviors, which make debugging much harder when issue occurs. b) doing DLQ in client means the same logic has to be implemented in different language client. The implementation can quickly become inconsistent between different language client. c) regarding the resource concerns, I don't think the resource usage is different if it is still broker tracking the redelivery count. unless there is something I am missing. d) doing DLQ in broker allows us doing optimization in future, like persisting delivery count to cursors or a retry queue, without changing client behaviors. especially if we want to support consistent behavior across different language clients. > regarding "negative acks" "negative acks" is a good thing to have. However I don't think DLQ has to depend on "negative acks". because the current "negative acks" pulsar has is ack timeout. for DLQ, it doesn't really matter it is a ack timeout or explicit ack timeout. We don't really need to tie things together. If DLQ goes first, we still can introduce "negative acks" later, and it will not affect DLQ logic. >From this spective, it is making more sense to do DLQ in broker side. because >when DLQ happens at broker side, when we introduce explicitly "negative acks" >later, there will be very minimal code changes at both client and broker sides. > regarding message retention in DLQ topic I think the simplest thing is just let user configure message retention in DLQ topic. That is what typically cloud providers offer. Not sure we need to get into the business of managing subscriptions for DLQ. [ Full content available at: https://github.com/apache/incubator-pulsar/pull/2400 ] This message was relayed via gitbox.apache.org for [email protected]
