Thanks Penghui & Sijie for the comments, updated the wiki link.

1. we should add the new ordering_key, it is as Penghui mentioned, There
may features that all enabled, but want to use key for different purposes.
2. regarding consumer listener in "failover", it is originally for
failover/exclusive mode, which only let one consumer alive.  In
Key_Failover, it is more like shared mode, all alive consumer should be
able to consume messages.

Best Regards.


Jia Zhai

Beijing, China

Mobile: +86 15810491983




On Fri, Apr 5, 2019 at 1:04 AM Sijie Guo <guosi...@gmail.com> wrote:

> Penguin,
>
> Thank you for your clarification! That makes sense to me then.
>
> Sijie
>
> On Thu, Apr 4, 2019 at 2:21 AM PengHui Li <codelipeng...@gmail.com> wrote:
>
> > @Sijie Guo <si...@apache.org>
> >
> > Sorry, I may not have described my point in the previous email.
> > I try to describe it again. :)
> >
> > User can use key to achieve business logic(e.g. use user_id as message
> key)
> > At the same time, want to use another identifier as the basis for the
> > order.
> >
> > As the message key has many uses, avoid duplicate messages, compaction
> > topics.
> > I prefer provide a mechanism to overwrite key based sorting behavior.
> > Of course this is not mandatory.
> >
> > If the basis for sorting is exactly the same as the real purpose of the
> > message key,
> > just use the message key. Otherwise user provide the order key to
> > overwrite.
> >
> > Best Regards.
> > Penghui
> >
> > Sijie Guo <guosi...@gmail.com> 于2019年4月4日周四 下午3:36写道:
> >
> >> Jia thank you for raising this up.
> >>
> >> A couple of comments here
> >>
> >> 1. technically you don't need a consistent hashing mechanism for
> dividing
> >> hash ranges for consumers. Any hash mechanism that can map a key to a
> >> consumer should work here. The only advantage of consistent hashing
> >> mechanism is to reduce the disruptions when a consumer joins and leaves.
> >> So
> >> I would prefer making the hashing mechanism pluggable.
> >>
> >> 2. regarding using key or introducing a new ordering key. I would prefer
> >> using `key` here. having two `keys` actually makes people confused. The
> >> `key` was used for compaction but the compaction doesn't mutate the
> >> original data. it just creates a new compacted ledger. in most of the
> >> time,
> >> when people use `key_failover`, they are probably accessing the original
> >> data. Event they are trying to subscribe to compacted topic using
> >> `key_failover`, the ordering of messages by key will still be true. so I
> >> wouldn't worry about this part that much.
> >>
> >> 3. in `failover` there is a consumer listener that listens on the
> >> partition
> >> assignment changes. Is there any requirements or plans on adding similar
> >> listeners for `key_failover`?
> >>
> >> Thanks,
> >> Sijie
> >>
> >> On Wed, Apr 3, 2019 at 2:09 AM Jia Zhai <zhai...@apache.org> wrote:
> >>
> >> > Hi all,
> >> >
> >> > I would like to init a discuss of "PIP 34: Add new subscribe type --
> >> > Key_Failover" here.
> >> >
> >> > At present, there are 3 kinds of subscription modes: exclusive,
> shared,
> >> and
> >> > failover. Shared mode subscription is used a lot, because consumers
> >> could
> >> > effectively parallel consume messages of same partition.  But using
> >> shared
> >> > mode will not keep the message order of same key, It would be good to
> >> > leverage share mode, while also keeping the ordering of messages.
> >> >
> >> > This Proposal trying to introduce a new subscribe type Key_Failover to
> >> > extend shared type. By Key_Failover, one partition could have several
> >> > consumers to parallel consume messages, while all messages with the
> same
> >> > key will be dispatched to only one consumer.
> >> >
> >> > Here is the link contains more information of this PIP:
> >> >
> >> >
> >>
> https://github.com/apache/pulsar/wiki/PIP-34%3A-Add-new-subscribe-type-Key_Failover
> >> >
> >> > Thanks.
> >> >
> >>
> >
>

Reply via email to