I see a mention of compatibility in the PIP but with no details. The docs about schema compatibility state this:
> Consequently, those events need to go in the same Pulsar partition to > maintain order. This application can use ALWAYS_COMPATIBLE to allow different > kinds of events co-exist in the same topic. With this PIP, this limitation can be relaxed, and schema compatibility should be able to be strengthened, since each type of message on a topic can have its own schema, and compatibility can then be checked against only other schemas for the same type. Kafka does this via the concept of "subjects" in the schema registry, and subjects default to just the topic name (plus a "-key" or "-value" suffix since keys and values can both have their own schemas), but can also include (via an injectable strategy) the message type. Compatibility is managed at the subject level. Is this something that should be addressed in this PIP, or in future follow-on work? This is critical to supporting ordering across different message types, with schema compatibility verification by Pulsar. Regards, Raman On 2019/09/03 05:12:32, 唐谊 <ssnailt...@gmail.com> wrote: > Hi all; > > I am drafting a proposal to support the producer to send messages with > different schema. > > ## Motivation > For now, Pulsar producer can only produce messages of one type of schema > which is determined by user when it is created, or by fecthing the latest > version of schema from registry if AUTO_PRODUCE_BYTES type is specified. > Schema, however, can be updated by external system after producer started, > which would lead to inconsistency between messsage payload and schema > version metadata. Also some senarios like replicating from kafka require a > single producer for replicating messages of different schemas from one > Kafka partition to one Pulsar partition to guarantee the order and no > duplicates. > > Here proposing that messages can indicate the associated schema by itself, > for more detail, > https://gist.github.com/yittg/56c6dedf7509f634ec7effc4f6f3631d#file-pip-md > > Looking forward to any feedback. > > Thanks, > Yi >
