>> When the schemas of a topic are lost, all of the messages in the topic can not be consumed successfully, and producers can not publish messages anymore.
Well, there are already many issues created related to the lost schema ledgers so, it is a very well known issue that the topic becomes unavailable and producers are not able to publish messages anymore just because of server issues which is extremely critical for mission critical usecases and not acceptable as well. Because of that brokers must be fault tolerant to recover in case of missing ledgers. Broker already handles such issues during manged-ledger or managed-cursor legers but it should also handle in case of schema ledger as well and make sure that broker won't impact topic's unavailability. It is also very important to just not rely on operational effort to fix such unavailability issues manually but broker should have a mechanism to recover by itself, Therefore, this PR is important to make sure tenants don't face topic unavailability due to well known issues of missing schema ledgers. Thanks, Rajan On Tue, Oct 8, 2024 at 8:29 PM Yubiao Feng <yubiao.f...@streamnative.io.invalid> wrote: > Hi all > > Background: When the schemas of a topic are lost, all of the messages in > the topic can not be consumed successfully, and producers can not publish > messages anymore. This mechanism alerts users to try to recover their > schemas or recreate their topics. > > https://github.com/apache/pulsar/pull/23395 added a patch: producers will > rebuild schemas if the original schemas were lost, which will mix the old > schema and new schema as the same schema ID. For example: > - send M1 with schema `Int32`, get schema id: `1` > - send M2 with schema `String`, get schema id: `2` > - schemas are lost > - send M3 with schema `String`, get schema id `1` > > The messages `M1` and `M3` use different schemas, but have the same schema > id, but users assume all things are fine, which is dangerous. So I want to > revert this PR. > > Thanks > Yubiao Feng >
