+1, +1 The load balancing has a good migration guide ( https://pulsar.apache.org/docs/4.0.x/concepts-broker-load-balancing-migration/ )
There are two scenarios we need to consider and handle: 1. How to migrate topics that are already using Pulsar's built-in schema to an external one. This scenario should probably not be rejected. Users must first manually migrate the Pulsar built-in schema to the external registry and then switch to the external schema registry. After this, the built-in schema information can be asynchronously deleted. 2. When older version consumers, which don't support the external schema registry, connect to this topic. This might require stopping the consumers and updating them to a new version. Are there any scenarios where such older version consumers *must* continue to exist? Perhaps. If so, we could consider that after successfully migrating to the external schema registry and cleaning up the built-in schema, we switch to a new `SchemaStorage` implementation (which can only `get` from the external schema registry and does not allow `put` or `delete` operations). Older version consumers would then only be able to consume the topic using the schema that existed before the migration. This might require more discussion, and the behavior of the admin API in these contexts might also need further consideration. Adding a new schema type is a good start. Thanks, sinan PengHui Li <peng...@apache.org>于2025年6月3日 周二08:16写道： > Thanks for the proposal. > > The motivation looks good to me, users can connect to their preferred or > customized schema registry with this proposal, which can get rid of the > limitation from the Pulsar built-in schema registry. > > I have a few questions about the migration or compatibility. > > - For a topic that already uses pulsar's built-in schema, will the client > be able to switch to an external schema registry? As I understand, we > should reject this case since it will mess up the schema compatibility with > 2 schema registries > - And how about the old version(without external schema registry support) > consumers connected to the topic that has schema from external schema > registry? > > We probably need to consider adding another schema type instead of using > the bytes schema. If the topic has a schema from an external schema > registry, which means the Pulsar broker will not manage schemas for this > topic. > > We should add more details about the compatibility to let users understand > the proper way to move to the external schema registry. > > Regards, > Penghui > > On Wed, May 21, 2025 at 9:23 AM Ran Gao <r...@apache.org> wrote: > > > Hi, Pulsar Community. > > > > I opened a new PIP to provide the Pulsar client with the ability to > > integrate with a third-party schema registry service. I'm looking > > forward to your suggestions! > > > > link: https://github.com/apache/pulsar/pull/24328 > > > > Thanks, > > Ran Gao > > > > -- > > This email and any attachments are intended solely for the recipient(s) > > named above and may contain confidential, privileged, or proprietary > > information. If you are not the intended recipient, you are hereby > > notified > > that any disclosure, copying, distribution, or reproduction of this > > information is strictly prohibited. If you have received this email in > > error, please notify the sender immediately by replying to this email and > > delete it from your system. > > >
