+1 - Mattison
On Tue, 17 Mar 2026 at 06:50, Matteo Merli <[email protected]> wrote: > https://github.com/apache/pulsar/pull/25329 > > # PIP-462: Remove Etcd metadata store backend > > ## Motivation > > Apache Pulsar supports multiple metadata store backends through the > pluggable `MetadataStore` interface introduced in PIP-45. One of these > backends is Etcd, implemented via the `jetcd` client library. While > the Etcd backend has been available since the metadata store > abstraction was introduced, it has seen very limited adoption in > production environments. > > Maintaining Etcd support imposes several costs on the project: > > 1. **Dependency burden**: The `jetcd-core` library pulls in a large > transitive dependency tree (gRPC, Netty, Vert.x, and others) that > conflicts with dependencies used elsewhere in Pulsar. This required > creating a dedicated shading module (`jetcd-core-shaded`) with > relocations for `io.vertx` and `io.netty` packages. > > 2. **Distribution size**: The shaded jetcd JAR and its transitive > dependencies increase both the Pulsar tarball and Docker image size > significantly, even for users who never use Etcd. > > 3. **Library maintenance concerns**: The `jetcd` library is not > actively maintained, making it increasingly risky to depend on for a > production metadata backend. Security patches and compatibility > updates may lag behind. > > 4. **Maintenance overhead**: The `EtcdMetadataStore` implementation, > `EtcdSessionWatcher`, and associated test infrastructure must be > maintained, tested, and considered in every metadata store interface > change. > > 5. **Strategic direction**: Both ZooKeeper and Oxia are fully > supported and battle-tested metadata store backends. Oxia is a > purpose-built metadata store designed for Pulsar with superior > operational characteristics, and the community's future efforts are > focused there. Maintaining a third, unused backend dilutes this focus. > > Given that Etcd is not used in production deployments, adds > significant distribution size, relies on a poorly maintained client > library, and competes for attention with the strategically preferred > Oxia backend, removing it simplifies the project and reduces the > maintenance surface area. > > ## Goals > > ### In Scope > > - Remove the `EtcdMetadataStore` and `EtcdSessionWatcher` > implementation classes from `pulsar-metadata` > - Remove the `EtcdMetadataStoreProvider` registration from > `MetadataStoreFactoryImpl` > - Remove the `jetcd-core-shaded` shading module entirely > - Remove `jetcd-core` and `jetcd-test` dependency declarations from > the root `pom.xml` > - Remove Etcd-specific tests (`EtcdMetadataStoreTest`) > - Remove or update any documentation references to Etcd as a supported > backend > - Provide a clear error message if a user configures an `etcd:` > metadata store URL, guiding them to use ZooKeeper or Oxia instead > > ### Out of Scope > > - Changes to ZooKeeper or Oxia metadata store implementations > - Changes to the `MetadataStore` interface or plugin system > > ## High Level Design > > The removal is a straightforward deletion of code and dependencies: > > 1. **Implementation removal**: Delete `EtcdMetadataStore.java` and > `EtcdSessionWatcher.java` from `pulsar-metadata`. > > 2. **Provider deregistration**: Remove the Etcd provider from > `MetadataStoreFactoryImpl`. Add a check that produces a clear error > message when an `etcd:` URL is configured, informing operators that > Etcd is no longer supported and recommending ZooKeeper or Oxia. > > 3. **Module removal**: Delete the `jetcd-core-shaded` module and > remove it from the root `pom.xml` modules list. > > 4. **Dependency cleanup**: Remove `jetcd-core`, `jetcd-test`, and > `jetcd-core-shaded` dependency declarations from the root `pom.xml` > dependency management section. Remove these dependencies from > `pulsar-metadata/pom.xml`. > > 5. **Test removal**: Delete `EtcdMetadataStoreTest.java`. > > ## Detailed Design > > ### Implementation Classes > > The following source files will be deleted: > - > `pulsar-metadata/src/main/java/org/apache/pulsar/metadata/impl/EtcdMetadataStore.java` > (includes inner `EtcdMetadataStoreProvider` class) > - > `pulsar-metadata/src/main/java/org/apache/pulsar/metadata/impl/EtcdSessionWatcher.java` > > ### Factory Update > > In `MetadataStoreFactoryImpl`, the Etcd provider registration will be > removed from `loadProviders()`. A runtime check will be added so that > if a user configures `metadataStoreUrl=etcd:...`, the broker fails > fast with a clear error: > > ``` > Etcd metadata store backend has been removed in Pulsar 5.0. > Please use ZooKeeper (zk:) or Oxia (oxia:) as your metadata store. > See PIP-462 for details. > ``` > > ### Module Removal > > The `jetcd-core-shaded/` directory will be deleted entirely, and the > `<module>jetcd-core-shaded</module>` entry will be removed from the > root `pom.xml`. > > ### Dependency Removal > > The following entries will be removed from the root `pom.xml`: > > From `<dependencyManagement>`: > - `io.etcd:jetcd-core` > - `io.etcd:jetcd-test` > - `org.apache.pulsar:jetcd-core-shaded` > > From `pulsar-metadata/pom.xml`: > - `org.apache.pulsar:jetcd-core-shaded` (compile dependency) > - `io.etcd:jetcd-test` (test dependency) > > ### Test Removal > > The following test file will be deleted: > - > `pulsar-metadata/src/test/java/org/apache/pulsar/metadata/impl/EtcdMetadataStoreTest.java` > > ## Public-facing Changes > > ### Configuration > > The `metadataStoreUrl` and `configurationMetadataStoreUrl` > configuration keys will no longer accept `etcd:` URLs. Attempting to > start a broker, proxy, or other Pulsar component with an Etcd metadata > URL will result in a clear error message at startup. > > ### Binary Protocol > > No changes. > > ### CLI > > No changes. CLI tools that accept metadata store URLs (e.g., `pulsar > initialize-cluster-metadata`) will reject `etcd:` URLs with a > descriptive error. > > ### Metrics > > No changes. > > ## General Notes > > The `MetadataStore` plugin system (PIP-45) remains fully intact. If > there is community interest in continuing Etcd support, it can be > maintained as an external plugin outside the core Pulsar repository, > without any changes to the Pulsar codebase. > > ### Related PIPs > > - PIP-45: Pluggable metadata interface (introduced the MetadataStore > abstraction and Etcd backend) > - PIP-454: Metadata Store Migration Framework (provides migration path > for users needing to switch backends) > > > > -- > Matteo Merli > <[email protected]> >
