david-streamlio commented on issue #99: URL: https://github.com/apache/pulsar-connectors/issues/99#issuecomment-4941520548
Root-caused this on a Linux x86-64 host while verifying #98. The findings narrow the issue considerably: **1. The protobuf 4.x on the canal test classpath came from `mysql-connector-j`, not the platform.** `gradle :canal:dependencyInsight` shows `protobuf-java:4.31.1` selected "by conflict resolution: between versions 4.31.1, 3.6.1 and 3.25.5" — the 4.31.1 is a transitive of the test-only `com.mysql:mysql-connector-j:9.4.0` (X DevAPI support, unused by plain JDBC). The platform itself pins 3.25.5. **2. `makeExtensionsImmutable()` exists in protobuf-java 3.25.5 — it was removed in the 4.x line.** Verified empirically, not just by reading release notes: a standalone canal 1.1.x client run against exactly `protobuf-java-3.25.5.jar` completes the canal-server handshake and delivers CDC entries (`GOT MESSAGE id=1 ... table=products`). So the incompatibility is canal ↔ protobuf **4.x**, not "modern protobuf" generally. **3. Production is not affected today.** The NAR bundles no protobuf (the `com.google.protobuf` group is excluded from NARs by convention; the Pulsar runtime's java-instance provides it), and the Pulsar runtime ships the 3.25.x line, which retains the method. The connector works against the runtime-provided protobuf. **4. Upgrading to canal 1.1.8 does not fix the 4.x incompatibility.** Its regenerated `CanalPacket$Packet` still calls `makeExtensionsImmutable()` (verified via `javap -c` on `canal.protocol-1.1.8.jar` — note the call sites are in the *inner* `Packet` classes, easy to miss when inspecting the outer class). An earlier attempt to run the integration test with client+server 1.1.8 and no pin failed with the same `NoSuchMethodError`. **Resolution taken in #98:** drop the module-wide 3.6.1 force entirely and instead exclude `protobuf-java` from the test-only `mysql-connector-j` dependency. The test classpath then resolves the platform's 3.25.5 — the same protobuf line production uses — and the integration test passes with no pin. **What remains of this issue:** a forward-looking constraint, not a current bug — if/when the Pulsar runtime moves to protobuf 4.x, the canal client (any released version, 1.1.8 included) breaks on `connect()`. Options then are shading protobuf into the canal NAR or an upstream canal fix. Suggest either re-titling this issue to track that constraint or closing it in favor of a note in the module. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
