codelipenghui opened a new issue #11593: URL: https://github.com/apache/pulsar/issues/11593
**Describe the bug** We have introduced managed ledger metadata compression in https://github.com/apache/pulsar/pull/11490, but we will get `java.lang.NoSuchMethodError: java.nio.ByteBuffer.position(I)Ljava/nio/ByteBuffer;` when run Pulsar on JDK1.8. ``` 17:36:54.799 [mock-pulsar-bk-OrderedExecutor-0-0] ERROR org.apache.bookkeeper.common.util.SafeRunnable - Unexpected throwable caught java.lang.NoSuchMethodError: java.nio.ByteBuffer.position(I)Ljava/nio/ByteBuffer; at io.airlift.compress.zstd.ZstdCompressor.compress(ZstdCompressor.java:101) ~[aircompressor-0.19.jar:0.19] at org.apache.pulsar.common.compression.CompressionCodecZstd.encode(CompressionCodecZstd.java:75) ~[pulsar-common-2.8.1-SNAPSHOT.jar:2.8.1-SNAPSHOT] at org.apache.bookkeeper.mledger.impl.MetaStoreImpl.compressLedgerInfo(MetaStoreImpl.java:321) ~[managed-ledger-2.8.1-SNAPSHOT.jar:2.8.1-SNAPSHOT] at org.apache.bookkeeper.mledger.impl.MetaStoreImpl.asyncUpdateLedgerIds(MetaStoreImpl.java:134) ~[managed-ledger-2.8.1-SNAPSHOT.jar:2.8.1-SNAPSHOT] at org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl.lambda$null$1(ManagedLedgerImpl.java:491) ~[managed-ledger-2.8.1-SNAPSHOT.jar:2.8.1-SNAPSHOT] at org.apache.bookkeeper.mledger.util.SafeRun$1.safeRun(SafeRun.java:32) ~[managed-ledger-2.8.1-SNAPSHOT.jar:2.8.1-SNAPSHOT] at org.apache.bookkeeper.common.util.SafeRunnable.run(SafeRunnable.java:36) [bookkeeper-common-4.14.1.jar:4.14.1] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_281] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_281] at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) [netty-common-4.1.66.Final.jar:4.1.66.Final] at java.lang.Thread.run(Thread.java:748) [?:1.8.0_281] 17:36:58.356 [pulsar-load-manager-120-1] INFO org.apache.pulsar.broker.loadbalance.impl.ModularLoadManagerImpl - Writing local data to metadata store because maximum change 59.76470708847046% exceeded threshold 10%; time since last report written is 5.02 seconds 17:37:11.277 [TestNG-method=testRestartBrokerEnableManagedLedgerInfoCompression-1] INFO org.apache.pulsar.client.impl.ConsumerImpl - [prop/ns-abc/topic-5d9785f202336c4c] [test] Closed Consumer (not connected) 17:37:11.277 [TestNG-method=testRestartBrokerEnableManagedLedgerInfoCompression-1] INFO org.apache.pulsar.client.impl.ProducerImpl - [prop/ns-abc/topic-5d9785f202336c4c] [test-1-0] Closed Producer (not connected) !!!!!!!!! FAILURE-- [TestClass name=class org.apache.pulsar.broker.service.ManagedLedgerCompressionTest].testRestartBrokerEnableManagedLedgerInfoCompression([])------- ``` For branch-2.8 and master, build on JDK8 or JDK11, run on JDK8 can works. For branch-2.8 and master, build on JDK8 or JDK11, run on JDK11 can works. But for branch-2.7, both are works. After investigating the issue, it's related to the `aircompressor` upgrade, we have upgraded `aircompressor` from 0.16 to 0.19 https://github.com/apache/pulsar/pull/10983. After upgrade the `aircompressor` version for branch-2.7, the error appears. Looks like related to the `aircompressor`, maybe need to add --release 1.8 for `aircompressor`.  The producer also using the `aircompressor` to compress data, but after the test on both branch-2.7, branch-2.8, and master, it works, no error happens. The difference is the producer run into this branch https://github.com/apache/pulsar/blob/6704f12104219611164aa2bb5bbdfc929613f1bf/pulsar-common/src/main/java/org/apache/pulsar/common/compression/CompressionCodecZstd.java#L65 but the managed ledger metadata compression run into here https://github.com/apache/pulsar/blob/6704f12104219611164aa2bb5bbdfc929613f1bf/pulsar-common/src/main/java/org/apache/pulsar/common/compression/CompressionCodecZstd.java#L72 **To Reproduce** Steps to reproduce the behavior: 1. Build the master branch 2. Change managedLedgerInfoCompressionType=ZSTD in broker.conf 3. Start the standalone 4. See error -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
