zhaoyajun2009 opened a new issue, #21688: URL: https://github.com/apache/pulsar/issues/21688
### Search before asking - [X] I searched in the [issues](https://github.com/apache/pulsar/issues) and found nothing similar. ### Version Can not offloading the data to HDFS. the error is ` java.lang.NoSuchMethodError: `java.lang.String org.apache.pulsar.common.naming.TopicName.fromPersistenceNamingEncoding(java.lang.String)' ` ### Minimal reproduce step Versions: pulsar: 3.0.0 tiered-storage-file-system: 3.0.1 hdfs:3.2.1 1、Prepare the HDFS enviroment 2、download the tiered-storage-file-system-3.0.1.nar to $PULSAR_HOME/offloaders 3、edit the configuration of standalone.conf ``` managedLedgerOffloadDriver=filesystem # offloadersDirectory=./offloaders fileSystemURI=hdfs://10.10.x.x:9000 fileSystemProfilePath=../conf/filesystem_offload_core_site.xml ``` 4、edit the configuration of filesystem_offload_core_site.xml ``` <property> <name>fs.defaultFS</name> <value>hdfs://10.10.x.x:9000</value> </property> <property> <name>hadoop.tmp.dir</name> <value>/tmp/hadoop-root/dfs/data</value> </property> ``` 5、Start standalone service `./bin/pulsar standalone` 6、 find a topic which have the closed ledgers `bin/pulsar-admin topics stats-internal public/default/fs-test3` `"ledgers" : [ { "ledgerId" : 2042, "entries" : 1001, "size" : 75819, "offloaded" : false, "underReplicated" : false }, { "ledgerId" : 2044, "entries" : 999, "size" : 75797, "offloaded" : false, "underReplicated" : false }, { "ledgerId" : 2419, "entries" : 0, "size" : 0, "offloaded" : false, "underReplicated" : false } ], ` 7、Trigger the offload manually `[root@k8s-node2 apache-pulsar-3.0.0]# bin/pulsar-admin topics offload -s 10 public/default/fs-test3 Offload triggered for persistent://public/default/fs-test3 for messages before 2419:0:-1 ` Then the error log of broker is as follows, and can not offload the data to filesystem. ### What did you expect to see? The output of the offload is as follow: Offload was a success ### What did you see instead? the logs of broker ``` 2023-12-07T17:08:54,789+0800 [pulsar-web-57-4] INFO org.apache.pulsar.broker.service.persistent.PersistentTopic - [persistent://public/default/fs-test3] Starting offload operation at messageId 2400:0:-1 2023-12-07T17:08:54,790+0800 [pulsar-web-57-4] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/fs-test3] Start ledgersOffload. ledgers=[2042, 2044, 2400] totalSize=151616 2023-12-07T17:08:54,791+0800 [pulsar-web-57-4] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/fs-test3] Going to offload ledgers [2042, 2044] 2023-12-07T17:08:54,792+0800 [pulsar-web-57-4] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/fs-test3] Preparing metadata to offload ledger 2042 with uuid 5a4e4bf0-997e-4cf3-8df3-5f05a3562a02 2023-12-07T17:08:54,794+0800 [pulsar-web-57-4] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/fs-test3] Found previous offload attempt for ledger 2042, uuid 5a4e4bf0-997e-4cf3-8df3-5f05a3562a02, cleaning up 2023-12-07T17:08:54,797+0800 [pulsar-web-57-4] INFO org.apache.bookkeeper.mledger.offload.OffloadUtils - [public/default/persistent/fs-test3] Cleanup offload for ledgerId 2042 uuid 7ead0b15-96cb-4f7c-84a9-95a3ae2cb803 because of the reason Previous failed offload. 2023-12-07T17:08:54,818+0800 [bookkeeper-ml-scheduler-OrderedScheduler-2-0] WARN org.apache.bookkeeper.common.util.SingleThreadSafeScheduledExecutorService - Unexpected throwable from task class org.apache.bookkeeper.common.util.OrderedExecutor$TimedRunnable: 'java.lang.String org.apache.pulsar.common.naming.TopicName.fromPersistenceNamingEncoding(java.lang.String)' java.lang.NoSuchMethodError: 'java.lang.String org.apache.pulsar.common.naming.TopicName.fromPersistenceNamingEncoding(java.lang.String)' at org.apache.bookkeeper.mledger.offload.filesystem.impl.FileSystemManagedLedgerOffloader.deleteOffloaded(FileSystemManagedLedgerOffloader.java:377) ~[?:?] at org.apache.bookkeeper.mledger.offload.OffloadUtils.lambda$cleanupOffloaded$6(OffloadUtils.java:203) ~[org.apache.pulsar-managed-ledger-3.0.0.jar:3.0.0] at org.apache.bookkeeper.common.util.Retries.execute(Retries.java:118) ~[org.apache.bookkeeper-bookkeeper-common-4.16.1.jar:4.16.1] at org.apache.bookkeeper.common.util.Retries.lambda$run$1(Retries.java:98) ~[org.apache.bookkeeper-bookkeeper-common-4.16.1.jar:4.16.1] at org.apache.bookkeeper.common.util.OrderedExecutor$TimedRunnable.run(OrderedExecutor.java:201) ~[org.apache.bookkeeper-bookkeeper-common-4.16.1.jar:4.16.1] at org.apache.bookkeeper.common.util.SingleThreadSafeScheduledExecutorService$SafeRunnable.run(SingleThreadSafeScheduledExecutorService.java:46) ~[org.apache.bookkeeper-bookkeeper-common-4.16.1.jar:4.16.1] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539) ~[?:?] at java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[?:?] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304) ~[?:?] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) ~[?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) ~[?:?] at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[io.netty-netty-common-4.1.89.Final.jar:4.1.89.Final] at java.lang.Thread.run(Thread.java:833) ~[?:?] 2023-12-07T17:08:54,819+0800 [bookkeeper-ml-scheduler-OrderedScheduler-2-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/fs-test3] Metadata prepared for offload of ledger 2042 with uuid 5a4e4bf0-997e-4cf3-8df3-5f05a3562a02 2023-12-07T17:08:54,822+0800 [offloader-OrderedScheduler-1-0] WARN org.apache.bookkeeper.common.util.SingleThreadSafeScheduledExecutorService - Unexpected throwable from task class org.apache.bookkeeper.mledger.offload.filesystem.impl.FileSystemManagedLedgerOffloader$LedgerReader: 'java.lang.String org.apache.pulsar.common.naming.TopicName.fromPersistenceNamingEncoding(java.lang.String)' java.lang.NoSuchMethodError: 'java.lang.String org.apache.pulsar.common.naming.TopicName.fromPersistenceNamingEncoding(java.lang.String)' at org.apache.bookkeeper.mledger.offload.filesystem.impl.FileSystemManagedLedgerOffloader$LedgerReader.run(FileSystemManagedLedgerOffloader.java:204) ~[?:?] at org.apache.bookkeeper.common.util.SingleThreadSafeScheduledExecutorService$SafeRunnable.run(SingleThreadSafeScheduledExecutorService.java:46) ~[org.apache.bookkeeper-bookkeeper-common-4.16.1.jar:4.16.1] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539) ~[?:?] at java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[?:?] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304) ~[?:?] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) ~[?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) ~[?:?] at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[io.netty-netty-common-4.1.89.Final.jar:4.1.89.Final] at java.lang.Thread.run(Thread.java:833) ~[?:?] 2023-12-07T17:08:54,825+0800 [pulsar-web-57-4] INFO org.eclipse.jetty.server.RequestLog - 127.0.0.1 - - [07/12月/2023:17:08:54 +0800] "PUT /admin/v2/persistent/public/default/fs-test3/offload HTTP/1.1" 204 0 "-" "Pulsar-Java-v3.0.0" 45 ``` ### Anything else? _No response_ ### Are you willing to submit a PR? - [ ] I'm willing to submit a PR! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
