kaykillyou commented on issue #5668: bookie server's log error: Failed to restore rocksdb 000000000000000000/000000000000000001/000000000000000000 URL: https://github.com/apache/pulsar/issues/5668#issuecomment-558916765 @jiazhai Thanks for your reply! today my bookie server found this issue again, when i restart bookie service ,it also hapen. And each bookie's heap memory usage reaches 97%. I try to start the broker with command "./pulsar-daemon start broker -nss", then failed! I don’t know if I use it like this！ Looking forward to your reply！ log error: 11:27:48.175 [BookieReadThreadPool-OrderedExecutor-1-0] INFO org.apache.bookkeeper.proto.ReadEntryProcessorV3 - No ledger found while reading entry: 0 from ledger: 118609 11:27:48.175 [BookieReadThreadPool-OrderedExecutor-4-0] INFO org.apache.bookkeeper.proto.ReadEntryProcessorV3 - No ledger found while reading entry: 0 from ledger: 116820 11:27:48.176 [BookieReadThreadPool-OrderedExecutor-1-0] INFO org.apache.bookkeeper.proto.ReadEntryProcessorV3 - No ledger found while reading entry: 0 from ledger: 118609 11:27:48.176 [ReplicationWorker] INFO org.apache.bookkeeper.replication.ReplicationWorker - Ledger replicated successfully. ledger id is: 118609 11:27:48.188 [io-write-scheduler-OrderedScheduler-1-0] INFO org.apache.distributedlog.BKDistributedLogManager - Create sync reader starting from DLSN{logSegmentSequenceNo=1, entryId=0, slotId=0} 11:27:48.197 [io-write-scheduler-OrderedScheduler-1-0] ERROR org.apache.bookkeeper.statelib.impl.rocksdb.checkpoint.RocksCheckpointer - Failed to restore rocksdb 000000000000000006/000000000000000001/000000000000000000 java.io.FileNotFoundException: 000000000000000006/000000000000000001/000000000000000000/checkpoints/781fb64f-dcc8-4ec6-bd33-2b2e0b7d8e0c/metadata at org.apache.bookkeeper.statelib.impl.rocksdb.checkpoint.dlog.DLCheckpointStore.openInputStream(DLCheckpointStore.java:92) ~[org.apache.bookkeeper-statelib-4.9.2.jar:4.9.2] at org.apache.bookkeeper.statelib.impl.rocksdb.checkpoint.RocksCheckpointer.getLatestCheckpoint(RocksCheckpointer.java:117) ~[org.apache.bookkeeper-statelib-4.9.2.jar:4.9.2] at org.apache.bookkeeper.statelib.impl.rocksdb.checkpoint.RocksCheckpointer.restore(RocksCheckpointer.java:52) ~[org.apache.bookkeeper-statelib-4.9.2.jar:4.9.2] at org.apache.bookkeeper.statelib.impl.kv.RocksdbKVStore.loadRocksdbFromCheckpointStore(RocksdbKVStore.java:161) ~[org.apache.bookkeeper-statelib-4.9.2.jar:4.9.2] at org.apache.bookkeeper.statelib.impl.kv.RocksdbKVStore.init(RocksdbKVStore.java:223) ~[org.apache.bookkeeper-statelib-4.9.2.jar:4.9.2] at org.apache.bookkeeper.statelib.impl.journal.AbstractStateStoreWithJournal.lambda$initializeLocalStore$5(AbstractStateStoreWithJournal.java:202) ~[org.apache.bookkeeper-statelib-4.9.2.jar:4.9.2] at org.apache.bookkeeper.statelib.impl.journal.AbstractStateStoreWithJournal.lambda$executeIO$16(AbstractStateStoreWithJournal.java:471) ~[org.apache.bookkeeper-statelib-4.9.2.jar:4.9.2] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [?:1.8.0_144] at com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:111) [com.google.guava-guava-21.0.jar:?] at com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:58) [com.google.guava-guava-21.0.jar:?] at com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:75) [com.google.guava-guava-21.0.jar:?] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [?:1.8.0_144] at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_144] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) [?:1.8.0_144] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) [?:1.8.0_144] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_144] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_144] at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) [io.netty-netty-all-4.1.32.Final.jar:4.1.32.Final] at java.lang.Thread.run(Thread.java:748) [?:1.8.0_144] 11:27:48.198 [io-write-scheduler-OrderedScheduler-1-0] INFO org.apache.bookkeeper.stream.storage.impl.store.MVCCStoreFactoryImpl - Clearing resources hold by stream(1)/range(0) at storage container (6) 11:27:48.198 [io-write-scheduler-OrderedScheduler-1-0] WARN org.apache.bookkeeper.stream.storage.impl.sc.StorageContainerRegistryImpl - De-registered StorageContainer ('6') when failed to start java.util.concurrent.CompletionException: org.apache.bookkeeper.statelib.api.exceptions.StateStoreException: Failed to restore rocksdb 000000000000000006/000000000000000001/000000000000000000 at java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:292) ~[?:1.8.0_144] at java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:308) ~[?:1.8.0_144] at java.util.concurrent.CompletableFuture.uniCompose(CompletableFuture.java:943) ~[?:1.8.0_144] at java.util.concurrent.CompletableFuture$UniCompose.tryFire(CompletableFuture.java:926) ~[?:1.8.0_144] at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474) ~[?:1.8.0_144] at java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:1977) ~[?:1.8.0_144] at org.apache.bookkeeper.statelib.impl.journal.AbstractStateStoreWithJournal.lambda$executeIO$16(AbstractStateStoreWithJourn
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
