disserakt opened a new issue, #3445:
URL: https://github.com/apache/bookkeeper/issues/3445
Hi all!
Sometimes we see this ERROR message in the Bookkeeper logs:
`"log.level":"ERROR","message":"Failed to compact entry log 1111111111 due
to unexpected error"`
We use Pulsar version for now - `2.9.2` and Bookkkeeper - `4.14.4 `
Our cluster consists of 9 brokers Pulsar and 9 bookies Bookkeeper. We use
partitioned topics with 6 partitions. And our installation Pulsar <> Bookkeeper
<> Zookeeper - use SSL.
We see several types of them for this error.
Complete stacktrace this error with - `"error.message": "Negative position"`
`{
"log.level": "ERROR",
"message": "Failed to compact entry log 19385682 due to unexpected
error",
"process.thread.name": "GarbageCollectorThread-9-1",
"log.logger": "org.apache.bookkeeper.bookie.GarbageCollectorThread",
"error.type": "java.lang.IllegalArgumentException",
"error.message": "Negative position",
"error.stack_trace": "java.lang.IllegalArgumentException: Negative
position\n\tat
java.base/sun.nio.ch.FileChannelImpl.read(FileChannelImpl.java:785)\n\tat
org.apache.bookkeeper.bookie.BufferedReadChannel.read(BufferedReadChannel.java:93)\n\tat
org.apache.bookkeeper.bookie.BufferedReadChannel.read(BufferedReadChannel.java:65)\n\tat
org.apache.bookkeeper.bookie.EntryLogger.readFromLogChannel(EntryLogger.java:418)\n\tat
org.apache.bookkeeper.bookie.EntryLogger.scanEntryLog(EntryLogger.java:996)\n\tat
org.apache.bookkeeper.bookie.EntryLogCompactor.compact(EntryLogCompactor.java:61)\n\tat
org.apache.bookkeeper.bookie.GarbageCollectorThread.compactEntryLog(GarbageCollectorThread.java:568)\n\tat
org.apache.bookkeeper.bookie.GarbageCollectorThread.doCompactEntryLogs(GarbageCollectorThread.java:487)\n\tat
org.apache.bookkeeper.bookie.GarbageCollectorThread.runWithFlags(GarbageCollectorThread.java:382)\n\tat
org.apache.bookkeeper.bookie.GarbageCollectorThread.safeRun(GarbageCollectorTh
read.java:329)\n\tat
org.apache.bookkeeper.common.util.SafeRunnable.run(SafeRunnable.java:36)\n\tat
java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)\n\tat
java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305)\n\tat
java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305)\n\tat
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)\n\tat
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)\n\tat
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)\n\tat
java.base/java.lang.Thread.run(Thread.java:834)\n"
}`
And complete stacktrace this error with - `"error.message": "Keys and
values must be >= 0"`
`{
"log.level": "ERROR",
"message": "Failed to compact entry log 19382915 due to unexpected
error",
"process.thread.name": "GarbageCollectorThread-9-1",
"log.logger": "org.apache.bookkeeper.bookie.GarbageCollectorThread",
"error.type": "java.lang.IllegalArgumentException",
"error.message": "Keys and values must be >= 0",
"error.stack_trace": "java.lang.IllegalArgumentException: Keys and
values must be >= 0\n\tat
org.apache.bookkeeper.util.collections.ConcurrentLongLongHashMap.checkBiggerEqualZero(ConcurrentLongLongHashMap.java:731)\n\tat
org.apache.bookkeeper.util.collections.ConcurrentLongLongHashMap.get(ConcurrentLongLongHashMap.java:141)\n\tat
org.apache.bookkeeper.util.collections.ConcurrentLongLongHashMap.containsKey(ConcurrentLongLongHashMap.java:147)\n\tat
org.apache.bookkeeper.bookie.EntryLogMetadata.containsLedger(EntryLogMetadata.java:51)\n\tat
org.apache.bookkeeper.bookie.EntryLogCompactor$CompactionScannerFactory$1.accept(EntryLogCompactor.java:90)\n\tat
org.apache.bookkeeper.bookie.EntryLogger.scanEntryLog(EntryLogger.java:1006)\n\tat
org.apache.bookkeeper.bookie.EntryLogCompactor.compact(EntryLogCompactor.java:61)\n\tat
org.apache.bookkeeper.bookie.GarbageCollectorThread.compactEntryLog(GarbageCollectorThread.java:568)\n\tat
org.apache.bookkeeper.bookie.GarbageCollectorThread.doC
ompactEntryLogs(GarbageCollectorThread.java:487)\n\tat
org.apache.bookkeeper.bookie.GarbageCollectorThread.runWithFlags(GarbageCollectorThread.java:382)\n\tat
org.apache.bookkeeper.bookie.GarbageCollectorThread.safeRun(GarbageCollectorThread.java:329)\n\tat
org.apache.bookkeeper.common.util.SafeRunnable.run(SafeRunnable.java:36)\n\tat
java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)\n\tat
java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305)\n\tat
java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305)\n\tat
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)\n\tat
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)\n\tat
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)\n\tat
java.base/java.lang.Thread.run(Thread.java:834)\n"
} `
I made this issue based on a thread in Pulsar Slack:
https://apache-pulsar.slack.com/archives/C5Z4T36F7/p1660056439897459
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
