[
https://issues.apache.org/jira/browse/BOOKKEEPER-695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13797294#comment-13797294
]
Matteo Merli commented on BOOKKEEPER-695:
-----------------------------------------
I'm not sure where it would be best to handle this case. My understanding is
that we should ignore the truncated entries and compact/gc the entry log as
needed.
Perhaps it would be easier to handle it in
GarbageCollectorThead.extractMetaFromEntryLog, so that it will only affect the
GC scanning. If there are truncated/corrupted entries, most likely they will be
replayed by the journal, but in any case we cannot do anything to bring them
back to life, so we can just skip them.
Something like:
{code:java}
try {
entryLogger.scanEntryLog(entryLogId, scanner);
} catch (IOException e) {
/// print warning but continue
}
{code}
> Some entry logs are not removed from the bookie storage
> -------------------------------------------------------
>
> Key: BOOKKEEPER-695
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-695
> Project: Bookkeeper
> Issue Type: Bug
> Reporter: Matteo Merli
> Priority: Minor
>
> Some entry logs appear to be truncated (possible at a momemnt when the bookie
> was shut down) and the compaction is never getting rid of them:
> {code}
> 00:00:06.448 [GarbageCollectorThread] INFO
> o.a.b.bookie.GarbageCollectorThread - Extracting entry log meta from
> entryLogId: 1497
> 00:00:07.140 [GarbageCollectorThread] WARN
> o.a.b.bookie.GarbageCollectorThread - Premature exception when processing
> 1497 recovery will take care of the problem
> java.io.IOException: Short read for ledger entry from entryLog
> 1497@718702792(31356!=32840)
> at
> org.apache.bookkeeper.bookie.EntryLogger.scanEntryLog(EntryLogger.java:514)
> ~[bookkeeper-server-4.2.1.15.jar:4.2.2-SNAPSHOT]
> at
> org.apache.bookkeeper.bookie.GarbageCollectorThread.extractMetaFromEntryLog(GarbageCollectorThread.java:572)
> [bookkeeper-server-4.2.1.15.jar:4.2.2-SNAPSHOT]
> at
> org.apache.bookkeeper.bookie.GarbageCollectorThread.extractMetaFromEntryLogs(GarbageCollectorThread.java:549)
> [bookkeeper-server-4.2.1.15.jar:4.2.2-SNAPSHOT]
> at
> org.apache.bookkeeper.bookie.GarbageCollectorThread.run(GarbageCollectorThread.java:268)
> [bookkeeper-server-4.2.1.15.jar:4.2.2-SNAPSHOT]
> {code}
> These entry logs are not being removed:
> {code}
> $ ll -h *.log
> -rw-r--r-- 1 yahoo users 686M Jul 17 10:45 5d9.log
> -rw-r--r-- 1 yahoo users 634M Jul 27 19:31 b2d.log
> -rw-r--r-- 1 yahoo users 1.5G Jul 28 15:22 b35.log
> -rw-r--r-- 1 yahoo users 2.0G Aug 4 04:57 dbd.log
> -rw-r--r-- 1 yahoo users 2.0G Aug 4 04:58 dbe.log
> .....
> {code}
--
This message was sent by Atlassian JIRA
(v6.1#6144)
