[
https://issues.apache.org/jira/browse/BOOKKEEPER-447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13499013#comment-13499013
]
Ivan Kelly commented on BOOKKEEPER-447:
---------------------------------------
By BOOKKEEPER-432, do you mean the skiplist approach specifically? Or simply
sorting before flushing? I have a flight next week, so I have a plan to try out
the slab based approach. I'll let you know what I find.
I was thinking about this JIRA again last night. I think it can be very easily
be solved by a Semaphore, and removing the explicit flush in #grabCleanPage().
The crux of the problem here is that we flush an index file before the
entrylogger has flushed. The entrylogger should be flushing every 100ms. The
flush mechanism is, flush the entrylog, and then flush all index page. If we
cannot grab a free index, it means that index pages are not currently being
flushed, i.e. the entrylog is flushing or we're waiting for the next flush
(this is doubtful, if we're running out of pages, flushes are taking longer
than 100ms, so once one flush ends another begins immediately).
Now, it is only safe to grab a clean page once the entrylog is flushed. Once
the entrylog has finished flushing, it will start flushing pages and freeing
them. Therefore, to grab a clean page, we just need to wait for the current
running flush to free one rather than flushing it outselves.
What will happen from the client perspective is that messages will queue while
we're waiting for a free page. But this is fine, as the system is over
capacity, so queuing is inevitable.
> Bookie can fail to recover if index pages flushed before ledger flush
> acknowledged
> ----------------------------------------------------------------------------------
>
> Key: BOOKKEEPER-447
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-447
> Project: Bookkeeper
> Issue Type: Bug
> Components: bookkeeper-server
> Affects Versions: 4.2.0
> Reporter: Yixue (Andrew) Zhu
> Assignee: Yixue (Andrew) Zhu
> Labels: patch
> Fix For: 4.2.0, 4.1.1
>
> Attachments: BOOKKEEPER-447.diff
>
>
> Bookie index page steal (LedgerCacheImpl::grabCleanPage) can cause index file
> to reflect unacknowledged entries (due to flushLedger). Suppose ledger and
> entry fail to flush due to Bookkeeper server crash, it will cause ledger
> recovery not able to use the bookie afterward, due to
> InterleavedStorageLedger::getEntry throws IOException.
> If the ackSet bookies all experience this problem (DC environment), the
> ledger will not be able to recover.
> The problem here essentially a violation of WAL. One reasonable fix is to
> track ledger flush progress (either per-ledger entry, or per-topic message).
> Do not flush index pages which tracks entries whose ledger (log) has not been
> flushed.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
