[
https://issues.apache.org/jira/browse/BOOKKEEPER-664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13758940#comment-13758940
]
Ivan Kelly commented on BOOKKEEPER-664:
---------------------------------------
The attached patch falls to the same issue that caused us to go through the
journal in the first place. We need to defer the add to the index, otherwise
the index could be flushed before entrylog, and a crash would cause entry loss.
I remembered the solution to the sync problem after I went home. You guys added
double buffering of a sort, that entrylogger was only flushed after being
rotated, thereby taking it out of the critical path. That's too big of a change
for 4.2.2 though. Throttling should give a similar affect though.
I think the solution is a combination of both patches. I'll have a go.
> Compaction increases latency on journal writes
> ----------------------------------------------
>
> Key: BOOKKEEPER-664
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-664
> Project: Bookkeeper
> Issue Type: Bug
> Components: bookkeeper-server
> Affects Versions: 4.2.1
> Reporter: Ivan Kelly
> Assignee: Ivan Kelly
> Fix For: 4.2.2, 4.3.0
>
> Attachments:
> 0001-BOOKKEEPER-664-branch-4.2-Compaction-increases-laten.patch,
> 0001-BOOKKEEPER-664-Compaction-increases-latency-on-journ_branch4.2.patch,
> 0001-BOOKKEEPER-664-Compaction-increases-latency-on-journ-branch-4.2.patch,
> 0001-BOOKKEEPER-664-Compaction-increases-latency-on-journ.patch,
> 0001-BOOKKEEPER-664-Compaction-increases-latency-on-journ_trunk.patch,
> 0002-BOOKKEEPER-664-trunk-Compaction-increases-latency-on.patch, bench.png,
> BOOKKEEPER-664.patch
>
>
> Compaction writes to the journal to avoid data loss (see BOOKKEEPER-530).
> BOOKKEEPER-530 correctly identified that this may affect latency on the
> journal but we have observed this since in production. It is possible to
> avoid the journal completely, as twitter do in their github branch.
> Basically, we need to write to the entrylogger first, flush the entry log and
> then add to the index.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
