[
https://issues.apache.org/jira/browse/BOOKKEEPER-664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13760461#comment-13760461
]
Ivan Kelly edited comment on BOOKKEEPER-664 at 9/6/13 6:15 PM:
---------------------------------------------------------------
{quote}
1) it would be better to add a test case to cover the case you described in the
comment to ensure future changes would not break it.
{quote}
test attached.
{quote}
2) rotate listener defers the flushing, which would break some test cases in
compaction test if I remembered correctly. so you might need to change those
test cases.
{quote}
Moving somethingWritten into EntryLogger fixes this problem.
{quote}
3) add compactionMaxOutstandingRequests and compactionRate to document & config
file
{quote}
Ive added to the config. I dont want to add to the doc yet. We havent played
with the parameters ourselves yet. I'd prefer if users didnt either without
asking us.
{quote}
4) use {} rather than "+" in log statements.
{quote}
fixed
{quote}5) you might need to notify flushLock when shutting down similar as what
I do in the patch. since entry logger might not rotate during shutting
down.{quote}
Fixed by checking 'running' in the wait loop
was (Author: ikelly):
{quote}
1) it would be better to add a test case to cover the case you described in the
comment to ensure future changes would not break it.
{quote}
test attached.
{quote}
2) rotate listener defers the flushing, which would break some test cases in
compaction test if I remembered correctly. so you might need to change those
test cases.
{quote}
Moving somethingWritten into EntryLogger fixes this problem.
{quote}
3) add compactionMaxOutstandingRequests and compactionRate to document & config
file
{quote}
Ive added to the config. I dont want to add to the doc yet. We havent played
with the parameters ourselves yet. I'd prefer if users didnt either without
asking us.
{fixed}
4) use {} rather than "+" in log statements.
{quote}
fixed
{quote}5) you might need to notify flushLock when shutting down similar as what
I do in the patch. since entry logger might not rotate during shutting
down.{quote}
Fixed by checking 'running' in the wait loop
> Compaction increases latency on journal writes
> ----------------------------------------------
>
> Key: BOOKKEEPER-664
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-664
> Project: Bookkeeper
> Issue Type: Bug
> Components: bookkeeper-server
> Affects Versions: 4.2.1
> Reporter: Ivan Kelly
> Assignee: Ivan Kelly
> Fix For: 4.2.2, 4.3.0
>
> Attachments:
> 0001-BOOKKEEPER-664-branch-4.2-Compaction-increases-laten.patch,
> 0001-BOOKKEEPER-664-branch-4.2-Compaction-increases-laten.patch,
> 0001-BOOKKEEPER-664-branch-4.2-Compaction-increases-laten.patch,
> 0001-BOOKKEEPER-664-Compaction-increases-latency-on-journ_branch4.2.patch,
> 0001-BOOKKEEPER-664-Compaction-increases-latency-on-journ-branch-4.2.patch,
> 0001-BOOKKEEPER-664-Compaction-increases-latency-on-journ.patch,
> 0001-BOOKKEEPER-664-Compaction-increases-latency-on-journ_trunk.patch,
> 0002-BOOKKEEPER-664-trunk-Compaction-increases-latency-on.patch, bench.png,
> BOOKKEEPER-664.patch
>
>
> Compaction writes to the journal to avoid data loss (see BOOKKEEPER-530).
> BOOKKEEPER-530 correctly identified that this may affect latency on the
> journal but we have observed this since in production. It is possible to
> avoid the journal completely, as twitter do in their github branch.
> Basically, we need to write to the entrylogger first, flush the entry log and
> then add to the index.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
