[
https://issues.apache.org/jira/browse/KAFKA-551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13472460#comment-13472460
]
Jay Kreps commented on KAFKA-551:
---------------------------------
I think (hope) that is what we are doing. Here is the logic in
OffsetIndex.truncateTo:
/* There are 3 cases for choosing the new size
* 1) if there is no entry in the index <= the offset, delete everything
* 2) if there is an entry for this exact offset, delete it and
everything larger than it
* 3) if there is no entry for this offset, delete everything larger than
the next smallest
*/
val newEntries =
if(slot < 0)
0
else if(logical(idx, slot) == offset)
slot
else
slot + 1
The value of newEntries is the number of remaining entries after the
truncation. If we truncate to an offset lower than what is contained in the
index (say because there is no entry) we truncate to 0 entries. If we are given
an offset which is in the index, we delete that entry and all greater entries.
Otherwise if we find an entry smaller than the given entry we should not delete
that but instead delete all entries larger than it.
For example in OffsetIndexTest.truncate:
idx.truncateTo(9)
assertEquals("Index should truncate off last entry", OffsetPosition(8, 8),
idx.lookup(10))
This is asserting that in an index with an entry for every offset, if we
truncate to 9, then the largest entry that remains will be 8.
I think this is right, what was the problem you saw?
> Log.truncateTo() may need to trucate immutable log segment
> ----------------------------------------------------------
>
> Key: KAFKA-551
> URL: https://issues.apache.org/jira/browse/KAFKA-551
> Project: Kafka
> Issue Type: Bug
> Components: core
> Affects Versions: 0.8
> Reporter: Jun Rao
> Priority: Blocker
> Labels: bugs
> Attachments: KAFKA-551.patch
>
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> In makeFollower, we need to first truncate the local log to high watermark.
> It's possible that we need to truncate into segments before the last one. The
> problem is that all segments except the last one are immutable. So the
> truncation will fail which prevents the replica fetcher from being started.
> One solution is to reopen the segment as mutable during truncation, if it's
> not mutable already.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
