[
https://issues.apache.org/jira/browse/KAFKA-588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13487086#comment-13487086
]
Jay Kreps commented on KAFKA-588:
---------------------------------
Okay, I fooled around a little more and figured it out.
Here is the issue. In OffsetIndex.truncateTo we have the following code:
val newEntries =
if(slot < 0)
0
else if(logical(idx, slot) == offset)
slot
else
slot + 1
This code looks right due to the bad naming. The arithmetic is actually
correct, but the comparison is wrong as logical() returns the relative offset
which we compare to the global offset. The fixed code looks like this:
val newEntries =
if(slot < 0)
0
else if(relativeOffset(idx, slot) == offset - baseOffset)
slot
else
slot + 1
> Index truncation doesn't seem to remove the last entry properly
> ---------------------------------------------------------------
>
> Key: KAFKA-588
> URL: https://issues.apache.org/jira/browse/KAFKA-588
> Project: Kafka
> Issue Type: Bug
> Components: core
> Affects Versions: 0.8
> Reporter: Jun Rao
> Assignee: Jay Kreps
> Priority: Blocker
> Labels: bugs
> Attachments: KAFKA-588.patch
>
>
> [2012-10-26 08:04:13,333] INFO [Kafka Log on Broker 3], Truncated log segment
> /tmp/kafka_server_3_logs/test_1-0/00000000000000130500.log to target offset
> 429050 (kafka.log.
> Log)
> [2012-10-26 08:04:13,333] INFO [ReplicaFetcherManager on broker 3] adding
> fetcher on topic test_1, partion 0, initOffset 429050 to broker 2 with
> fetcherId 0 (kafka.server.R
> eplicaFetcherManager)
> [2012-10-26 08:04:13,335] INFO Replica Manager on Broker 3: Handling leader
> and isr request LeaderAndIsrRequest(1,,1000,Map((test_1,1) ->
> PartitionStateInfo({ "ISR":"2,3","leader":"2","leaderEpoch":"2" },3),
> (test_1,0) -> PartitionStateInfo({ "ISR":"2,3","leader":"2","leaderEpoch":"2"
> },3))) (kafka.server.ReplicaManager)
> [2012-10-26 08:04:13,335] INFO Replica Manager on Broker 3: Starting the
> follower state transition to follow leader 2 for topic test_1 partition 1
> (kafka.server.ReplicaManager)
> [2012-10-26 08:04:13,335] INFO Partition [test_1, 1] on broker 3: Current
> leader epoch [2] is larger or equal to the requested leader epoch [2],
> discard the become follower request (kafka.cluster.Partition)
> [2012-10-26 08:04:13,336] INFO Replica Manager on Broker 3: Starting the
> follower state transition to follow leader 2 for topic test_1 partition 0
> (kafka.server.ReplicaManager)
> [2012-10-26 08:04:13,336] INFO Partition [test_1, 0] on broker 3: Current
> leader epoch [2] is larger or equal to the requested leader epoch [2],
> discard the become follower request (kafka.cluster.Partition)
> [2012-10-26 08:04:13,588] ERROR [ReplicaFetcherThread-2-0-on-broker-3], Error
> due to (kafka.server.ReplicaFetcherThread)
> java.lang.IllegalArgumentException: Attempt to append an offset (429050) no
> larger than the last offset appended (429050).
> at kafka.log.OffsetIndex.append(OffsetIndex.scala:180)
> at kafka.log.LogSegment.append(LogSegment.scala:56)
> at kafka.log.Log.append(Log.scala:273)
> at
> kafka.server.ReplicaFetcherThread.processPartitionData(ReplicaFetcherThread.scala:51)
> at
> kafka.server.AbstractFetcherThread$$anonfun$doWork$5.apply(AbstractFetcherThread.scala:116)
> at
> kafka.server.AbstractFetcherThread$$anonfun$doWork$5.apply(AbstractFetcherThread.scala:99)
> at scala.collection.immutable.Map$Map2.foreach(Map.scala:127)
> at
> kafka.server.AbstractFetcherThread.doWork(AbstractFetcherThread.scala:99)
> at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:50)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
