[
https://issues.apache.org/jira/browse/KAFKA-748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13574067#comment-13574067
]
Jay Kreps commented on KAFKA-748:
---------------------------------
Okay the problem here is the resize method. We generalized index.trimInvalid to
index.resize to be able to enlarge the index when we load a segment. This was
done to avoid rolling the log on a full index, I think, which had other
problems with empty indexes. However it looks like this never actually was
tried, because doing this resets the position in the mmap to 0, so we start
overwriting the entries in the index from the beginning but expanding what we
think the valid segment of the memory map is. When we close we then have a
bunch of zeros at the end of the index and have overwritten the front of the
file.
> Append to index fails due to invalid offset
> -------------------------------------------
>
> Key: KAFKA-748
> URL: https://issues.apache.org/jira/browse/KAFKA-748
> Project: Kafka
> Issue Type: Bug
> Affects Versions: 0.8
> Reporter: Sriram Subramanian
> Assignee: Jay Kreps
> Priority: Blocker
> Labels: p1
> Fix For: 0.8
>
> Attachments: outindex, outmsg
>
>
> We seem to be appending to the index and it checks to make sure we do not
> insert an entry with an offset that is larger than the actual offset of the
> message. We seem to be trying to insert an offset = 1 in the index while
> lastOffset is 24463. This seems to get fixed on restarting the broker.
> java.lang.IllegalArgumentException: Attempt to append an offset (1) to
> position 21703 no larger than the last offset appended (24463).
> at kafka.log.OffsetIndex.append(OffsetIndex.scala:183)
> at kafka.log.LogSegment.append(LogSegment.scala:60)
> at kafka.log.Log.append(Log.scala:286)
> at
> kafka.server.KafkaApis$$anonfun$appendToLocalLog$2.apply(KafkaApis.scala:188)
> at
> kafka.server.KafkaApis$$anonfun$appendToLocalLog$2.apply(KafkaApis.scala:181)
> at
> scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:206)
> at
> scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:206)
> at scala.collection.immutable.Map$Map1.foreach(Map.scala:105)
> at scala.collection.TraversableLike$class.map(TraversableLike.scala:206)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
