Yes, in Kafka 0.7, the offset is the byte position of the message in the log for the topic partition. In Kafka 0.8, each message is assigned a monotonically increasing, contiguous sequence number per partition, starting with 1. So each message is addressable using this sequence number instead of the byte position.
Also, the offset keeps increasing over the lifetime of a cluster, even if Kafka deletes older log segments. Thanks, Neha On Thursday, November 22, 2012, Paul Garner wrote: > from what I read, the message offset is the byte position of the message in > the log file that Kafka writes to > > the logs are rotated and eventually deleted by Kafka > > ...does this mean the message offset periodically goes back to start at > zero again? or the offset keeps increasing for the life of the cluster as > if it was a single big file back to the beginning of time? >
