[
https://issues.apache.org/jira/browse/KAFKA-1744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14201374#comment-14201374
]
Evan Huus commented on KAFKA-1744:
----------------------------------
[~nehanarkhede] this was discovered in the golang consumer I maintain - the
scala consumer (as I linked) seems to handle this case already. I have not
checked the java consumer.
The [spec for the fetch
API|https://cwiki.apache.org/confluence/display/KAFKA/A+Guide+To+The+Kafka+Protocol#AGuideToTheKafkaProtocol-FetchAPI]
implies (though it does not explicitly state) that if I perform a fetch
request for offset X, the fetch response will contain messages whose offset is
strictly >= X. If this is not true (in practice I have seen messages with
offsets < X) I would suggest explicitly noting this in the spec to avoid
confusion.
Alternatively it may be a real bug in the broker, in which case the spec is
fine and the broker should be fixed. I don't have enough information to say for
sure.
> Fetch Response contains messages prior to the requested offset
> --------------------------------------------------------------
>
> Key: KAFKA-1744
> URL: https://issues.apache.org/jira/browse/KAFKA-1744
> Project: Kafka
> Issue Type: Bug
> Components: core
> Affects Versions: 0.8.1.1
> Reporter: Evan Huus
>
> As reported in https://github.com/Shopify/sarama/issues/166 there are cases
> where a FetchRequest for a particular offset returns some messages prior to
> that offset.
> The spec does not seem to indicate that this is possible; it does state that
> "As an optimization the server is allowed to return a partial message at the
> end of the message set." but otherwise implies that a request for offset X
> will only return complete messages starting at X.
> The scala consumer does seem to handle this case gracefully though, if I am
> reading it correctly (my scala is not the best):
> https://github.com/apache/kafka/blob/0.8.1/core/src/main/scala/kafka/consumer/ConsumerIterator.scala#L96-L99
> So is this a bug or just a case that needs to be added to the spec? Something
> like "As an optimization the server is allowed to return some messages in the
> message set prior to the requested offset. Clients should handle this case."?
> Although I can't imagine why sending extra data would be faster than only
> sending the necessary messages...
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
