The broker maintains a timestamp index and uses this timestamp index to
answer the "offsetForTimes" request.

The returned offset guarantees, that there is no record with a smaller
timestamp and smaller offset in the topic. Thus, if there are
out-of-order records in the topic, and you start reading from the
returned offset, you might see record with _smaller_ timestamps than the
specified in offsetForTimes().

Ie, it is guaranteed, that you receive _all_ records that have a
timestamp equal or larger than the specified one (however, it is not
guaranteed that you get _only_ record with equal or larger timestamps --
there might be later records with smaller timestamps, too).


-Matthias



On 2/19/18 10:00 AM, Steve Jang wrote:
> If you set *message.timestamp.type* (or *log.message.timestamp.type*) to be
> LogAppendTime, this would make sense.
> 
> I am new to Kafka, too, and if this was set to CreateTime, I don't know
> what the behavior would be.  There is *message.timestamp.difference.max.ms
> <http://message.timestamp.difference.max.ms>* setting too, so there seem to
> be certain "boundedness" of how much clock skew is allowed between the
> producer and the broker, so you could implement various types of policies
> (min, max, etc) for this API.
> 
> 
> On Mon, Feb 19, 2018 at 7:36 AM, Xavier Noria <f...@hashref.com> wrote:
> 
>> In the mental model I am building of how Kafka works (new to this), the
>> broker keeps offsets by consumer group, and individual consumers basically
>> depend on the offset of the consumer group they join. Also consumer groups
>> may opt to start from the beginning.
>>
>> OK, in that mental model there is a linearization of messages per
>> partition. As the documentation says, there is a total order per partition,
>> and the order is based on the offset, unrelated to the timestamp.
>>
>> But I see the Java library has timestamp-oriented methods like:
>>
>>
>> https://kafka.apache.org/0102/javadoc/org/apache/kafka/
>> clients/consumer/Consumer.html#offsetsForTimes(java.util.Map)
>>
>> How does that make sense given the model described above? How is that
>> implemented? Does the broker has builtin support for this? What happens if
>> due to race conditions or machines with clocks out of sync you have
>> messages with timestamps interleaved?
>>
>> Could anyone concile that API with the intrinsec offset-based contract?
>>
> 
> 
> 

Attachment: signature.asc
Description: OpenPGP digital signature

Reply via email to