Currently Kafka only have a very coarse solution to find offset by time, which is based on the segment last modified time. This interface is only available in simple consumer. You may issue an OffsetRequest to specify a timestamp. The offset returned will be the first offset of segments whose last modification time is earlier than the timestamp you provided.
Thanks, Jiangjie (Becket) Qin On 6/30/15, 1:16 PM, "Adam Dubiel" <dubiel.a...@gmail.com> wrote: >We faced similar problem and ended up with implementing variant of golden >section search, that reads message using simple consumer and checks the >timestamp (timestamps are appended by our producer though, they do not >come >from any Kafka metadata) till it finds message closest to given date. > >Adam > >2015-06-30 21:52 GMT+02:00 Shushant Arora <shushantaror...@gmail.com>: > >> Is it possible using low level consumer to get kafka messages based on >> timestamp, say I want to get all messages from last 5 minutes. I don't >>know >> what were offsets of partitions 5 minutes back. >> >> In low level consumer : when I gave epoch for whichTime , it failed. >> requestInfo.put(topicAndPartition, new >> PartitionOffsetRequestInfo(whichTime, >> 1)); >> >> Is only latest and earliest supported in timestamp,Is there any way to >> filter messages based on timestamp? >> >> Thanks >> Shushant >>
