Hi Team , Any help here Pls ? Cheers, Senthil
On Sat, May 27, 2017 at 8:25 PM, SenthilKumar K <senthilec...@gmail.com> wrote: > Hello Kafka Developers , Users , > > We are exploring the SearchMessageByTimestamp feature in Kafka for our > use case . > > Use Case : Kafka will be realtime message bus , users should be able > to pull Logs by specifying start_date and end_date or Pull me last five > minutes data etc ... > > I did POC on SearchMessageByTimestamp , here is the code > https://gist.github.com/senthilec566/16e8e28b32834666fea132afc3a4e2f9 . > And i observed that Searching Messages is slow .. > > Here is small test i did : > Query :Fetch Logs of Last *5 minutes*: > Result: > No of Records fetched : *30* > Fetch Time *6210* ms > > Above test performed in a topic which has 4 partitions. In each partition > search & query processing happened .. in other words > consumer.offsetsForTimes() > consumer.assign(Arrays.asList(partition)) > consumer.seek(this.partition, offsetTimestamp.offset()) > consumer.poll(100) > > are the API calls of each partition.. I realized that , this was the > reason for Kafka taking more time.. > > What is efficient way of implementing SerachMessageByTimeStamp ? Is Kafka > right candidate for our Use Case ? > > Pls add your thoughts here ... > > > Cheers, > Senthil >
