btw. I got a blog post where I show how I work around the blocking hasNext() thing. May be helpful: http://ingest.tips/2014/10/12/kafka-high-level-consumer-frequently-missing-pieces/
On Thu, Oct 16, 2014 at 12:52 PM, Neha Narkhede <neha.narkh...@gmail.com> wrote: > Josh, > > The consumer's API doesn't allow you to specify N messages, but you can > invoke iter.next() as Gwen suggested and count the messages. Note that the > iterator can block if you have less than N messages so you will have to > careful design around it. The new consumer's API provides a non blocking > poll() API so this sort of use case is better handled. In any case, getting > messages based on a count is something that has to happen on the consumer > side since the server sends the bytes using the sendfile API that doesn't > allow it to inspect the bytes. > > Thanks, > Neha > > On Thu, Oct 16, 2014 at 8:37 AM, <gshap...@cloudera.com> wrote: > >> Using the high level consumer, each consumer in the group can call >> iter.next () in a loop until they get the number of messages you need. >> >> — >> Sent from Mailbox >> >> On Thu, Oct 16, 2014 at 10:18 AM, Josh J <joshjd...@gmail.com> wrote: >> >> > hi, >> > How do I read N items from a topic? I also would like to do this for a >> > consumer group, so that each consumer can specify an N number of tuples >> to >> > read, and each consumer reads distinct tuples. >> > Thanks, >> > Josh >>
