Hi Richard, > which defaults to a very large large number, will affect the number of records returned by each call to poll()
No, it will affect the total sum of the message sizes fetched. This is not the same as "number of messages". The upcoming release of 9.1 (not out yet) will contain a setting that allows you to set a cap on the maximum number of messages that poll() returns. See also https://cwiki.apache.org/confluence/display/KAFKA/KIP-41%3A+KafkaConsumer+Max+Records . Cheers, Jens On Sat, Apr 23, 2016 at 2:20 AM Richard Rodseth <rrods...@gmail.com> wrote: > To answer my own question (partially), I have learned that > > max.partition.fetch.bytes > > , which defaults to a very large large number, will affect the number of > records returned by each call to poll() > > I also learned that seekToBeginning is a partition-level thing, but > > props.put("auto.offset.reset","earliest") > has the desired effect. > > On Fri, Apr 22, 2016 at 11:08 AM, Richard Rodseth <rrods...@gmail.com> > wrote: > > > Do I understand correctly that poll() will return a subset of the > messages > > in a topic each time it is called? So if I want to replay all messages, I > > would seek to the beginning and call poll in a loop? Not easily knowing > > when I was done, without a high watermark > > > > https://issues.apache.org/jira/browse/KAFKA-2076 > > > > This is a pretty basic question, but I don't think it is explained in the > > JavaDoc > > > > > > > http://kafka.apache.org/090/javadoc/index.html?org/apache/kafka/clients/consumer/KafkaConsumer.html > > > > Thanks > > > -- Jens Rantil Backend Developer @ Tink Tink AB, Wallingatan 5, 111 60 Stockholm, Sweden For urgent matters you can reach me at +46-708-84 18 32.
