Thanks. Yes I get that it's bytes. Good to know about the new setting. On Sun, Apr 24, 2016 at 10:19 AM, Jens Rantil <jens.ran...@tink.se> wrote:
> Hi Richard, > > > which defaults to a very large large number, will affect the number of > records returned by each call to poll() > > No, it will affect the total sum of the message sizes fetched. This is not > the same as "number of messages". The upcoming release of 9.1 (not out yet) > will contain a setting that allows you to set a cap on the maximum number > of messages that poll() returns. See also > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-41%3A+KafkaConsumer+Max+Records > . > > Cheers, > Jens > > On Sat, Apr 23, 2016 at 2:20 AM Richard Rodseth <rrods...@gmail.com> > wrote: > > > To answer my own question (partially), I have learned that > > > > max.partition.fetch.bytes > > > > , which defaults to a very large large number, will affect the number of > > records returned by each call to poll() > > > > I also learned that seekToBeginning is a partition-level thing, but > > > > props.put("auto.offset.reset","earliest") > > has the desired effect. > > > > On Fri, Apr 22, 2016 at 11:08 AM, Richard Rodseth <rrods...@gmail.com> > > wrote: > > > > > Do I understand correctly that poll() will return a subset of the > > messages > > > in a topic each time it is called? So if I want to replay all > messages, I > > > would seek to the beginning and call poll in a loop? Not easily knowing > > > when I was done, without a high watermark > > > > > > https://issues.apache.org/jira/browse/KAFKA-2076 > > > > > > This is a pretty basic question, but I don't think it is explained in > the > > > JavaDoc > > > > > > > > > > > > http://kafka.apache.org/090/javadoc/index.html?org/apache/kafka/clients/consumer/KafkaConsumer.html > > > > > > Thanks > > > > > > -- > > Jens Rantil > Backend Developer @ Tink > > Tink AB, Wallingatan 5, 111 60 Stockholm, Sweden > For urgent matters you can reach me at +46-708-84 18 32. >
