Hard to say why the second consumer does not fetch data. I would try to inspect the logs at DEBUG level to get more insight what going on.
-Matthias On 12/10/19 12:27 AM, Frank Zhou wrote: > "earliest", as we need everything from the start > > On Tue, Dec 10, 2019 at 4:22 PM Matthias J. Sax <[email protected]> > wrote: > >> What is your `auto.offset.reset` policy? >> >> On 12/10/19 12:02 AM, Frank Zhou wrote: >>> Hi, >>> >>> Thanks for your prompt reply. Consumers have different group.id in our >>> case, as we just need all the data in the topic, so every time the >>> application starts, the consumer will be in a new group. For current >>> testing environment, we just one partition for one topic. though we have >>> tried with multiple partitions, and situation happened as well. >>> >>> On Tue, Dec 10, 2019 at 3:56 PM Matthias J. Sax <[email protected]> >>> wrote: >>> >>>> Are both consumers in the same consumer group, ie, use the same ` >> group.id >>>> `? >>>> >>>> If yes, how many partitions does the topic have? If it has only one >>>> partition, the observed behavior is expected, because a single >>>> partitions can only be read by a single consumer instance per consumer >>>> group. The second consumer won't have any assigned partitions to read >> from. >>>> >>>> -Matthias >>>> >>>> On 12/9/19 11:12 PM, Frank Zhou wrote: >>>>> Hi, >>>>> >>>>> I am quite new to Kafka, and I have encountered a weird case during QA >>>>> stage for our application. Now we have 2 consumers consuming same topic >>>>> in kafka cluster. The first started consumer works fine and get closed >>>>> after getting all the messages. After that, the second one started and >>>>> just hang there in the poll(), which we set the Duration to 10 seconds, >>>>> and will quit the loop if no records are fetched. From kafka server >> log, >>>>> I don't see warning or error for the second consumer. >>>>> >>>>> Attached client/server log related to both consumers. I am using 2.3.1 >>>>> client, 2.2.2 kafka servers. Looking forward to you feedback. >>>>> >>>>> >>>>> Best regards, >>>>> -- >>>>> *Frank Zhou* >>>>> R&D, Itiviti >>>>> Java Developer >>>>> D +852 2521 7480 >>>>> [email protected] <mailto:[email protected]> >>>>> >>>>> ______________________________ >>>>> >>>>> itiviti.com <https://www.itiviti.com/> >>>>> >>>>> /The information contained in or attached to this email is strictly >>>>> confidential. If you are not the intended recipient, please notify us >>>>> immediately by telephone and return the message to us./ >>>>> >>>>> /Email communications by definition contain personal information. The >>>>> Itiviti group of companies is subject to European data protection >>>>> regulations. Itiviti’s Privacy Notice is available at www.itiviti.com >>>>> <http://www.itiviti.com/>. Itiviti expects the recipient of this email >>>>> to be compliant with Itiviti’s Privacy Notice and applicable >>>>> regulations. Please advise us immediately at >>>>> [email protected] if you are not compliant with these./ >>>>> >>>>> >>>>> ______________________________ >>>>> >>>>> itiviti.com <https://www.itiviti.com/> >>>>> <https://www.linkedin.com/company/itiviti> ^Follow Itiviti on Linkedin >>>>> <https://www.linkedin.com/company/10438325/> >>>>> >>>>> The information contained in or attached to this email is strictly >>>>> confidential. If you are not the intended recipient, please notify us >>>>> immediately by telephone and return the message to us. Email >>>>> communications by definition contain personal information. The Itiviti >>>>> group of companies is subject to European data protection regulations. >>>>> >>>>> Itiviti’s Privacy Notice is available at www.itiviti.com >>>>> <http://www.itiviti.com/>. Itiviti expects the recipient of this email >>>>> to be compliant with Itiviti’s Privacy Notice and applicable >>>>> regulations. Please advise us immediately at >>>>> [email protected] if you are not compliant with these. >>>>> >>>> >>>> >>> >> >> >
signature.asc
Description: OpenPGP digital signature
