Re: ConsumerOffsets uneven partition distribution

Hi Andras, Thanks for that information, handcraft group.id to make sure it spread across broker is a way, I will give that a go. I understand the benefit of consumer group, my concern at the moment is the potential to create a hot spot on one or the broker... Thanks, Johnny Luo On

RE: Kafka consumer issue

So the documentation says "advertised.host.name" and "advertised.port" are used only when "listeners" or "advertised.listeners" are not set. Sorry I missed that you had already set them. Though, they are deprecated and it is recommended to use "advertised.listeners" instead. I just tested this

RE: Kafka consumer issue

Getting below repeated warnings and messages are not consumed -: [2018-03-20 14:09:56,787] WARN [Consumer clientId=consumer-1, groupId=console-consumer-35712] Connection to node -1 could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient) -Original

RE: Kafka consumer issue

Yes, without the spaces: listeners=PLAINTEXT://kfk03.mp.com:9092 From: "Anand, Uttam" To: "users@kafka.apache.org" Date: 03/20/2018 11:50 AM Subject:RE: Kafka consumer issue You meant listeners=PLAINTEXT:// kfk03.mp.com:9092

RE: Kafka consumer issue

You meant listeners=PLAINTEXT:// kfk03.mp.com:9092 -Original Message- From: Vahid S Hashemian [mailto:vahidhashem...@us.ibm.com] Sent: Tuesday, March 20, 2018 1:45 PM To: users@kafka.apache.org Subject: RE: Kafka consumer issue EXTERNAL EMAIL Thanks. Have you tried setting the

RE: Kafka consumer issue

Thanks. Have you tried setting the listeners property to the actual domain name of the broker? That might be the issue. --Vahid From: "Anand, Uttam" To: "users@kafka.apache.org" Date: 03/20/2018 11:34 AM Subject:RE: Kafka

RE: Kafka consumer issue

Server.properties broker.id=0 listeners=PLAINTEXT://:9092 port=9092 host.name=kfk03.mp.com advertised.host.name=kfk03.mp.com advertised.port=9092 num.network.threads=3 num.io.threads=8 socket.send.buffer.bytes=102400 socket.receive.buffer.bytes=102400 socket.request.max.bytes=104857600

RE: Kafka consumer issue

Could you paste the server.properties content? It wasn't attached to the original note. Thanks. --Vahid From: "Anand, Uttam" To: "users@kafka.apache.org" Date: 03/20/2018 11:10 AM Subject:RE: Kakfa consumer issue Yes checked

RE: Kakfa consumer issue

Yes checked everything. No errors. Replication factor is 1 only. -Original Message- From: Manikumar [mailto:manikumar.re...@gmail.com] Sent: Tuesday, March 20, 2018 12:59 PM To: users@kafka.apache.org Subject: Re: Kakfa consumer issue EXTERNAL EMAIL check broker logs for any errors.

Re: Kakfa consumer issue

check broker logs for any errors. also enable consumer debug logs. check the health of the __consumer_offsets topic. make sure to set offsets.topic.replication.factor=1 for single node cluster. On Tue, Mar 20, 2018 at 11:21 PM, Anand, Uttam wrote: > I don’t want to use

RE: Kakfa consumer issue

I don’t want to use --new-consumer as it is the default, so this option is deprecated and will be removed in a future release. -Original Message- From: Anand, Uttam Sent: Tuesday, March 20, 2018 12:43 PM To: 'users@kafka.apache.org' Subject: RE: Kakfa consumer

RE: Kakfa consumer issue

You mean by executing below command ? /kafka/bin/kafka-console-consumer.sh --new-consumer --bootstrap-server kfk03.mp.com:2181 --topic test --from-beginning -Original Message- From: Zakee [mailto:kzak...@netzero.net] Sent: Tuesday, March 20, 2018 12:35 PM To: users@kafka.apache.org

Re: Kakfa consumer issue

Did you try with --new-consumer ? -Zakee > On Mar 20, 2018, at 10:26 AM, Anand, Uttam wrote: > > I am facing an issue while consuming message using the bootstrap-server i.e. > Kafka server. Any idea why is it not able to consume messages without > zookeeper? > >

Kakfa consumer issue

I am facing an issue while consuming message using the bootstrap-server i.e. Kafka server. Any idea why is it not able to consume messages without zookeeper? Kafka Version -: kafka_2.11-1.0.0 Zookeeper Version -: kafka_2.11-1.0.0 Server.properties -: Attached Zookeeper Host and port -:

Re: Two instances of Kafka consumer reading same partition

Hi Ted, Attached the server side logs during this time. Could not retrieve the consumer side logs as it has hit the retention. Thanks, Narayan On Mon, Mar 19, 2018 at 9:13 PM, Ted Yu wrote: > Can you attach broker and consumer logs to the JIRA ? > > Thanks > > On Mon, Mar

Re: ConsumerOffsets uneven partition distribution

Hi Johnny, As you already mentioned, it depends on the group.id which broker will be the group leader. You can change the group.id to modify which _consumer_offsets partition the group will belong to, thus change which broker will manage a group. You can check which partition a group.id is

RE: Issue deleting topic on Windows

Sorry, I forgot to mention we're using version: kafka_2.12-1.0.0 -Original Message- From: Young, Ben [mailto:ben.yo...@fisglobal.com] Sent: 20 March 2018 09:37 To: users@kafka.apache.org Subject: Issue deleting topic on Windows Hi, We have an issue in our dev environments when running

Issue deleting topic on Windows

Hi, We have an issue in our dev environments when running on Windows. To provide a clean environment for testing we tear down the topics and recreate them, but deleting the topics is causing the server to crash. We then need to delete the logs directory for it to be able to restart, it doesn't