a shock when a zk script goes fubar
zookeeper devs can we get some help for sh zkServer.sh status?
Thanks!
Martin
______________________________________________
________________________________
From: Zac Harvey <zac.har...@welltok.com>
Sent: Wednesday, November 30, 2016 10:25 AM
To: us...@kafka.apache.org
Subject: Re: Messages intermittently get lost
Hi Martin, makes sense.
When I SSH into all 3 of my ZK nodes and run:
sh zkServer.sh status
All three of them give me the following output:
JMX enabled by default
zkServer.sh: 81: /opt/zookeeper/bin/zkEnv.sh: Syntax error: "(" unexpected
(expecting "fi")
Looks like a bug in the ZK shell script?
Best,
Zac
________________________________
From: Martin Gainty <mgai...@hotmail.com>
Sent: Tuesday, November 29, 2016 11:18:21 AM
To: us...@kafka.apache.org
Subject: Re: Messages intermittently get lost
Hi Zach
we dont know whats causing this intermittent problem..so lets Divide and
Conquer each part of this problem individually starting at the source of the
data feeds
Let us eliminate any potential problem with feeds from external sources
Once you verify the zookeeper feeds are 100% reliable lets move onto Kafka
Pingback when you have verifiable results from zookeeper feeds
Thanks
Martin
______________________________________________
________________________________
From: Zac Harvey <zac.har...@welltok.com>
Sent: Tuesday, November 29, 2016 10:46 AM
To: us...@kafka.apache.org
Subject: Re: Messages intermittently get lost
Does anybody have any idea why ZK might be to blame if messages sent by a Kafka
producer fail to be received by a Kafka consumer?
________________________________
From: Zac Harvey <zac.har...@welltok.com>
Sent: Monday, November 28, 2016 9:07:41 AM
To: us...@kafka.apache.org
Subject: Re: Messages intermittently get lost
Thanks Martin, I will look at those links.
But you seem to be 100% confident that the problem is with ZooKeeper...can I
ask why? What is it about my problem description that makes you think this is
an issue with ZooKeeper?
________________________________
From: Martin Gainty <mgai...@hotmail.com>
Sent: Friday, November 25, 2016 1:46:28 PM
To: us...@kafka.apache.org
Subject: Re: Messages intermittently get lost
________________________________
From: Zac Harvey <zac.har...@welltok.com>
Sent: Friday, November 25, 2016 6:17 AM
To: us...@kafka.apache.org
Subject: Re: Messages intermittently get lost
Hi Martin,
My server.properties looks like this:
listeners=PLAINTEXT://0.0.0.0:9092
advertised.host.name=<SERVER_PUBLIC_DNS>
broker.id=2
port=9092
num.partitions=4
zookeeper.connect=zkA:2181,zkB:2181,zkC:2181
MG>can you check status for each ZK Node in the quorum?
sh>$ZOOKEEPER_HOME/bin/zkServer.sh status
http://www.ibm.com/support/knowledgecenter/SSCRJU_4.0.0/com.ibm.streams.pd.doc/doc/containerstreamszookeeper.html
ZooKeeper problems and solutions -
IBM<http://www.ibm.com/support/knowledgecenter/SSCRJU_4.0.0/com.ibm.streams.pd.doc/doc/containerstreamszookeeper.html>
www.ibm.com
Use these solutions to resolve the problems that you might encounter with
Apache ZooKeeper.
ZooKeeper problems and solutions -
IBM<http://www.ibm.com/support/knowledgecenter/SSCRJU_4.0.0/com.ibm.streams.pd.doc/doc/containerstreamszookeeper.html>
ZooKeeper problems and solutions -
IBM<http://www.ibm.com/support/knowledgecenter/SSCRJU_4.0.0/com.ibm.streams.pd.doc/doc/containerstreamszookeeper.html>
www.ibm.com
Use these solutions to resolve the problems that you might encounter with
Apache ZooKeeper.
www.ibm.com<http://www.ibm.com>
[http://upload.wikimedia.org/wikipedia/commons/thumb/5/51/IBM_logo.svg/200px-IBM_logo.svg.png]<http://www.ibm.com/>
IBM - United States<http://www.ibm.com/>
www.ibm.com
For more than a century IBM has been dedicated to every client's success and to
creating innovations that matter for the world
Use these solutions to resolve the problems that you might encounter with
Apache ZooKeeper.
ZooKeeper problems and solutions -
IBM<http://www.ibm.com/support/knowledgecenter/SSCRJU_4.0.0/com.ibm.streams.pd.doc/doc/containerstreamszookeeper.html>
ZooKeeper problems and solutions -
IBM<http://www.ibm.com/support/knowledgecenter/SSCRJU_4.0.0/com.ibm.streams.pd.doc/doc/containerstreamszookeeper.html>
www.ibm.com
Use these solutions to resolve the problems that you might encounter with
Apache ZooKeeper.
ZooKeeper problems and solutions -
IBM<http://www.ibm.com/support/knowledgecenter/SSCRJU_4.0.0/com.ibm.streams.pd.doc/doc/containerstreamszookeeper.html>
ZooKeeper problems and solutions -
IBM<http://www.ibm.com/support/knowledgecenter/SSCRJU_4.0.0/com.ibm.streams.pd.doc/doc/containerstreamszookeeper.html>
www.ibm.com
Use these solutions to resolve the problems that you might encounter with
Apache ZooKeeper.
www.ibm.com<http://www.ibm.com>
[http://upload.wikimedia.org/wikipedia/commons/thumb/5/51/IBM_logo.svg/200px-IBM_logo.svg.png]<http://www.ibm.com/>
IBM - United States<http://www.ibm.com/>
www.ibm.com
For more than a century IBM has been dedicated to every client's success and to
creating innovations that matter for the world
Use these solutions to resolve the problems that you might encounter with
Apache ZooKeeper.
www.ibm.com<http://www.ibm.com>
[http://upload.wikimedia.org/wikipedia/commons/thumb/5/51/IBM_logo.svg/200px-IBM_logo.svg.png]<http://www.ibm.com/>
IBM - United States<http://www.ibm.com/>
www.ibm.com
For more than a century IBM has been dedicated to every client's success and to
creating innovations that matter for the world
[http://upload.wikimedia.org/wikipedia/commons/thumb/5/51/IBM_logo.svg/200px-IBM_logo.svg.png]<http://www.ibm.com/>
IBM - United States<http://www.ibm.com/>
[http://upload.wikimedia.org/wikipedia/commons/thumb/5/51/IBM_logo.svg/200px-IBM_logo.svg.png]<http://www.ibm.com/>
IBM - United States<http://www.ibm.com/>
www.ibm.com
For more than a century IBM has been dedicated to every client's success and to
creating innovations that matter for the world
www.ibm.com<http://www.ibm.com>
[http://upload.wikimedia.org/wikipedia/commons/thumb/5/51/IBM_logo.svg/200px-IBM_logo.svg.png]<http://www.ibm.com/>
IBM - United States<http://www.ibm.com/>
www.ibm.com
For more than a century IBM has been dedicated to every client's success and to
creating innovations that matter for the world
For more than a century IBM has been dedicated to every client's success and to
creating innovations that matter for the world
Use these solutions to resolve the problems that you might encounter with
Apache ZooKeeper.
MG>*greetings from Plimoth Mass*
MG>M
num.network.threads=3
num.io.threads=8
socket.send.buffer.bytes=102400
socket.receive.buffer.bytes=102400
log.dirs=/tmp/kafka-logs
num.recovery.threads.per.data.dir=1
log.retention.hours=168
log.segment.bytes=1073741824
log.retention.check.interval.ms=300000
zookeeper.connection.timeout.ms=6000
delete.topic.enable=true
auto.leader.rebalance.enable=true
Above, 'zkA', 'zkB' and 'zkC' are defined in /etc/hosts and are valid ZK
servers, and <SERVER_PUBLIC_DNS> is the public DNS of the EC2 (AWS) node that
this Kafka is running on.
Anything look incorrect to you?
And yes, yesterday was a holiday, but there is only work! I'll celebrate one
big, long holiday when I'm dead!
Thanks for any-and-all input here!
Best,
Zac
________________________________
From: Martin Gainty <mgai...@hotmail.com>
Sent: Thursday, November 24, 2016 9:03:33 AM
To: us...@kafka.apache.org
Subject: Re: Messages intermittently get lost
Hi Zach
there is a rumour that today thursday is a holiday?
in server.properties how are you configuring your server?
specifically what are these attributes?
num.network.threads=
num.io.threads=
socket.send.buffer.bytes=
socket.receive.buffer.bytes=
socket.request.max.bytes=
num.partitions=
num.recovery.threads.per.data.dir=
?
Martin
______________________________________________
________________________________
From: Zac Harvey <zac.har...@welltok.com>
Sent: Thursday, November 24, 2016 7:05 AM
To: us...@kafka.apache.org
Subject: Re: Messages intermittently get lost
Anybody?!? This is very disconcerting!
________________________________
From: Zac Harvey <zac.har...@welltok.com>
Sent: Wednesday, November 23, 2016 5:07:45 AM
To: us...@kafka.apache.org
Subject: Messages intermittently get lost
I am playing around with Kafka and have a simple setup:
* 1-node Kafka (Ubuntu) server
* 3-node ZK cluster (each on their own Ubuntu server)
I have a consumer written in Scala and am using the kafka-console-producer
(v0.10) that ships with the distribution.
I'd say about 20% of the messages I send via the producer never get consumed by
the Scala process (which is running continuously). No errors on either side
(producer or consumer): the producer sends, and, nothing...
Any ideas as to what might be going on here, or how I could start
troubleshooting?
Thanks!
