Hi,
I have created an Ubuntu image docker on a physical host running RHES75.
I installed both Zookeeper and Kafka binaries in this container and they
both started OK, I ran my Zookeeper on port 3400 and Kafka running on port
12092
hduser@bbe7e47d5609: /home/hduser> jps
7026 QuorumPeerMain
7079 Kafka
I can connect from contain to any host as long /etc/hosts has the relevant
host information.
I mapped port 3400 and 12092 on the physical host running the container as
port forwarding.
[root@rhes75 ~]# docker ps -a
CONTAINER ID IMAGE COMMAND
CREATED STATUS PORTS NAMES
bbe7e47d5609 ubuntu "bash" 28 hours
ago Up 25 hours testdocker
I did the mapping of ports as follows:
iptables -t nat -A DOCKER -p tcp --dport 4300 -j DNAT --to-destination
172.17.0.2:4300
iptables -t nat -A DOCKER -p tcp --dport 12092 -j DNAT --to-destination
172.17.0.2:12092
So if I go to another host and try to telnet to port 4300 on the docker it
works
hduser@rhes564: /home/hduser/dba/log> telnet rhes75 4300
Trying 50.140.197.220...
Connected to rhes75 (50.140.197.220).
Escape character is '^]'.
hduser@rhes564: /home/hduser/dba/log> telnet rhes75 12092
Trying 50.140.197.220...
Connected to rhes75 (50.140.197.220).
Escape character is '^]'.
If I stop Zookeeper (running on 4300), the connection will be refused
I created a topic called "test" on a remote host as follows
${KAFKA_HOME}/bin/kafka-topics.sh --create --zookeeper rhes75:4300
--replication-factor 1 --partitions 1 --topic test
And if I go into docker I can see that topic on localhost:4300
${KAFKA_HOME}/bin/kafka-topics.sh --describe --zookeeper *localhost:4300*
--topic test
Topic:test PartitionCount:1 ReplicationFactor:1 Configs:
Topic: test Partition: 0 Leader: 1 Replicas: 1
Isr: 1
The problem I see is that I cannot publish messages to the broker on this
container
>From remote host I start publishing some test prices
cat ${IN_FILE} | ${KAFKA_HOME}/bin/kafka-console-producer.sh --broker-list
rhes75:12092 --topic test --property "parse.key=true" --property
"key.separator=:"
I get this error that indicates there is an issue talking to broker on the
container!
[2018-07-14 19:49:22,838] WARN [Producer clientId=console-producer]
Connection to node 1 could not be established. Broker may not be available.
(org.apache.kafka.clients.NetworkClient)
[2018-07-14 19:49:22,925] WARN [Producer clientId=console-producer]
Connection to node 1 could not be established. Broker may not be available.
(org.apache.kafka.clients.NetworkClient)
[2018-07-14 19:49:23,088] WARN [Producer clientId=console-producer]
Connection to node 1 could not be established. Broker may not be available.
(org.apache.kafka.clients.NetworkClient)
[2018-07-14 19:49:23,491] WARN [Producer clientId=console-producer]
Connection to node 1 could not be established. Broker may not be available.
(org.apache.kafka.clients.NetworkClient)
[2018-07-14 19:49:24,188] WARN [Producer clientId=console-producer]
Connection to node 1 could not be established. Broker may not be available.
(org.apache.kafka.clients.NetworkClient)
[2018-07-14 19:49:25,124] WARN [Producer clientId=console-producer]
Connection to node 1 could not be established. Broker may not be available.
(org.apache.kafka.clients.NetworkClient)
[2018-07-14 19:49:25,326] ERROR Error when sending message to topic test
with key: 54 bytes, value: 12 bytes with error:
(org.apache.kafka.clients.producer.internals.ErrorLoggingCallback)
org.apache.kafka.common.errors.TimeoutException: Expiring 10 record(s) for
test-0: 1542 ms has passed since batch creation plus linger time
[2018-07-14 19:49:25,329] ERROR Error when sending message to topic test
with key: 54 bytes, value: 12 bytes with error:
(org.apache.kafka.clients.producer.internals.ErrorLoggingCallback)
org.apache.kafka.common.errors.TimeoutException: Expiring 10 record(s) for
test-0: 1542 ms has passed since batch creation plus linger time
[2018-07-14 19:49:25,329] ERROR Error when sending message to topic test
with key: 55 bytes, value: 11 bytes with error:
(org.apache.kafka.clients.producer.internals.ErrorLoggingCallback)
org.apache.kafka.common.errors.TimeoutException: Expiring 10 record(s) for
test-0: 1542 ms has passed since batch creation plus linger time
I have been working on this for a day or two and have not found any
resolution yet!
Thanks
Dr Mich Talebzadeh
LinkedIn *
https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
<https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>*
http://talebzadehmich.wordpress.com
*Disclaimer:* Use it at your own risk. Any and all responsibility for any
loss, damage or destruction of data or any other property which may arise
from relying on this email's technical content is explicitly disclaimed.
The author will in no case be liable for any monetary damages arising from
such loss, damage or destruction.
