In order to operate Kafka requires zookeeper to store and retrieve some configuration data. So the reason it fails is because there is no zookeeper running on port 2181.
I am not sure how to run 2 OSv instances on same host (one kafka and one zookeeper) due to some networking port conflicts. But you can run zookeeper natively on the host just to test kafka by going to apps/apache-zookeeper/ROOTFS/zookeeper and call ./bin/zkServer.sh start. Then you have to modify apps//apache-kafka/assets/config/server.properties to point to zookeeper. It cannot be localhost:2181 but instead 192.168.122.1:2181. The 192.168.122.1 is what you need to use to make OSv instance running in QEMU/KVM to communicate with apps running on the same host. So change this line: zookeeper.connect=localhost:2181 to zookeeper.connect=192.168.122.1:2181 Waldek PS. It would be nice to upgrade kafka and zookeeper to newer versions as well. But I think it would be event better to upgrade the apps build process to use new mikelamgelo capstan. I will describe my thoughts in a separate email. On Monday, January 15, 2018 at 10:29:15 AM UTC-5, Geraldo Netto wrote: > > Hello Nadav/All, > > Indeed, I'm also facing the same issue and I think the problem is this: > > [2018-01-15 14:27:27,398] INFO Opening socket connection to server > *localhost/127.0.0.1:2181 > <http://127.0.0.1:2181>*. Will not attempt to authenticate using SASL > (unknown error) (org.apache.zookeeper.ClientCnxn) > [2018-01-15 14:27:27,409] ERROR* Unable to open socket to > localhost/127.0.0.1:2181 <http://127.0.0.1:2181>* > (org.apache.zookeeper.ClientCnxnSocketNIO) > > I have the feeling that Kafka is concatenating localhost and 127.0.0.1 as > a single string > Maybe some config. is wrong? > I try to check that later > > In any case, not only Kafka but there are some other issues > dpdk-example looks broken to me > erjang is pointing to a dropbox url to download the jar and the file is > not available anymore > most of the apps are using wget without -c > > > Full log: > OSv v0.24-477-g29fa668 > eth0: 192.168.122.89 > java.so: Starting JVM app using: io/osv/isolated/RunIsolatedJvmApp > java.so: Setting Java system classloader to IsolatingOsvSystemClassLoader > and lo gging manager to IsolatingLogManager > [2018-01-15 14:27:26,928] INFO Verifying properties > (kafka.utils.VerifiableProperties) > [2018-01-15 14:27:27,010] INFO Property broker.id is overridden to 0 > (kafka.utils.VerifiableProperties) > [2018-01-15 14:27:27,013] INFO Property log.cleaner.enable is overridden > to false (kafka.utils.VerifiableProperties) > [2018-01-15 14:27:27,016] INFO Property log.dirs is overridden to > /tmp/kafka-logs (kafka.utils.VerifiableProperties) > [2018-01-15 14:27:27,018] INFO Property log.retention.check.interval.ms > is overridden to 300000 (kafka.utils.VerifiableProperties) > [2018-01-15 14:27:27,021] INFO Property log.retention.hours is overridden > to 168 (kafka.utils.VerifiableProperties) > [2018-01-15 14:27:27,023] INFO Property log.segment.bytes is overridden to > 1073741824 (kafka.utils.VerifiableProperties) > [2018-01-15 14:27:27,026] INFO Property num.io.threads is overridden to 8 > (kafka.utils.VerifiableProperties) > [2018-01-15 14:27:27,028] INFO Property num.network.threads is overridden > to 3 (kafka.utils.VerifiableProperties) > [2018-01-15 14:27:27,030] INFO Property num.partitions is overridden to 1 > (kafka.utils.VerifiableProperties) > [2018-01-15 14:27:27,032] INFO Property num.recovery.threads.per.data.dir > is overridden to 1 (kafka.utils.VerifiableProperties) > [2018-01-15 14:27:27,034] INFO Property port is overridden to 9092 > (kafka.utils.VerifiableProperties) > [2018-01-15 14:27:27,036] INFO Property socket.receive.buffer.bytes is > overridden to 102400 (kafka.utils.VerifiableProperties) > [2018-01-15 14:27:27,039] INFO Property socket.request.max.bytes is > overridden to 104857600 (kafka.utils.VerifiableProperties) > [2018-01-15 14:27:27,041] INFO Property socket.send.buffer.bytes is > overridden to 102400 (kafka.utils.VerifiableProperties) > [2018-01-15 14:27:27,044] INFO Property zookeeper.connect is overridden to > localhost:2181 (kafka.utils.VerifiableProperties) > [2018-01-15 14:27:27,046] INFO Property zookeeper.connection.timeout.ms > is overridden to 6000 (kafka.utils.VerifiableProperties) > [2018-01-15 14:27:27,179] INFO [Kafka Server 0], starting > (kafka.server.KafkaServer) > [2018-01-15 14:27:27,188] INFO [Kafka Server 0], Connecting to zookeeper > on localhost:2181 (kafka.server.KafkaServer) > [2018-01-15 14:27:27,239] INFO Starting ZkClient event thread. > (org.I0Itec.zkclient.ZkEventThread) > [2018-01-15 14:27:27,257] INFO Client > environment:zookeeper.version=3.4.6-1569965, built on 02/20/2014 0 9:09 GMT > (org.apache.zookeeper.ZooKeeper) > [2018-01-15 14:27:27,261] INFO Client environment:host.name=osv > (org.apache.zookeeper.ZooKeeper) > [2018-01-15 14:27:27,265] INFO Client environment:java.version=1.7.0_55 > (org.apache.zookeeper.ZooKeeper) > [2018-01-15 14:27:27,267] INFO Client environment:java.vendor=Oracle > Corporation (org.apache.zookeeper.ZooKeeper) > [2018-01-15 14:27:27,269] INFO Client > environment:java.home=/usr/lib/jvm/java/jre (org.apache.zookeeper.ZooKeeper) > [2018-01-15 14:27:27,271] INFO Client > environment:java.class.path=:/kafka/core/build/dependant-libs-2.10.4*/*.jar:/kafka/contrib/hadoop-consumer/build/libs/kafka-hadoop-consumer*.jar:/kafka/contrib/hadoop-pro > > ducer/build/libs/kafka-hadoop-producer*.jar:/kafka/clients/build/libs/kafka-clients*.jar:/kafka/libs/jop > > t-simple-3.2.jar:/kafka/libs/kafka_2.10-0.8.2.1.jar:/kafka/libs/kafka-clients-0.8.2.1.jar:/kafka/libs/lo > > g4j-1.2.16.jar:/kafka/libs/lz4-1.2.0.jar:/kafka/libs/metrics-core-2.2.0.jar:/kafka/libs/scala-library-2. > > 10.4.jar:/kafka/libs/slf4j-api-1.7.6.jar:/kafka/libs/slf4j-log4j12-1.6.1.jar:/kafka/libs/snappy-java-1.1 > > .1.6.jar:/kafka/libs/zkclient-0.3.jar:/kafka/libs/zookeeper-3.4.6.jar:/kafka/core/build/libs/kafka_2.10* > > .jar (org.apache.zookeeper.ZooKeeper) > [2018-01-15 14:27:27,283] INFO Client > environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib > > (org.apache.zookeeper.ZooKeeper) > [2018-01-15 14:27:27,285] INFO Client environment:java.io.tmpdir=/tmp > (org.apache.zookeeper.ZooKeeper) > [2018-01-15 14:27:27,287] INFO Client environment:java.compiler=<NA> > (org.apache.zookeeper.ZooKeeper) > [2018-01-15 14:27:27,289] INFO Client environment:os.name=Linux > (org.apache.zookeeper.ZooKeeper) > [2018-01-15 14:27:27,291] INFO Client environment:os.arch=amd64 > (org.apache.zookeeper.ZooKeeper) > [2018-01-15 14:27:27,294] INFO Client environment:os.version=3.7.0 > (org.apache.zookeeper.ZooKeeper) > [2018-01-15 14:27:27,296] INFO Client environment:user.name=osv > (org.apache.zookeeper.ZooKeeper) > [2018-01-15 14:27:27,298] INFO Client environment:user.home=/ > (org.apache.zookeeper.ZooKeeper) > [2018-01-15 14:27:27,300] INFO Client environment:user.dir=/kafka > (org.apache.zookeeper.ZooKeeper) > [2018-01-15 14:27:27,304] INFO Initiating client connection, > connectString=localhost:2181 sessionTimeout=6000 > watcher=org.I0Itec.zkclient.ZkClient@6cbc2704 > (org.apache.zookeeper.ZooKeeper) > [2018-01-15 14:27:27,398] INFO Opening socket connection to server > *localhost/127.0.0.1:2181 > <http://127.0.0.1:2181>*. Will not attempt to authenticate using SASL > (unknown error) (org.apache.zookeeper.ClientCnxn) > [2018-01-15 14:27:27,409] ERROR* Unable to open socket to > localhost/127.0.0.1:2181 <http://127.0.0.1:2181>* > (org.apache.zookeeper.ClientCnxnSocketNIO) > [2018-01-15 14:27:27,412] WARN Session 0x0 for server null, unexpected > error, closing socket connection and attempting reconnect > (org.apache.zookeeper.ClientCnxn) > java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) > at sun.nio.ch.Net.connect(Net.java:465) > at sun.nio.ch.Net.connect(Net.java:457) > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) > at > org.apache.zookeeper.ClientCnxnSocketNIO.registerAndConnect(ClientCnxnSocketNIO.java:277) > at > org.apache.zookeeper.ClientCnxnSocketNIO.connect(ClientCnxnSocketNIO.java:287) > at > org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:967) > at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1003) > [2018-01-15 14:27:28,530] INFO Opening socket connection to server > localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL > (unknown error) (org.apache.zookeeper.ClientCnxn) > [2018-01-15 14:27:28,536] ERROR Unable to open socket to localhost/ > 127.0.0.1:2181 (org.apache.zookeeper.ClientCnxnSocketNIO) > [2018-01-15 14:27:28,540] WARN Session 0x0 for server null, unexpected > error, closing socket connection and attempting reconnect > (org.apache.zookeeper.ClientCnxn) > java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) > at sun.nio.ch.Net.connect(Net.java:465) > at sun.nio.ch.Net.connect(Net.java:457) > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) > at > org.apache.zookeeper.ClientCnxnSocketNIO.registerAndConnect(ClientCnxnSocketNIO.java:277) > at > org.apache.zookeeper.ClientCnxnSocketNIO.connect(ClientCnxnSocketNIO.java:287) > at > org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:967) > at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1003) > [2018-01-15 14:27:29,653] INFO Opening socket connection to server > localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL > (unknown error) (org.apache.zookeeper.ClientCnxn) > [2018-01-15 14:27:29,659] ERROR Unable to open socket to localhost/ > 127.0.0.1:2181 (org.apache.zookeeper.ClientCnxnSocketNIO) > [2018-01-15 14:27:29,661] WARN Session 0x0 for server null, unexpected > error, closing socket connection and attempting reconnect > (org.apache.zookeeper.ClientCnxn) > java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) > at sun.nio.ch.Net.connect(Net.java:465) > at sun.nio.ch.Net.connect(Net.java:457) > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) > at > org.apache.zookeeper.ClientCnxnSocketNIO.registerAndConnect(ClientCnxnSocketNIO.java:277) > at > org.apache.zookeeper.ClientCnxnSocketNIO.connect(ClientCnxnSocketNIO.java:287) > at > org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:967) > at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1003) > [2018-01-15 14:27:30,790] INFO Opening socket connection to server > localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL > (unknown error) (org.apache.zookeeper.ClientCnxn) > [2018-01-15 14:27:30,798] WARN Session 0x0 for server null, unexpected > error, closing socket connection and attempting reconnect > (org.apache.zookeeper.ClientCnxn) > java.net.ConnectException: Connection refused > at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) > at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739) > at > org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361) > at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081) > [2018-01-15 14:27:31,912] INFO Opening socket connection to server > localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL > (unknown error) (org.apache.zookeeper.ClientCnxn) > [2018-01-15 14:27:31,918] WARN Session 0x0 for server null, unexpected > error, closing socket connection and attempting reconnect > (org.apache.zookeeper.ClientCnxn) > java.net.ConnectException: Connection refused > at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) > at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739) > at > org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361) > at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081) > [2018-01-15 14:27:33,027] INFO Opening socket connection to server > localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL > (unknown error) (org.apache.zookeeper.ClientCnxn) > [2018-01-15 14:27:33,034] WARN Session 0x0 for server null, unexpected > error, closing socket connection and attempting reconnect > (org.apache.zookeeper.ClientCnxn) > java.net.ConnectException: Connection refused > at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) > at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739) > at > org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361) > at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081) > [2018-01-15 14:27:33,427] INFO Terminate ZkClient event thread. > (org.I0Itec.zkclient.ZkEventThread) > [2018-01-15 14:27:34,145] INFO Opening socket connection to server > localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL > (unknown error) (org.apache.zookeeper.ClientCnxn) > [2018-01-15 14:27:34,258] INFO Session: 0x0 closed > (org.apache.zookeeper.ZooKeeper) > [2018-01-15 14:27:34,263] INFO EventThread shut down > (org.apache.zookeeper.ClientCnxn) > [2018-01-15 14:27:34,269] FATAL [Kafka Server 0], Fatal error during > KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer) > org.I0Itec.zkclient.exception.ZkTimeoutException: Unable to connect to > zookeeper server within timeout: 6000 > at org.I0Itec.zkclient.ZkClient.connect(ZkClient.java:880) > at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:98) > at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:84) > at kafka.server.KafkaServer.initZk(KafkaServer.scala:157) > at kafka.server.KafkaServer.startup(KafkaServer.scala:82) > at > kafka.server.KafkaServerStartable.startup(KafkaServerStartable.scala:29) > at kafka.Kafka$.main(Kafka.scala:46) > at kafka.Kafka.main(Kafka.scala) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at io.osv.Jvm.runMain(Jvm.java:119) > at io.osv.isolated.IsolatedJvm.access$400(IsolatedJvm.java:30) > at io.osv.isolated.IsolatedJvm$3.run(IsolatedJvm.java:116) > [2018-01-15 14:27:34,303] INFO [Kafka Server 0], shutting down > (kafka.server.KafkaServer) > [2018-01-15 14:27:34,319] INFO [Kafka Server 0], shut down completed > (kafka.server.KafkaServer) > [2018-01-15 14:27:34,323] FATAL Fatal error during KafkaServerStartable > startup. Prepare to shutdown (kafka.server.KafkaServerStartable) > org.I0Itec.zkclient.exception.ZkTimeoutException: Unable to connect to > zookeeper server within timeout: 6000 > at org.I0Itec.zkclient.ZkClient.connect(ZkClient.java:880) > at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:98) > at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:84) > at kafka.server.KafkaServer.initZk(KafkaServer.scala:157) > at kafka.server.KafkaServer.startup(KafkaServer.scala:82) > at > kafka.server.KafkaServerStartable.startup(KafkaServerStartable.scala:29) > at kafka.Kafka$.main(Kafka.scala:46) > at kafka.Kafka.main(Kafka.scala) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at io.osv.Jvm.runMain(Jvm.java:119) > at io.osv.isolated.IsolatedJvm.access$400(IsolatedJvm.java:30) > at io.osv.isolated.IsolatedJvm$3.run(IsolatedJvm.java:116) > [2018-01-15 14:27:34,347] INFO [Kafka Server 0], shutting down > (kafka.server.KafkaServer) > > > Kind Regards, > > Geraldo Netto > Sapere Aude => Non dvcor, dvco > http://exdev.sf.net/ > > On 15 January 2018 at 11:32, Nadav Har'El <[email protected] <javascript:> > > wrote: > >> Thanks. I committed this (and going through your other patches as well). >> >> However, if I do >> scripts/build image=apache-kafka >> scripts/run.py >> >> I get all sorts of connection refused errors, and messages like >> org.I0Itec.zkclient.exception.ZkTimeoutException: Unable to connect to >> zookeeper server within timeout: 6000 >> until a few seconds later, the application shuts down. >> >> Were you able to use this package? How? >> >> >> >> -- >> Nadav Har'El >> [email protected] <javascript:> >> >> On Fri, Jan 12, 2018 at 4:31 AM, geraldo netto <[email protected] >> <javascript:>> wrote: >> >>> Signed-off-by: geraldo netto <[email protected] <javascript:>> >>> --- >>> apache-kafka/Makefile | 2 +- >>> 1 file changed, 1 insertion(+), 1 deletion(-) >>> >>> diff --git a/apache-kafka/Makefile b/apache-kafka/Makefile >>> index 63ed524..6fbc98d 100644 >>> --- a/apache-kafka/Makefile >>> +++ b/apache-kafka/Makefile >>> @@ -3,7 +3,7 @@ VERSION:=0.8.2.1 >>> NAME:=kafka >>> DIR:=kafka_$(SCALA)-$(VERSION) >>> TARBALL:=$(DIR).tgz >>> -URL:=http://mirror.ventraip.net.au/apache/$(NAME)/$(VERSION)/$(TARBALL) >>> +URL:=http://archive.apache.org/dist/$(NAME)/$(VERSION)/$(TARBALL) >>> <http://mirror.ventraip.net.au/apache/$(NAME)/$(VERSION)/$(TARBALL)+URL:=http://archive.apache.org/dist/$(NAME)/$(VERSION)/$(TARBALL)> >>> >>> >>> module: ROOTFS usr.manifest >>> -- >>> 2.7.4 >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "OSv Development" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected] <javascript:>. >>> For more options, visit https://groups.google.com/d/optout. >>> >> >> > -- You received this message because you are subscribed to the Google Groups "OSv Development" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
