Hey folks! Recently I found Apache Storm and believed it could be of use to my current use case, but I'm having trouble setting it up on Kubernetes.
I built a custom image based off this Storm image: https://hub.docker.com/_/storm/ that sets up Nimbus, the UI and the Supervisors. Also zookeeper, but not as part of that particular image. However, when submitting a topology to test it, I get the following error: 02:50:33.358 [main] INFO o.a.s.u.NimbusClient - Found leader nimbus : 100.70.85.165:6627 03:03:39.494 [main] WARN o.a.s.u.NimbusClient - Ignoring exception while trying to get leader nimbus info from a64dd9130b80311e990e50a85bf2b2e8-1766957537.sa-east-1.elb.amazonaws.com. will retry with a different seed host. java.lang.RuntimeException: java.lang.RuntimeException: org.apache.storm.thrift.transport.TTransportException: java.net.ConnectException: Connection timed out (Connection timed out) at org.apache.storm.security.auth.ThriftClient.reconnect(ThriftClient.java:110) ~[storm-client-2.0.0.jar:2.0.0] My Nimbus/UI/Supervisor Dockerfile is fairly simple: FROM storm:2.0.0 MAINTAINER Bruno Panuto <[email protected]> ADD start.sh / EXPOSE 3772 6627 WORKDIR /opt/apache-storm ENTRYPOINT ["/start.sh"] and "start.sh" are variations of: #!/bin/sh CONFIG="$STORM_CONF_DIR/storm.yaml" echo $ZOOKEEPER_SERVICE_HOST cat << EOF > "$CONFIG" storm.zookeeper.servers: ["$ZOOKEEPER_SERVICE_HOST"] storm.local.hostname: $NIMBUS_SERVICE_HOST EOF cat $CONFIG exec storm nimbus You can download and explore these images on nubunto/storm-nimbus, nubunto/storm-ui and nubunto/storm-worker. I'll be making it open source as soon as I fix the issues. For K8s configuration, I have a pod for Zookeeper, a pod for Nimbus, a pod for the UI and two pods for supervisors. Zookeeper is available only inside the cluster, and Nimbus gets a LoadBalancer Service so that we can expose it to the outside world. So does the UI. Any ideas on what I could be doing wrong? Or maybe a reference to a working implementation of Storm in Kubernetes? I could see a couple, but apparently none was up-to-date. Thanks in advance, Bruno
