Dear Stig Rohde, Thank you for fast reply!
In my pom, I added the following lines and now the topology runs as it always did: 60 <dependency> 61 <groupId>org.apache.zookeeper</groupId> 62 <artifactId>zookeeper</artifactId> 63 <version>3.4.6</version> 64 <exclusions> 65 <exclusion> 66 <groupId>org.slf4j</groupId> 67 <artifactId>slf4j-log4j12</artifactId> 68 </exclusion> 69 </exclusions> 70 </dependency> Also, thank you for creating STORM-2978. I'm following the issue now. Thank you and best regards, Shinhyung Yang - On Tue, Feb 27, 2018 at 8:14 AM, Stig Rohde Døssing <s...@apache.org> wrote: > Hi Shinhyung, > > I looked into this, it's a bug. Zookeeper 3.5.3-beta is ending up on the > classpath, which was not the intent. I've raised > https://issues.apache.org/jira/browse/STORM-2978 to track fixing the issue. > For now, you can work around it by adding an explicit dependency on > Zookeeper 3.4.6 to your pom. > > 2018-02-26 17:35 GMT+01:00 Shinhyung Yang <shinhyung.y...@yonsei.ac.kr>: >> >> Dear Storm users and developers, >> >> I have been running the Yahoo streaming benchmarks on Storm 0.9.7. The >> entire Cloud infrastructure includes Zookeeper 3.4.6 on 3 nodes, 5 Kafka >> 2.10-0.8.2.1 nodes, 1 Storm 0.9.7 Nimbus node, 10 Storm 0.9.7 supervisor >> nodes with 4 worker processes on each node, 10 kafka producer nodes, and 1 >> redis 3.0.5 node. >> >> With the introduction of Storm 1.2.0, I decided to upgrade from 0.9.7 to >> 1.2.0. Currently I'm testing Yahoo streaming benchmark's topology on the new >> setup and I end up getting the following exceptions: >> >> # >> streaming-benchmarks/apache-storm-1.2.0/logs/workers-artifacts/test-topo-1-1519628010/6700/worker.log >> >> 148 java.lang.RuntimeException: >> org.apache.zookeeper.KeeperException$UnimplementedException: KeeperErrorCode >> = Unimplemented for >> /ad-events/7183b5b2-4971-41a1-b86d-0788f646bc64/partition_0 >> 149 at org.apache.storm.kafka.ZkState.writeBytes(ZkState.java:84) >> ~[stormjar.jar:?] >> 150 at org.apache.storm.kafka.ZkState.writeJSON(ZkState.java:70) >> ~[stormjar.jar:?] >> 151 at >> org.apache.storm.kafka.PartitionManager.commit(PartitionManager.java:361) >> ~[stormjar.jar:?] >> 152 at >> org.apache.storm.kafka.KafkaSpout.commit(KafkaSpout.java:257) >> ~[stormjar.jar:?] >> 153 at >> org.apache.storm.kafka.KafkaSpout.nextTuple(KafkaSpout.java:156) >> ~[stormjar.jar:?] >> 154 at >> storm.benchmark.AdvertisingTopology$MyKafkaSpout.nextTuple(AdvertisingTopology.java:181) >> ~[stormjar.jar:?] >> 155 at >> org.apache.storm.daemon.executor$fn__9610$fn__9625$fn__9656.invoke(executor.clj:654) >> ~[storm-core-1.2.0.jar:1.2.0] >> 156 at >> org.apache.storm.util$async_loop$fn__553.invoke(util.clj:484) >> [storm-core-1.2.0.jar:1.2.0] >> 157 at clojure.lang.AFn.run(AFn.java:22) [clojure-1.7.0.jar:?] >> 158 at java.lang.Thread.run(Thread.java:745) [?:1.8.0_74] >> 159 Caused by: >> org.apache.zookeeper.KeeperException$UnimplementedException: KeeperErrorCode >> = Unimplemented for >> /ad-events/7183b5b2-4971-41a1-b86d-0788f646bc64/partition_0 >> 160 at >> org.apache.zookeeper.KeeperException.create(KeeperException.java:103) >> ~[stormjar.jar:?] >> 161 at >> org.apache.zookeeper.KeeperException.create(KeeperException.java:51) >> ~[stormjar.jar:?] >> 162 at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:1525) >> ~[stormjar.jar:?] >> 163 at >> org.apache.curator.framework.imps.CreateBuilderImpl$17.call(CreateBuilderImpl.java:1189) >> ~[stormjar.jar:?] >> 164 at >> org.apache.curator.framework.imps.CreateBuilderImpl$17.call(CreateBuilderImpl.java:1166) >> ~[stormjar.jar:?] >> 165 at >> org.apache.curator.connection.StandardConnectionHandlingPolicy.callWithRetry(StandardConnectionHandlingPolicy.java:64) >> ~[stormjar.jar:?] >> 166 at >> org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:100) >> ~[stormjar.jar:?] >> 167 at >> org.apache.curator.framework.imps.CreateBuilderImpl.pathInForeground(CreateBuilderImpl.java:1163) >> ~[stormjar.jar:?] >> 168 at >> org.apache.curator.framework.imps.CreateBuilderImpl.protectedPathInForeground(CreateBuilderImpl.java:603) >> ~[stormjar.jar:?] >> 169 at >> org.apache.curator.framework.imps.CreateBuilderImpl.forPath(CreateBuilderImpl.java:593) >> ~[stormjar.jar:?] >> 170 at >> org.apache.curator.framework.imps.CreateBuilderImpl.forPath(CreateBuilderImpl.java:47) >> ~[stormjar.jar:?] >> 171 at org.apache.storm.kafka.ZkState.writeBytes(ZkState.java:79) >> ~[stormjar.jar:?] >> 172 ... 9 more >> >> For each instance of benchmark execution, more than one of the spout >> instances will eventually generate exception messages in their log files >> like above. >> >> Some people suggest that it's probably the compatibility issue between >> Zookeeper and Curator: >> >> https://stackoverflow.com/a/35753999 >> > Apparently this error can occur if you're using the wrong combination of >> > Curator in combination with Zookeeper. From curator.apache.org : >> >> Curator 2.x.x - compatible with both ZooKeeper 3.4.x and ZooKeeper >> >> 3.5.x >> >> Curator 3.x.x - compatible only with ZooKeeper 3.5.x and includes >> >> support for new features such as dynamic reconfiguration, etc. >> >> http://curator.apache.org >> > Apache Curator is meant to be used with ZooKeeper 3.5+. However, it >> > is also compatible with ZooKeeper 3.4.x. See Compatibility for details. >> >> http://curator.apache.org/zk-compatibility.html >> > While ZooKeeper 3.5.x is still considered "beta" by the ZooKeeper >> > development team, the reality is that it is used in production by many >> > users. >> > However, ZooKeeper 3.4.x is also used in production. Prior to Apache >> > Curator 4.0, >> > both versions of ZooKeeper were supported via two versions of Apache >> > Curator. >> > Starting with Curator 4.0 both versions of ZooKeeper are supported via >> > the same Curator libraries. >> > >> > ZooKeeper 3.5.x >> > Curator 4.0 has a hard dependency on ZooKeeper 3.5.x >> > If you are using ZooKeeper 3.5.x there's nothing additional to do - >> > just use Curator 4.0 >> > >> > ZooKeeper 3.4.x >> > Curator 4.0 supports ZooKeeper 3.4.x ensembles in a soft-compatibility >> > mode. To use this mode you must exclude ZooKeeper when adding >> > Curator to your dependency management tool. >> >> >> Could it be possible that something went wrong when Curator 4.0.0 is >> working against Zookeeper 3.4.6? (I'm really wondering about this >> soft-compatibility mode introduced in Curator 4.0) >> >> Thank you for reading. >> >> Best regards, >> Shinhyung Yang >> >> >
