Hey Oliver, We're not seeing any errors in our logs after switching to the zmq transport. The reason we tried the zmq transport was several users on the mailing list seemed to have had some success with switching to it after seeing intermittent issues with netty. We were seeing timeouts between workers communication. The reconnect errors looked like the ones referenced here: http://pastebin.com/XXZBsEj1 (from http://mail-archives.apache.org/mod_mbox/incubator-storm-user/201403.mbox/%3CCA%2BB%2BUZujQ7m_Y%2BZOv61hc7FK4EL3%3D%2B_6bpnnOK7quh-9ksN6tg%40mail.gmail.com%3E )
On Tue, Apr 15, 2014 at 6:52 AM, Oliver Hall <[email protected]> wrote: > Hi Michael, > > We've been seeing a similar issue, after upgrading from Storm 0.8.2 to > 0.9.0.1. When we start our topology, it times out every batch of > events. We're using Trident, with nodes set up on AWS (see > > http://mail-archives.apache.org/mod_mbox/incubator-storm-user/201404.mbox/%3CCAMij6%3Dc1drvX7QR6gk7JYZG9gk0%3DHbS0fKcTOEdKpvr%2BTqcSyg%40mail.gmail.com%3E > for our mail to the list a few days back). We're not getting any > errors in the logs at all, do you get any? > > > Oli Hall > > On 15 April 2014 10:37, 朱春来 <[email protected]> wrote: > > Hi Michael Chang, > > > > > > > > Did you ack or fail tuple in the bolt timely and please check > the > > bolt processing speed of a tuple. > > > > > > > > > > > > > > > > 发件人: Michael Chang [mailto:[email protected]] > > 发送时间: 2014年4月15日 16:41 > > 收件人: [email protected] > > 主题: Storm Topology Halts > > > > > > > > [email protected] all, > > > > > > > > Issue: > > > > > > > > We are having issues with stuck topologies. When submitted and started, > our > > topology will start processing for a while, then completely halt for > around > > topology.max.spout.pending seconds, after which it seems that all the > > in-flight tuples are failed. This cycle will loop continuously. Has > > anybody seen this issue / have suggestions about how to debug? > > > > > > > > Environment: > > > > > > > > We are running a storm cluster in AWS, non-vpc. We’re running 0.9.1 but > > using guava 16.0.1 and httpclient 4.3.1 in the lib path. We were > originally > > trying this with the regular netty transport, and reverting back to the > zmq > > transport seemed to help at first, but now we’re seeing the same > behavior as > > well, so it seems like a deeper rooted problem than just the transport. > > > > > > > > Any help would be appreciated. > > > > > > > > Thanks, > > > > > > > > Michael >
