Two things. First storm uses zookeeper to store state and offers a watch that can inform a client when a particular znode in zookeeper changes. The callbacks provide a way for a client to be informed that the particular part of the state they care about has changed, or nil if you don't want it. The ZK functionality is not always perfect in getting the callbacks, so it is always a good idea to back up the callback with polling periodically. Also at least for people I know working on DRUID the order of the callbacks on certain versions of ZK can be out of order, so if there are a lot of changes happening to a single ZNode you may need to be careful, luckily this is not an issue for how storm currently uses ZK.
Now for the second thing. JStorm is a fork of Apache Storm where the clojure code was translated into java code, so if you are having trouble reading the clojure code you can look at what JStorm is doing for a hint. Be aware that the two projects have diverged somewhat so they are not going to be identical in their functionality. That being said both projects have been talking with one another about combining. https://issues.apache.org/jira/browse/STORM-717 is a JIRA to work through making that happen. If you or anyone else has an opinion on this please feel free to discuss it on that JIRA on here on the dev list, I'll send out another e-mail to provide a better place than piggybacking it here. The goal would be to maintain binary compatibility, except possibly in the case of dependencies. It would be great to have both groups working together instead of duplication of effort. We also felt that having more of the code base in java would possibly make it more accessible to a wider range of developers. That being said this project is a community effort so anyone who wants to help, or has opinions on this please help out/let us know what you think. - Bobby On Tuesday, March 24, 2015 12:51 AM, Erik Weathers <[email protected]> wrote: hi Longda, thanks for the response. Interesting project there. I would appreciate seeing the architecture/work-flow file if you could please share it. However, we are using standard Storm, and I need to continue supporting our existing system -- which requires me to dig into the Clojure-based storm code. Any other kind passersby that can help me? - Erik On Monday, March 23, 2015, 封仲淹(纪君祥) <[email protected]> wrote: > > You can read the source code of Jstorm, It is java Storm. It is easy to > read > https://github.com/alibaba/jstorm > I can share one freemind file to you to demonstrate the architecture and > work flow. > > > > Best Regards > Longda > > > -----邮件原件----- > 发件人: Erik Weathers [mailto:[email protected]] > 发送时间: 2015年3月24日 8:58 > 收件人: [email protected] > 主题: begging for a bit of help reading storm-core code > > hi Storm Devs! > > I've been trying to discern what the purpose is for the ubiquitous > "callback" variables in cluster.clj and supervisor.clj code. It occurred > to me that some of you fine folks on the dev list must have already braved > this test and could help dispel my ignorance. Specifically, I'm trying to > determine what the mk-storm-cluster-state function is actually doing, and > understanding the callbacks seems critical to that endeavor. > > I had hoped that commit log messages might help me infer the point of > these callbacks, but this particular pattern hasn't changed since the > initial commit by Nathan Marz. > > Thanks for whatever help you can provide! > > - Erik > > P.S., I have many other similar questions about the storm-core Clojure > code, so I wonder if someone might have a suggestion of a different route > for to requesting such code-reading help? > >
