Hi! Since it's about multilang feature and you can use your own implementation of multilang (and I believe multilang library developers are subscribing user group), I wanna get opinion about changing multilang heartbeat mechanism.
At Storm 0.9.3, Storm introduces multilang heartbeat feature. http://storm.apache.org/documentation/Multilang-protocol.html If you use Storm 0.9.3 and higher, and didn't know about the change, you may skip this mail. Since it contains some design constraint, I'm trying my best to add workarounds, but it cannot cover whole situation (STORM-738 <https://issues.apache.org/jira/browse/STORM-738>). That's why I want to change mechanism to get rid of design constraint. AS-IS (STORM-513 <https://issues.apache.org/jira/browse/STORM-513>) - When subprocess receives heartbeat tuple, subprocess sends sync to parent. - ShellSpout / ShellBolt updates last heartbeat timestamp when it receives sync. -- added workaround : ShellSpout / ShellBolt updates timestamp when it receives any kind of message. (It doesn't applied to ShellBolt yet, but it's ready for review. STORM-742 <https://issues.apache.org/jira/browse/STORM-742>) - ShellSpout / ShellBolt checks last heartbeat timestamp periodically, and if timestamp is not updated well, it suicides itself. TO-BE (STORM-871 <https://issues.apache.org/jira/browse/STORM-871>) - Subprocess has to update pid file's modified time periodically. -- In default implementation, it updates pid file every 1 sec. -- It should be handled concurrently with executing pending tuples. -- Some languages couldn't implement this clearly, but I don't have an idea what languages could be. - ShellSpout / ShellBolt checks last heartbeat timestamp by reading pid file's modified time periodically, and if timestamp is not updated well, it suicides itself. - Heartbeat tuple is removed. Please let me know your opinion, especially when you're developing multilang libraries. Thanks, Jungtaek Lim (HeartSaVioR)
