Thanks Taylor. I was afraid creating the JRuby runtime this way might be expensive. Initially I did create it inside of the prepare() method but I ran into some trouble because the Ruby class is not serializable. I played around with it a little more today and I had some success creating a static class to hold my Ruby objects. That way I could initialize it in prepare() and just call my processing code in execute.
When I initialize my Ruby objects this way I'm generally receiving my Acks the way I expect to, but there are still some that I don't receive. This is different than when I had the Ruby initialization in execute. In that scenario I failed to receive any Acks at all. I'm still new to Storm so it's possible that I'm just missing something obvious. The thing is though when I take the JRuby code out everything works fine. Is it possible that I'm just not waiting long enough? In my tests it takes about 10 seconds for my test data to flow through the topology (single node) and I shut it down after 30 seconds. The Acks i do receive happen pretty much instantly after I call the ack() in the last Bolt though. On Wed, Feb 26, 2014 at 6:13 PM, P. Taylor Goetz <[email protected]> wrote: > Hi Jonathan, > > I've used jruby fairly extensively with storm (though with the trident > API), but it's been a while so I'm rusty. > > Initializing the jruby runtime is very expensive, so you should do that in > the prepare() method of your bolt. That means you'll have to store it as an > instance variable in your bolt, which in turn opens the door for potential > concurrency issues in your jruby code. Be warned. > > It can get kind of crazy. I forget what the magic jruby runtime > configuration was off hand. But it works. > > I'll try to unarchive those memories and reply. > > -Taylor > > > > > > > On Feb 25, 2014, at 10:32 PM, Jonathan Nilsson < > [email protected]> wrote: > > > > I'm trying to write a Storm Bolt that does some processing with JRuby. > When my data goes through this Bolt I see that the Spout does not appear to > be receiving any acks. I'm pretty sure I'm anchoring my tuples correctly. > If I take the JRuby Bolt out of the topology everything works fine again. > In trying to isolate the problem I wrote a Bolt that does no processing at > all but does call Ruby.getGlobalRuntime(). That call alone seems to be > enough to stop the acks from flowing. I've boiled the execute method down to > > > > public void execute(Tuple input) { > > Ruby.getGlobalRuntime(); > > LOG.info("Sending Ack for "+input); > > collector.ack(input); > > } > > > > and I get the log message but no messages in the Spout's ack or fail > methods. If I remove the Ruby.getGlobalRuntime(); line everything works. > I've tried using Ruby.getThreadLocalRuntime() but it doesn't seem to make a > difference. Has anyone seen a similar problem? Are there any tricks to > calling JRuby code from within Storm? >
