At my company we have implemented this using a node server to take care of the HTTP part.
I've set up a storm node to run the drpc daemon. Currently we are suing the LinearDRPCTopology, that is deprecated. It works as required though. On Sunday 20 April 2014 04:21:30 PM IST, Joël Kuiper wrote: > Unfortunately using Trident is not an option due to the multi-language > requirement. > There’s a lot of peer-reviewed stuff that cannot be (trivially) ported to the > JVM. > > I guess creating a generic “call Storm over HTTP” with a nice protocol would > be an interesting project in itself though. > > On 18 Apr 2014, at 17:29, Marc Vaillant <[email protected]> wrote: > >> Have you looked at Trident + DRPC? >> >> https://github.com/nathanmarz/storm/wiki/Trident-tutorial >> >> Also, I came across the following once but I've never tried it and I'm >> not sure how mature it is: >> >> https://github.com/chriskchew/restexpress-storm >> >> Marc >> >> On Fri, Apr 18, 2014 at 04:40:06PM +0200, Joël Kuiper wrote: >>> Hey, >>> >>> So I’m contemplating using Storm for processing for doing rather >>> complicated analyses on user submitted data (either through HTTP or >>> WebSockets). >>> Storm seems perfect for the multi-stage processing that I need, and it’s >>> real-time nature would fit the type of interactions I require. >>> Furthermore many steps would involve already written analyses in Python and >>> R, so using bolts for that would be great. >>> >>> However, hooking up Storm behind an HTTP like Ring (optionally with >>> http-kit) seems non-trivial. >>> >>> I first thought of pushing the messages on a core.async queue and having a >>> Spout consume them. But I realise this might fail in a cluster. >>> So the current thinking is >>> >>> * HTTP Request -> create job & push job on Kafka jobs topic >>> * Inform the user about the created job, which includes a (WebSocket) url >>> to listen for results >>> * Storm consumes from Kafka >>> * End results are pushed to bolts that push on a Kafka topic for results >>> * Make server listen on results topic & push results to appropriate jobs >>> (i.e. notify user on job url) >>> >>> But to be honest … this seems a bit of hassle to set-up. It would require >>> server/developers to set-up Kafka, Storm and all related dependencies. >>> It’s a lot of “stuff” just to get it running, which might hamper developer >>> adaptation at our shop. >>> >>> Is there a simpeler way of getting this going, or does this seem to be the >>> most appropriate way? >>> >>> Many thanks, >>> Joël >>> >> >> >
