It makes sense to me since the bolts can be distributed over different supervisors and may have different DB connections. In this case, you have to detach the hibernate entities from one connection and then re-attach them to the other, if you want to pass them between bolts. — Sincerely, Fan Jiang
On Thu, May 14, 2015 at 9:58 AM, Stephen Powis <[email protected]> wrote: > Hello everyone! > I'm currently toying around with a prototype built ontop of Storm and have > been running into some not so easy going while trying to work with > Hibernate and storm. I was hoping to get input on if this is just a case > of "I'm doing it wrong" or maybe get some useful tips. > In my prototype, I have a need to fan out a single tuple to several bolts > which do data retrieval from our database in parallel, which then get > merged back into a single stream. These data retrieval bolts all find > various hibernate entities and pass them along to the merge bolt. We've > written a kryo serializer that converts from the hibernate entities into > POJOs, which get sent to the merge bolt in tuples. Once all the tuples get > to the merge bolt, it collects them all into a single tuple and passes it > downstream to a bolt which does processing using the entities. > So it looks something like this. > ---- (retrieve bolt a) ---- > / ---- (retrieve bolt b) ----\ > /------(retrieve bolt c) -----\ > --- (split bolt)------(retrieve bolt d)-------(merge bolt) ----- > (processing bolt) > So dealing with detaching the hibernate entities from the session to > serialize them, and then further downstream when we want to work with the > entities again, we have to reattach them to a new session....this seems > kind of awkward. > Does doing the above make sense? Has anyone attempted to do the above? > Any tips or things we should watch out for? Basically looking for any kind > of input for this use case. > Thanks!
