Hey Taylor, Thanks a bunch for getting back to me.
In bolt A, there are two big things going on (1) very CPU-intensive parsing of a large binary object where there are 20-40 parts of the object that are individually processed. Bolt A is the performance bottleneck as the parsing and processing is very CPU-intensive and the capacity of Bolt A is way up in the 0.7 to 1.0 range.This is where I think multi-threading could help out as now I am serially looping through the object fields and processing them individually and (2) the 20-40 resulting tuples are individually emitted with the anchor tuple. The other important fact is that I need to use fieldsGrouping because I am taking the output of Bolt A and caching them in Bolt B, eventually persisting the processed values via batch to HBase My concern is that I have a large fan-out immediately followed by a dramatic fan-in consisting of two factors: (1) output of 1000 Bolt A executors to 200 or less Bolt B executors and (2) one tuple from KS to ack -> 1000 * 20-40 tuples to ack from Bolt A. I am thinking if I did multi-threading in Bolt A to arrive at perhaps 200 Bolt A executors this would dramatically decrease messaging overhead associated with this topology and hopefully improve throughput. Specifically, I am thinking the acking overhead back to 200 Bolt A executors will be way less than 1000 Bolt A executors. In addition, as I noted above, I am serially processing 20-40 fields, and processing 'em in parallel could provide an additional performance boost. Please let me know if I am on the right track and, if not, a better track. :) Thanks! --John On Mon, Nov 23, 2015 at 7:36 PM, P. Taylor Goetz <[email protected]> wrote: > I think it depends on what exactly you're doing in those bolts. > > In other words, we'd need more detail, like code or pseudo code. Otherwise > we are guessing. > > -Taylor > > > On Nov 23, 2015, at 4:37 PM, John Yost <[email protected]> > wrote: > > > > Hi Everyone, > > > > I have a large fan out in my topology 20 spouts -> 1000 boltA executors > followed by a large fan-in (1000 boltA executors to 200 boltB executors). > The performance of the fan-in is really bad. I am wondering if it would > make more sense to instead do multithreading in boltA so that I go 20 KS -> > 200 boltA -> 200 boltB. > > > > Any opinions/ideas/comments would be greatly appreciated. > > > > Thanks! :) > > > > --John >
