Hi John, Your understanding is right, and I think it makes sense since your bolt is CPU-intensive so you may don't want to let cores spending their time to enqueue and dequeue. Lowering parallelism could help in that case.
Btw, experimenting yourself is the best practice. Just give it a try and see usages of each core. Thanks, Jungtaek Lim (HeartSaVioR) 2016년 3월 26일 (토) 오후 8:34, John Yost <[email protected]>님이 작성: > Hi Everyone, > > I have a very CPU-intensive bolt that requires a high (at least I think > it's high) number of executors per worker--in my case, 10. I am finding > that about 40% of the time for the CPU-intensive bolt executor threads is > spent in the LMAX messaging layer. > > My understanding of Storm executor internals is that each executor has two > threads to process LMAX messages--one incoming and one outgoing--and I am > thinking I am getting a lot of context switching due to the large number of > executors overall in my workers (15 between bolts and spouts). > Consequently, I am thinking of multithreading the CPU-intensive bolt so I > can get the same number of threads processing the data while decreasing > context switching I think is occurring between all of the CPU-intensive > bolt executor threads. > > Question--does this make sense? Any thoughts by people who understand > Storm better than me are always welcome. :) > > Thanks > > --John >
