Do you think Java serialization might cause such a huge overhead? I definitely have to optimize my software.
By the way does it make sense to increase the number of tasks? I don't think so, because they are executed serially. Are the following assumptions correct? A good topology config would be one worker per node and one executor for each node and each core. e.g., two 16-core nodes = 2 worker and max 30 executors (1 acker per worker) These 30 executors have to be shared between all bolts and spouts. I think there would be a performance benefit if every worker runs all bolts. Because if a worker runs only a part of bolts the tuples have to be transferred to another worker. Thanks! 2015-01-30 13:35 GMT+01:00 Nathan Leung <[email protected]>: > Assuming you are truly cpu bound and not waiting on io, and 100ms / tuple, > each core can do 10 tuples / s. Each node can do 160, and 19 nodes can do > 3040 / s. So then you have to optimize your software or add more nodes; > it's not a storm issue. > > If you are doing io and not just purely cpu bound you can add more threads > to hide latency and would be able to get higher throughout. > On Jan 30, 2015 4:28 AM, "wlqpku" <[email protected]> wrote: > >> i met the same issue >> >> Sent from X1 7.0 >> >> Martin Illecker <[email protected]>编写： >> >> Hello, >> >> I'm observing a huge performance problem with my topology. >> The topology consists of 5 bolts and two of them are really slow about 60 >> ms and 30 ms of execute latency. (please see attached UI screenshot) >> >> My topology configuration consists of one worker per node and 19 16-core >> nodes, which is a total of 19 workers. >> Every worker runs 3 executors, one for each of the three fast bolts, 5 >> executors for the 60 ms bolt, 3 executors for the 30ms bolt and 3 hidden >> threads (acker, worker buffer receive, worker buffer transfer). >> This leads to a total number of 14 threads per worker, which is the upper >> limit for a 16-core nodes. In my case the CPU is the limiting factor. >> >> With this configuration I could only measure 2000 to 3000 tuples per >> second at the end of this pipeline. I have tried multiple MaxSpoutPending >> settings but 2000 to 3000 tuples per second seem to be the maximum. >> I think, this might be the upper limit for a total execution latency of >> 100ms? >> >> How can I achieve x0,000 tuples for my topology? Or is this impossible >> with these two slow bolts? >> Is there anything I could try because horizontal scaling will not solve >> my problem. >> >> Thanks! >> >> Best regards >> Martin >> >> >> >> >>
