The number of tasks depends. We run the default 4 workers per supervisor, also on fairly large machines. However, we have more executors than cpu cores because a lot of our bolts do communications on the network.
You should serialize with kryo if at all possible. That said I find it highly doubtful that java serialization is your problem, unless you are sending a lot of data. And even then if it stays in process it won't get serialized at all. I would run a standalone check of the performance of the MaxExt library (which I gather you are using from your other thread). On Jan 30, 2015 9:07 AM, "Martin Illecker" <[email protected]> wrote: > Do you think Java serialization might cause such a huge overhead? > I definitely have to optimize my software. > > By the way does it make sense to increase the number of tasks? > I don't think so, because they are executed serially. > > Are the following assumptions correct? > > A good topology config would be one worker per node and one executor for > each node and each core. > e.g., two 16-core nodes = 2 worker and max 30 executors (1 acker per > worker) > These 30 executors have to be shared between all bolts and spouts. > > I think there would be a performance benefit if every worker runs all > bolts. > Because if a worker runs only a part of bolts the tuples have to be > transferred to another worker. > > Thanks! > > 2015-01-30 13:35 GMT+01:00 Nathan Leung <[email protected]>: > >> Assuming you are truly cpu bound and not waiting on io, and 100ms / >> tuple, each core can do 10 tuples / s. Each node can do 160, and 19 nodes >> can do 3040 / s. So then you have to optimize your software or add more >> nodes; it's not a storm issue. >> >> If you are doing io and not just purely cpu bound you can add more >> threads to hide latency and would be able to get higher throughout. >> On Jan 30, 2015 4:28 AM, "wlqpku" <[email protected]> wrote: >> >>> i met the same issue >>> >>> Sent from X1 7.0 >>> >>> Martin Illecker <[email protected]>编写： >>> >>> Hello, >>> >>> I'm observing a huge performance problem with my topology. >>> The topology consists of 5 bolts and two of them are really slow about >>> 60 ms and 30 ms of execute latency. (please see attached UI screenshot) >>> >>> My topology configuration consists of one worker per node and 19 16-core >>> nodes, which is a total of 19 workers. >>> Every worker runs 3 executors, one for each of the three fast bolts, 5 >>> executors for the 60 ms bolt, 3 executors for the 30ms bolt and 3 hidden >>> threads (acker, worker buffer receive, worker buffer transfer). >>> This leads to a total number of 14 threads per worker, which is the >>> upper limit for a 16-core nodes. In my case the CPU is the limiting factor. >>> >>> With this configuration I could only measure 2000 to 3000 tuples per >>> second at the end of this pipeline. I have tried multiple MaxSpoutPending >>> settings but 2000 to 3000 tuples per second seem to be the maximum. >>> I think, this might be the upper limit for a total execution latency of >>> 100ms? >>> >>> How can I achieve x0,000 tuples for my topology? Or is this impossible >>> with these two slow bolts? >>> Is there anything I could try because horizontal scaling will not solve >>> my problem. >>> >>> Thanks! >>> >>> Best regards >>> Martin >>> >>> >>> >>> >>> >
