Our producers/consumers actually logs the elapsed time. This was slowing down the test. I deactivated the logging and with a dispatcher only, I am at around 47 000 msg/s with asynchronous sending.
> From: [email protected] > To: [email protected] > Subject: RE: [Performance] Benchmarking Qpid dispatch router 0.6.0 with Qpid > Java Broker 6.0.0 > Date: Wed, 3 Aug 2016 18:39:23 +0200 > > And how do you measure your throughput? > > > From: [email protected] > > To: [email protected] > > Subject: RE: [Performance] Benchmarking Qpid dispatch router 0.6.0 with > > Qpid Java Broker 6.0.0 > > Date: Wed, 3 Aug 2016 18:38:12 +0200 > > > > Hello Ulf, > > > > I am sending messages with a byte array of 100 bytes and I am using Berkley > > DB as a message store (which should be slower than having memory only > > message store, no?) > > > > With 1 consumer, 1 producer and no broker, I am at 33k msgs/sec if they are > > all on the same machine and I have set "jms.forceAsyncSend=true" on the > > producer and "jms.sendAcksAsync=true" for the consumer. > > > > Are you using other options to get 190k? Do you think JMS might be a > > bottleneck? Or something else in my config/test? > > > > JMS client 0.9.0 > > Qpid Java Broker 6.0.1 > > Dispatcher 0.6.0 > > > > Adel > > > > > Subject: Re: [Performance] Benchmarking Qpid dispatch router 0.6.0 with > > > Qpid Java Broker 6.0.0 > > > To: [email protected] > > > From: [email protected] > > > Date: Wed, 3 Aug 2016 16:23:06 +0200 > > > > > > Hi, > > > > > > Excuse me if this was already mentioned somewhere, but what is the size > > > of the messages you are sending ? > > > > > > FWIW, I'm able to get around 30-40k msgs/sec sustained with 1 producer, > > > 1 consumer, 1 dispatch (4 worker threads) and 1 broker (activemq-5). The > > > sender sends unsettled messages as fast as it can using qpid-proton > > > reactor API which is sending async up to the window limit. > > > > > > With no broker involved, I'm getting ~190k msgs/sec. > > > > > > All of these numbers are from my 8 core laptop. Message size is 128 bytes. > > > > > > I don't know the dispatcher that well, but I think it should be able to > > > handle data from each connector just fine given the numbers I have seen. > > > > > > On 08/03/2016 02:41 PM, Adel Boutros wrote: > > > > > > > > > > > > > > > > Hello again, > > > > > > > > > > > > > > > > > As requested, I added a 2nd connector and the appropriate autoLinks on > > > > the same host/port but with a different name. It seems to have resolved > > > > the issue. > > > > > > > > 1 Broker, 1 Dispatcher, 3 producers, 0 consumers, 1 connectors --> 5000 > > > > msg/s. > > > > 1 Broker, 1 Dispatcher, 3 producers, 0 consumers, 2 connectors --> 6600 > > > > msg/s. > > > > 1 Broker, 1 Dispatcher, 4 producers, 0 consumers, 2 connectors --> 7700 > > > > msg/s. > > > > > > > > I think this confirms the problem is due to the fact a single > > > > connection is being shared by all clients (consumers/producers) and > > > > that having a sort of pool of connections or a connection per > > > > workerThread is a solution to consider. > > > > > > > > What do you think? > > > > > > > > I added a 3rd connector to see if it changes anything but it > > > > didn't. Do you think this is maybe because the dispatcher is not able > > > > to process fast enough and saturate the 2 connectors? > > > > 1 Broker, 1 Dispatcher, 4 producers, 0 consumers, 3 connectors --> 7700 > > > > msg/s. > > > > > > > > > > > > > > > > > Adel > > > > > > > >> From: [email protected] > > > >> To: [email protected] > > > >> Subject: RE: [Performance] Benchmarking Qpid dispatch router 0.6.0 > > > >> with Qpid Java Broker 6.0.0 > > > >> Date: Tue, 2 Aug 2016 22:21:54 +0200 > > > >> > > > >> Sorry for the typo. Indeed, it was with 3 producers. I used 4 and 8 > > > >> workerThread but there wasn't a difference. > > > >> We want to benchmark in the worst case scenarios actually to see what > > > >> is the minimum we can guarantee. This is why we are using synchronous > > > >> sending. In the future, we will also benchmark with full SSL/SASL to > > > >> see what impact it has on the performance. > > > >>> Subject: Re: [Performance] Benchmarking Qpid dispatch router 0.6.0 > > > >>> with Qpid Java Broker 6.0.0 > > > >>> To: [email protected] > > > >>> From: [email protected] > > > >>> Date: Tue, 2 Aug 2016 20:41:54 +0100 > > > >>> > > > >>> On 02/08/16 20:25, Adel Boutros wrote: > > > >>>> How about the tests we did with consumer/producers connected > > > >>>> directly to the dispatcher without any broker where we had 16 000 > > > >>>> msg/s with 4 producers. Is it also a very low value given that there > > > >>>> is no persistence or storing here? It was also synchronous sending. > > > >>> > > > >>> The rate is low because it is synchronous. One messages is sent to the > > > >>> consumer who acknowledges it, the acknowledgement is then conveyed > > > >>> back > > > >>> to the sender who then can send the next message. > > > >>> > > > >>> The rate for a single producer through the router was 6,000 per > > > >>> second. > > > >>> That works out as a roundtrip time of 167 microsecs or so. In your > > > >>> table, the 16,000 rate was listed as being for 3 producers. Based on > > > >>> the > > > >>> rate of a single producer, the best you could hope for there is 3 * > > > >>> 6,000 i.e 18,000. (How many worker threads did you have on the router > > > >>> for that case?) > > > >>> > > > >>> --------------------------------------------------------------------- > > > >>> To unsubscribe, e-mail: [email protected] > > > >>> For additional commands, e-mail: [email protected] > > > >>> > > > >> > > > > > > > > > > > > > > > > > > -- > > > Ulf > > > > > > --------------------------------------------------------------------- > > > To unsubscribe, e-mail: [email protected] > > > For additional commands, e-mail: [email protected] > > > > > >
