In CEP we have experienced latency issues chaining disrupter and don't go for PhasedBackOff strategy as there is a hit for calculating time and if you use spinning or yielding there you have to also correlate then with the number of cores on the system which will be a problem when it comes to deployment.
Suho On Sat, Apr 23, 2016 at 7:18 PM, Isuru Ranawaka <[email protected]> wrote: > Hi Chanaka, > > I have run the test for header based routing scenario (Pure CPU) and there > is small improvement on Disruptor. We can get clear performance difference > if we send huge load such that requests are get queued at ThreadPool and > will has huge contention for internal queues but for Disruptor it is > locking free .when profiling this test I didn't see contention in the > ThreadPool level as well. Actually advantage we got through Disruptor is > due to cache optimization at disruptor level. Another problem is we have > active Netty thread pool as well. So there is no probability that Disruptor > threads are always get high priority. Due to that also we cannot get > maximum out of Disruptor thread model. I will run long running test with > significant amount of CPU bound work and see how it behaves. > > @Suho, > > CPU bound Disruptor wait strategy is Sleep waiting and IO bound Disruptor > is Blocking Wait . Tested with PhasedBackOff with LiteLocking as well not > much difference in TPS but observed latency was varied in higher amount > for some requests. > > Number of Disruptors are configurable and I have used five CPU bound > Disruptors and one IO bound DIsruptor for testing which gives maximum Perf > according to testes we carried for Gateway. > > thanks > > > On Fri, Apr 22, 2016 at 8:20 PM, Sriskandarajah Suhothayan <[email protected]> > wrote: > >> Whats the waiting strategy that is being used in Disrupter? And is the >> maximum of Disrupter in all configuration is 2 ? >> >> Suho >> >> >> On Fri, Apr 22, 2016 at 8:01 PM, Chanaka Fernando <[email protected]> >> wrote: >> >>> Hi Isuru, >>> >>> Have we tested on the pure passthrough scenarios? According to the >>> results, we cannot see much performance difference in using disruptor over >>> thread pool. Shall we do some testing around passthrough scenarios and >>> verify? >>> >>> Cheers, >>> Chanaka >>> >>> On Fri, Apr 22, 2016 at 12:54 PM, Isuru Ranawaka <[email protected]> >>> wrote: >>> >>>> Hi all, >>>> >>>> We have working on decoupling engine thread model from transport thread >>>> model.Previously carbon transport was included with the Disruptor and >>>> according to the behaviour of MSF4J it is very hard to mapped custom logics >>>> written using MSF4J to Disruptor thread model in order to gain better >>>> performance. So we have moved Disruptor thread model to carbon-gateway and >>>> carbon-transport is kept with Netty thread pool. >>>> >>>> According to current implementation carbon-transport will dispatch >>>> events to registered message processor via Netty worker threads and there >>>> after it operates under engine level thread model. >>>> >>>> Following is the Thread model diagram for Integration Server. >>>> >>>> [image: gw_thread_model.png] >>>> >>>> Basically it works as follows >>>> >>>> - >>>> >>>> CPU bound mediators are working on CPU bound disruptor threads. >>>> - >>>> >>>> IO bound mediators are working on IO bound disruptor threads. >>>> - >>>> >>>> We assume custom mediators are written in bad way and may contain >>>> blocking calls so those are executed using IO bound mediator. >>>> - >>>> >>>> We have included ThreadPool implementation as well and can switch >>>> between ThreadPool based model or Disruptor based model. >>>> >>>> This is for we haven’t yet finalized exact thread model and we keep >>>> testing with different mediators how both are behaving according to >>>> different parameters like TPS, memory, startup time ,latency , .etc >>>> >>>> >>>> Following are some of the tests results we have conducted with both >>>> thread model implementations for two main scenarios. >>>> >>>> Machine Details >>>> >>>> Server :- 32 core machine with 64 GB memory >>>> >>>> Back End Service :- Netty based Echo Service which has TPS around >>>> 100000 >>>> >>>> Tested message size :- 4kb >>>> >>>> Server startup Time with Disruptor :- 1.34 s >>>> >>>> Server startup time without Disruptor :- 1.38 s >>>> >>>> Use case:- >>>> >>>> (CPU + IO) >>>> >>>> Header based routing with File Writing .One message path is writing >>>> message to file and other one sends messages to Echo service and respond >>>> back to client. >>>> >>>> TPS >>>> >>>> [image: image1.png] >>>> >>>> Latency >>>> >>>> [image: image2.png] >>>> >>>> Memory >>>> >>>> Disruptor >>>> >>>> [image: disruptor.png] >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> Thread Pool >>>> >>>> [image: threadpoolMemory.png] >>>> >>>> Use case:- >>>> >>>> (CPU ) >>>> >>>> Header based routing , send messages to Echo service and respond back >>>> to client. >>>> >>>> TPS >>>> >>>> [image: image3.png] >>>> >>>> Latency >>>> >>>> [image: image4.png] >>>> >>>> >>>> For test results please look in to [1] >>>> >>>> [1] >>>> https://docs.google.com/spreadsheets/d/1A2dxknP1xEJKBpl4ymbQD2Mt9kWywYCI-60j16JVLx0/edit#gid=0 >>>> >>>> >>>> >>>> Thanks >>>> IsuruR >>>> >>>> -- >>>> Best Regards >>>> Isuru Ranawaka >>>> M: +94714629880 >>>> Blog : http://isurur.blogspot.com/ >>>> >>> >>> >>> >>> -- >>> Thank you and Best Regards, >>> Chanaka Fernando >>> Senior Technical Lead >>> WSO2, Inc.; http://wso2.com >>> lean.enterprise.middleware >>> >>> mobile: +94 773337238 >>> Blog : http://soatutorials.blogspot.com >>> LinkedIn:http://www.linkedin.com/pub/chanaka-fernando/19/a20/5b0 >>> Twitter:https://twitter.com/chanakaudaya >>> >>> >>> >>> >>> >>> _______________________________________________ >>> Architecture mailing list >>> [email protected] >>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>> >>> >> >> >> -- >> >> *S. Suhothayan* >> Technical Lead & Team Lead of WSO2 Complex Event Processor >> *WSO2 Inc. *http://wso2.com >> * <http://wso2.com/>* >> lean . enterprise . middleware >> >> >> *cell: (+94) 779 756 757 <%28%2B94%29%20779%20756%20757> | blog: >> http://suhothayan.blogspot.com/ <http://suhothayan.blogspot.com/>twitter: >> http://twitter.com/suhothayan <http://twitter.com/suhothayan> | linked-in: >> http://lk.linkedin.com/in/suhothayan <http://lk.linkedin.com/in/suhothayan>* >> > > > > -- > Best Regards > Isuru Ranawaka > M: +94714629880 > Blog : http://isurur.blogspot.com/ > -- *S. Suhothayan* Technical Lead & Team Lead of WSO2 Complex Event Processor *WSO2 Inc. *http://wso2.com * <http://wso2.com/>* lean . enterprise . middleware *cell: (+94) 779 756 757 | blog: http://suhothayan.blogspot.com/ <http://suhothayan.blogspot.com/>twitter: http://twitter.com/suhothayan <http://twitter.com/suhothayan> | linked-in: http://lk.linkedin.com/in/suhothayan <http://lk.linkedin.com/in/suhothayan>*
_______________________________________________ Architecture mailing list [email protected] https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
