Hi Malith Yes, there is a drop on throughput if we are not using disruptor. Thats why we have not removed disruptor but rather made it configurable, such that based on the use-case we can pick latency or throughput. And yes getting both is not trivial and needs heavy use-case specific tuning.
Regards Suho On Mon, May 16, 2016 at 2:06 PM, Malith Jayasinghe <[email protected]> wrote: > Although there is an improvement in the latency, I notice that we get a > significant reduction in the throughput (in both scenarios) when not using > the disruptor. Is there a way to address this? I guess it will be > difficult to optimise both performance metrics at the same time? > > On Mon, May 16, 2016 at 11:10 AM, Sriskandarajah Suhothayan <[email protected] > > wrote: > >> >> >> On Mon, May 16, 2016 at 11:07 AM, Seshika Fernando <[email protected]> >> wrote: >> >>> Hi Suho, >>> >>> Looks good. >>> >>> If (bufferSize='xxxx') is optional, what is the default bufferSize that >>> will be taken if I just add @plan:async ? >>> >>> Yes the default one will be taken. & the default one is 1024. >> >> Suho >> >> >>> seshi >>> >>> On Sun, May 15, 2016 at 3:58 PM, Sriskandarajah Suhothayan < >>> [email protected]> wrote: >>> >>>> >>>> Hi >>>> >>>> We have made Disruptor as optional for Siddhi[1], currently its always >>>> enabled and it uses PhasedBackoffWaitStrategy, event though Disruptor was >>>> giving high throughput there are several issues identified. >>>> >>>> 1. It is adding latency, and tuning latency is subject to use-cases >>>> hence the deployment is becoming complicated. >>>> 2. PhasedBackoffWaitStrategy is not showing good results when there are >>>> lots of Disruptors. >>>> >>>> Hence we have removed disruptor by default and made is as an option to >>>> add via configurations. >>>> >>>> By using @plan:async(bufferSize='xxxx') you can enable Disruptor at all >>>> streams in an execution-plan with the queue size of xxxx. Here >>>> (bufferSize='xxxx') is optional. >>>> >>>> e.g >>>> >>>> *@plan:async(bufferSize='2')* >>>> >>>> define stream cseEventStream (symbol string, price float, volume int); >>>> >>>> @info(name = 'query1') >>>> from cseEventStream[70 > price] >>>> select * >>>> insert into innerStream ; >>>> >>>> @info(name = 'query2') >>>> from innerStream[volume > 90] >>>> select * >>>> insert into outputStream ; >>>> >>>> In this case cseEventStream, innerStream and outputStream will have >>>> async behaviors using Disruptor >>>> >>>> Alternatively we can also enable Disruptor for specific streams by >>>> annotating them as below. >>>> >>>> e.g >>>> >>>> *@async(bufferSize='2')* >>>> define stream cseEventStream (symbol string, price float, volume int); >>>> >>>> @info(name = 'query1') >>>> from cseEventStream[70 > price] >>>> select * >>>> insert into innerStream ; >>>> >>>> @info(name = 'query2') >>>> from innerStream[volume > 90] >>>> select * >>>> insert into outputStream ; >>>> >>>> >>>> Here only cseEventStream will have async behavior using Disruptor >>>> >>>> Performance stats after the improvements. >>>> >>>> Filter Query *without* *Disruptor* >>>> Throughput : 3.5M Events/sec >>>> Time spend : *2.29E-4 ms* >>>> >>>> Filter Query *with Disruptor * >>>> Throughput : *6.1M Events/sec * >>>> Time spend : 0.028464 ms >>>> >>>> Multiple Filter Query without Disruptor >>>> Throughput : 3.0M Events/sec >>>> Time spend : 2.91E-4 ms >>>> >>>> Multiple Filter Query with Disruptor >>>> Throughput : 5.5M Events/sec >>>> Time spend : 0.089888 ms >>>> >>>> [1]https://github.com/wso2/siddhi/tree/latency >>>> >>>> Regards >>>> Suho >>>> >>>> -- >>>> >>>> *S. Suhothayan* >>>> Technical Lead & Team Lead of WSO2 Complex Event Processor >>>> *WSO2 Inc. *http://wso2.com >>>> * <http://wso2.com/>* >>>> lean . enterprise . middleware >>>> >>>> >>>> *cell: (+94) 779 756 757 <%28%2B94%29%20779%20756%20757> | blog: >>>> http://suhothayan.blogspot.com/ <http://suhothayan.blogspot.com/>twitter: >>>> http://twitter.com/suhothayan <http://twitter.com/suhothayan> | linked-in: >>>> http://lk.linkedin.com/in/suhothayan >>>> <http://lk.linkedin.com/in/suhothayan>* >>>> >>> >>> >> >> >> -- >> >> *S. Suhothayan* >> Technical Lead & Team Lead of WSO2 Complex Event Processor >> *WSO2 Inc. *http://wso2.com >> * <http://wso2.com/>* >> lean . enterprise . middleware >> >> >> *cell: (+94) 779 756 757 <%28%2B94%29%20779%20756%20757> | blog: >> http://suhothayan.blogspot.com/ <http://suhothayan.blogspot.com/>twitter: >> http://twitter.com/suhothayan <http://twitter.com/suhothayan> | linked-in: >> http://lk.linkedin.com/in/suhothayan <http://lk.linkedin.com/in/suhothayan>* >> >> _______________________________________________ >> Architecture mailing list >> [email protected] >> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >> >> > > > -- > Malith Jayasinghe Ph.D. > > > > WSO2, Inc. (http://wso2.com) > Email : [email protected] > Mobile : 0770704040 > Lean . Enterprise . Middleware > > _______________________________________________ > Architecture mailing list > [email protected] > https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture > > -- *S. Suhothayan* Technical Lead & Team Lead of WSO2 Complex Event Processor *WSO2 Inc. *http://wso2.com * <http://wso2.com/>* lean . enterprise . middleware *cell: (+94) 779 756 757 | blog: http://suhothayan.blogspot.com/ <http://suhothayan.blogspot.com/>twitter: http://twitter.com/suhothayan <http://twitter.com/suhothayan> | linked-in: http://lk.linkedin.com/in/suhothayan <http://lk.linkedin.com/in/suhothayan>*
_______________________________________________ Architecture mailing list [email protected] https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
