Although there is an improvement in the latency, I notice that we get a significant reduction in the throughput (in both scenarios) when not using the disruptor. Is there a way to address this? I guess it will be difficult to optimise both performance metrics at the same time?
On Mon, May 16, 2016 at 11:10 AM, Sriskandarajah Suhothayan <[email protected]> wrote: > > > On Mon, May 16, 2016 at 11:07 AM, Seshika Fernando <[email protected]> > wrote: > >> Hi Suho, >> >> Looks good. >> >> If (bufferSize='xxxx') is optional, what is the default bufferSize that >> will be taken if I just add @plan:async ? >> >> Yes the default one will be taken. & the default one is 1024. > > Suho > > >> seshi >> >> On Sun, May 15, 2016 at 3:58 PM, Sriskandarajah Suhothayan <[email protected] >> > wrote: >> >>> >>> Hi >>> >>> We have made Disruptor as optional for Siddhi[1], currently its always >>> enabled and it uses PhasedBackoffWaitStrategy, event though Disruptor was >>> giving high throughput there are several issues identified. >>> >>> 1. It is adding latency, and tuning latency is subject to use-cases >>> hence the deployment is becoming complicated. >>> 2. PhasedBackoffWaitStrategy is not showing good results when there are >>> lots of Disruptors. >>> >>> Hence we have removed disruptor by default and made is as an option to >>> add via configurations. >>> >>> By using @plan:async(bufferSize='xxxx') you can enable Disruptor at all >>> streams in an execution-plan with the queue size of xxxx. Here >>> (bufferSize='xxxx') is optional. >>> >>> e.g >>> >>> *@plan:async(bufferSize='2')* >>> >>> define stream cseEventStream (symbol string, price float, volume int); >>> >>> @info(name = 'query1') >>> from cseEventStream[70 > price] >>> select * >>> insert into innerStream ; >>> >>> @info(name = 'query2') >>> from innerStream[volume > 90] >>> select * >>> insert into outputStream ; >>> >>> In this case cseEventStream, innerStream and outputStream will have >>> async behaviors using Disruptor >>> >>> Alternatively we can also enable Disruptor for specific streams by >>> annotating them as below. >>> >>> e.g >>> >>> *@async(bufferSize='2')* >>> define stream cseEventStream (symbol string, price float, volume int); >>> >>> @info(name = 'query1') >>> from cseEventStream[70 > price] >>> select * >>> insert into innerStream ; >>> >>> @info(name = 'query2') >>> from innerStream[volume > 90] >>> select * >>> insert into outputStream ; >>> >>> >>> Here only cseEventStream will have async behavior using Disruptor >>> >>> Performance stats after the improvements. >>> >>> Filter Query *without* *Disruptor* >>> Throughput : 3.5M Events/sec >>> Time spend : *2.29E-4 ms* >>> >>> Filter Query *with Disruptor * >>> Throughput : *6.1M Events/sec * >>> Time spend : 0.028464 ms >>> >>> Multiple Filter Query without Disruptor >>> Throughput : 3.0M Events/sec >>> Time spend : 2.91E-4 ms >>> >>> Multiple Filter Query with Disruptor >>> Throughput : 5.5M Events/sec >>> Time spend : 0.089888 ms >>> >>> [1]https://github.com/wso2/siddhi/tree/latency >>> >>> Regards >>> Suho >>> >>> -- >>> >>> *S. Suhothayan* >>> Technical Lead & Team Lead of WSO2 Complex Event Processor >>> *WSO2 Inc. *http://wso2.com >>> * <http://wso2.com/>* >>> lean . enterprise . middleware >>> >>> >>> *cell: (+94) 779 756 757 <%28%2B94%29%20779%20756%20757> | blog: >>> http://suhothayan.blogspot.com/ <http://suhothayan.blogspot.com/>twitter: >>> http://twitter.com/suhothayan <http://twitter.com/suhothayan> | linked-in: >>> http://lk.linkedin.com/in/suhothayan <http://lk.linkedin.com/in/suhothayan>* >>> >> >> > > > -- > > *S. Suhothayan* > Technical Lead & Team Lead of WSO2 Complex Event Processor > *WSO2 Inc. *http://wso2.com > * <http://wso2.com/>* > lean . enterprise . middleware > > > *cell: (+94) 779 756 757 <%28%2B94%29%20779%20756%20757> | blog: > http://suhothayan.blogspot.com/ <http://suhothayan.blogspot.com/>twitter: > http://twitter.com/suhothayan <http://twitter.com/suhothayan> | linked-in: > http://lk.linkedin.com/in/suhothayan <http://lk.linkedin.com/in/suhothayan>* > > _______________________________________________ > Architecture mailing list > [email protected] > https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture > > -- Malith Jayasinghe Ph.D. WSO2, Inc. (http://wso2.com) Email : [email protected] Mobile : 0770704040 Lean . Enterprise . Middleware
_______________________________________________ Architecture mailing list [email protected] https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
