We have completed the performance test for IS-Analytics for a Single DAS node after fixing the bottlenecks and performance issues we found in the initial testing. Following are the results,
*Case 1 : *Not persisting any data streams only siddhi queries were running. TPS : 150K. *Case 2* : Incoming events are summarized using siddhi queries as the first level of summarization only those summarized event streams are persisted. TPS : 94K *Case 3* : All streams including incoming raw event stream is also persisted. TPS 11K This test was conducted on a single laptop with 16GB of memory and 2.70Gz 8 core machine. MySQL was also running in the same machine. Thanks *,Sajith Ravindra* Senior Software Engineer WSO2 Inc.; http://wso2.com lean.enterprise.middleware mobile: +94 77 2273550 blog: http://sajithr.blogspot.com/ <http://lk.linkedin.com/pub/shani-ranasinghe/34/111/ab> On Thu, Jun 30, 2016 at 7:25 PM, Mohanadarshan Vivekanandalingam < [email protected]> wrote: > > > On Wed, Jun 29, 2016 at 9:23 PM, Dulitha Wijewantha <[email protected]> > wrote: > >> Hi Mohan/Sajith, >> Can you please explain what this particular scenario was in IS? Is this a >> counting scenario on a window? This would provide input for others who are >> writing real time analytics on CEP. >> > > Yes, Here we have mainly focused on some aggregations like Sum with > external time batch windows.. Queries can be found in [1] > > [1] > https://github.com/wso2/analytics-is/blob/master/features/org.wso2.carbon.analytics.is.feature/src/main/capp/AuthenticationAnalyticsExecutionPlan/IsAnalytics-ExecutionPlan-AuthenticationData.siddhiql > > Thanks, > Mohan > > >> >> Cheers~ >> >> On Mon, Jun 27, 2016 at 5:50 PM, Mohanadarshan Vivekanandalingam < >> [email protected]> wrote: >> >>> >>> >>> On Mon, Jun 27, 2016 at 8:03 PM, Seshika Fernando <[email protected]> >>> wrote: >>> >>>> Iranga, >>>> The optimization applies to siddhi queries that were written for IS >>>> analytics. Other product analytics would have different queries so this is >>>> not applicable there. >>>> This is not a siddhi level change, rather an optimization of the >>>> previously written queries. >>>> >>> Yes, above per-second and per-minute design is applied to some other >>> analytics product as well but it is not generic and too specific for the >>> usecase and query as mentioned by Seshi.. >>> >>> Thanks, >>> Mohan >>> >>> >>>> Seshi >>>> On 27 Jun 2016 16:48, "Iranga Muthuthanthri" <[email protected]> wrote: >>>> >>>>> >>>>> >>>>> On Mon, Jun 27, 2016 at 4:35 PM, Mohanadarshan Vivekanandalingam < >>>>> [email protected]> wrote: >>>>> >>>>>> >>>>>> >>>>>> On Mon, Jun 27, 2016 at 4:30 PM, Seshika Fernando <[email protected]> >>>>>> wrote: >>>>>> >>>>>>> Hey saith, >>>>>>> >>>>>>> This is great. So when you removed the duplicate windows, there were >>>>>>> no OOM issues? >>>>>>> >>>>>> Yes, we have introduced per-second windows and eventually per-minute >>>>>> event count get reduced. Then, there will not be an OOM.. >>>>>> >>>>> >>>>> is this specific for IS-Analytics or common to all other 'product >>>>> analytics'. >>>>> >>>>> >>>>>> Latest number is also seems good.. SajithR, can share those numbers.. >>>>>> >>>>>> Thanks, >>>>>> Mohan >>>>>> >>>>>> >>>>>>> Seshi >>>>>>> On 24 Jun 2016 14:28, "Sajith Ravindra" <[email protected]> wrote: >>>>>>> >>>>>>>> Hi Malith, >>>>>>>> >>>>>>>> Thanks for the explanation. >>>>>>>>> >>>>>>>>> I would expect some variation in the throughput. The aim should be >>>>>>>>> minimize the variation in the throughput (while maintaining the >>>>>>>>> throughput >>>>>>>>> at its highest level). >>>>>>>>> >>>>>>>> Agreed. Actually, our expectation is to minimize the fluctuation >>>>>>>> and to increase the throughput. >>>>>>>> >>>>>>>>> >>>>>>>>> It is possible to measure the latency as well? >>>>>>>>> >>>>>>>> In IS-Analytics we don't generate any output events or alerts >>>>>>>> currently. Therefore, we don't calculate the latency. What we do is >>>>>>>> summarize events using siddhi queries and persist them in a DB and >>>>>>>> further >>>>>>>> summarize persisted data using Spark and then displayed through >>>>>>>> dashbaords. >>>>>>>> >>>>>>>> >>>>>>>> Thanks >>>>>>>> *,Sajith Ravindra* >>>>>>>> Senior Software Engineer >>>>>>>> WSO2 Inc.; http://wso2.com >>>>>>>> lean.enterprise.middleware >>>>>>>> >>>>>>>> mobile: +94 77 2273550 >>>>>>>> blog: http://sajithr.blogspot.com/ >>>>>>>> <http://lk.linkedin.com/pub/shani-ranasinghe/34/111/ab> >>>>>>>> >>>>>>>> On Fri, Jun 24, 2016 at 1:57 PM, Malith Jayasinghe < >>>>>>>> [email protected]> wrote: >>>>>>>> >>>>>>>>> Hi Sajith, >>>>>>>>> >>>>>>>>> Thanks for the explanation. >>>>>>>>> >>>>>>>>> I would expect some variation in the throughput. The aim should be >>>>>>>>> minimize the variation in the throughput (while maintaining the >>>>>>>>> throughput >>>>>>>>> at its highest level). >>>>>>>>> >>>>>>>>> It is possible to measure the latency as well? >>>>>>>>> >>>>>>>>> regards >>>>>>>>> >>>>>>>>> Malith >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> On Thu, Jun 23, 2016 at 11:13 PM, Sajith Ravindra < >>>>>>>>> [email protected]> wrote: >>>>>>>>> >>>>>>>>>> Hi Malith, >>>>>>>>>> >>>>>>>>>> The deployment is just a standalone DAS server, and we are >>>>>>>>>> planning to do a test for HA deployment in recent future. >>>>>>>>>> >>>>>>>>>> The workload is generated by a .csv data file which has 100K >>>>>>>>>> sample events, 10M events are generated by iterating through the >>>>>>>>>> same data >>>>>>>>>> set 100 times. But we keep increasing the timestamp. A simple thrift >>>>>>>>>> client >>>>>>>>>> is used to publish data. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Thanks >>>>>>>>>> *,Sajith Ravindra* >>>>>>>>>> Senior Software Engineer >>>>>>>>>> WSO2 Inc.; http://wso2.com >>>>>>>>>> lean.enterprise.middleware >>>>>>>>>> >>>>>>>>>> mobile: +94 77 2273550 >>>>>>>>>> blog: http://sajithr.blogspot.com/ >>>>>>>>>> <http://lk.linkedin.com/pub/shani-ranasinghe/34/111/ab> >>>>>>>>>> >>>>>>>>>> On Fri, Jun 24, 2016 at 10:34 AM, Malith Jayasinghe < >>>>>>>>>> [email protected]> wrote: >>>>>>>>>> >>>>>>>>>>> Hi Sajith, >>>>>>>>>>> >>>>>>>>>>> Could you please provide some details about how you are actually >>>>>>>>>>> doing these performance tests. For example, what is deployment >>>>>>>>>>> model? How >>>>>>>>>>> are you generating these workloads/events? >>>>>>>>>>> >>>>>>>>>>> Thanks >>>>>>>>>>> >>>>>>>>>>> Malith >>>>>>>>>>> >>>>>>>>>>> On Thu, Jun 23, 2016 at 11:28 AM, Sajith Ravindra < >>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>> >>>>>>>>>>>> Hi all, >>>>>>>>>>>> >>>>>>>>>>>> This is to give an update on the performance study we conducted >>>>>>>>>>>> on is-analytics server on last. The idea of this test round was to >>>>>>>>>>>> evaluate >>>>>>>>>>>> the performance of Siddhi queries used for is-analytics, therefore >>>>>>>>>>>> we >>>>>>>>>>>> disabled event stream persistence and spark for this test. >>>>>>>>>>>> >>>>>>>>>>>> This test was conducted on a standalone DAS server with Xms2g >>>>>>>>>>>> and Xmx4g. >>>>>>>>>>>> >>>>>>>>>>>> On the initial round when input TPS reaches ~20K, the server >>>>>>>>>>>> went OOM after consuming around 1M events. The reason for this was >>>>>>>>>>>> the >>>>>>>>>>>> events accumulated inside 7 1min time batch windows used inside. >>>>>>>>>>>> To >>>>>>>>>>>> overcome this we implemented an extension to siddhi which allows >>>>>>>>>>>> us to >>>>>>>>>>>> avoid duplicating the window. >>>>>>>>>>>> >>>>>>>>>>>> After removing duplicate windows the server was able to consume >>>>>>>>>>>> events at a rate of ~22K, but there were fluctuations (see the >>>>>>>>>>>> graph >>>>>>>>>>>> bellow) of the throughput. With analysis, we found that intense GC >>>>>>>>>>>> causes >>>>>>>>>>>> this. We suspect that this intense GC is caused when expiring a >>>>>>>>>>>> large >>>>>>>>>>>> number of events accumulated inside 1-minute window. To overcome >>>>>>>>>>>> this we >>>>>>>>>>>> are planning to batch events in 1-second windows and then >>>>>>>>>>>> accumulate >>>>>>>>>>>> 1second batches in 1 min window in order to stop accumulating a >>>>>>>>>>>> large >>>>>>>>>>>> number of events. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> As the next steps, we are planning to test the performance with >>>>>>>>>>>> event stream persistence and then move on to check the performance >>>>>>>>>>>> in DAS >>>>>>>>>>>> minimum HA mode. >>>>>>>>>>>> >>>>>>>>>>>> We will keep updating this thread with our findings. >>>>>>>>>>>> >>>>>>>>>>>> Please share your thought, suggestions on this. >>>>>>>>>>>> >>>>>>>>>>>> Thanks >>>>>>>>>>>> *,Sajith Ravindra* >>>>>>>>>>>> Senior Software Engineer >>>>>>>>>>>> WSO2 Inc.; http://wso2.com >>>>>>>>>>>> lean.enterprise.middleware >>>>>>>>>>>> >>>>>>>>>>>> mobile: +94 77 2273550 >>>>>>>>>>>> blog: http://sajithr.blogspot.com/ >>>>>>>>>>>> <http://lk.linkedin.com/pub/shani-ranasinghe/34/111/ab> >>>>>>>>>>>> >>>>>>>>>>>> _______________________________________________ >>>>>>>>>>>> Architecture mailing list >>>>>>>>>>>> [email protected] >>>>>>>>>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> Malith Jayasinghe >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> WSO2, Inc. (http://wso2.com) >>>>>>>>>>> Email : [email protected] >>>>>>>>>>> Mobile : 0770704040 >>>>>>>>>>> Lean . Enterprise . Middleware >>>>>>>>>>> >>>>>>>>>>> _______________________________________________ >>>>>>>>>>> Architecture mailing list >>>>>>>>>>> [email protected] >>>>>>>>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> _______________________________________________ >>>>>>>>>> Architecture mailing list >>>>>>>>>> [email protected] >>>>>>>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> Malith Jayasinghe >>>>>>>>> >>>>>>>>> >>>>>>>>> WSO2, Inc. (http://wso2.com) >>>>>>>>> Email : [email protected] >>>>>>>>> Mobile : 0770704040 >>>>>>>>> Lean . Enterprise . Middleware >>>>>>>>> >>>>>>>>> _______________________________________________ >>>>>>>>> Architecture mailing list >>>>>>>>> [email protected] >>>>>>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> Architecture mailing list >>>>>>>> [email protected] >>>>>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>>>>>> >>>>>>>> >>>>>>> _______________________________________________ >>>>>>> Architecture mailing list >>>>>>> [email protected] >>>>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> *V. Mohanadarshan* >>>>>> *Associate Tech Lead,* >>>>>> *Data Technologies Team,* >>>>>> *WSO2, Inc. http://wso2.com <http://wso2.com> * >>>>>> *lean.enterprise.middleware.* >>>>>> >>>>>> email: [email protected] >>>>>> phone:(+94) 771117673 >>>>>> >>>>>> _______________________________________________ >>>>>> Architecture mailing list >>>>>> [email protected] >>>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> Thanks & Regards >>>>> >>>>> Iranga Muthuthanthri >>>>> (M) -0777-255773 >>>>> Team Product Management >>>>> >>>>> >>>>> _______________________________________________ >>>>> Architecture mailing list >>>>> [email protected] >>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>>> >>>>> >>>> _______________________________________________ >>>> Architecture mailing list >>>> [email protected] >>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>> >>>> >>> >>> >>> -- >>> *V. Mohanadarshan* >>> *Associate Tech Lead,* >>> *Data Technologies Team,* >>> *WSO2, Inc. http://wso2.com <http://wso2.com> * >>> *lean.enterprise.middleware.* >>> >>> email: [email protected] >>> phone:(+94) 771117673 >>> >>> _______________________________________________ >>> Architecture mailing list >>> [email protected] >>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>> >>> >> >> >> -- >> Dulitha Wijewantha (Chan) >> Software Engineer - Mobile Development >> WSO2 Inc >> Lean.Enterprise.Middleware >> * ~Email [email protected] <[email protected]>* >> * ~Mobile +94712112165 <%2B94712112165>* >> * ~Website dulitha.me <http://dulitha.me>* >> * ~Twitter @dulitharw <https://twitter.com/dulitharw>* >> *~Github @dulichan <https://github.com/dulichan>* >> *~SO @chan <http://stackoverflow.com/users/813471/chan>* >> >> _______________________________________________ >> Architecture mailing list >> [email protected] >> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >> >> > > > -- > *V. Mohanadarshan* > *Associate Tech Lead,* > *Data Technologies Team,* > *WSO2, Inc. http://wso2.com <http://wso2.com> * > *lean.enterprise.middleware.* > > email: [email protected] > phone:(+94) 771117673 > > _______________________________________________ > Architecture mailing list > [email protected] > https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture > >
_______________________________________________ Architecture mailing list [email protected] https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
