Hi Sajith, Do you have plot which shows how the TPS varies with the number of events (for the new results)? I would like to see if there is an improvement (reduction) in the variation in the TPS.
Also what are the main bottlenecks and performance issues which have been fixed? Thanks On Mon, Jul 11, 2016 at 4:11 PM, Sajith Ravindra <[email protected]> wrote: > We have completed the performance test for IS-Analytics for a Single DAS > node after fixing the bottlenecks and performance issues we found in the > initial testing. Following are the results, > > *Case 1 : *Not persisting any data streams only siddhi queries were > running. TPS : 150K. > > *Case 2* : Incoming events are summarized using siddhi queries as the > first level of summarization only those summarized event streams are > persisted. TPS : 94K > > *Case 3* : All streams including incoming raw event stream is also > persisted. TPS 11K > > This test was conducted on a single laptop with 16GB of memory and 2.70Gz > 8 core machine. MySQL was also running in the same machine. > > > Thanks > *,Sajith Ravindra* > Senior Software Engineer > WSO2 Inc.; http://wso2.com > lean.enterprise.middleware > > mobile: +94 77 2273550 > blog: http://sajithr.blogspot.com/ > <http://lk.linkedin.com/pub/shani-ranasinghe/34/111/ab> > > On Thu, Jun 30, 2016 at 7:25 PM, Mohanadarshan Vivekanandalingam < > [email protected]> wrote: > >> >> >> On Wed, Jun 29, 2016 at 9:23 PM, Dulitha Wijewantha <[email protected]> >> wrote: >> >>> Hi Mohan/Sajith, >>> Can you please explain what this particular scenario was in IS? Is this >>> a counting scenario on a window? This would provide input for others who >>> are writing real time analytics on CEP. >>> >> >> Yes, Here we have mainly focused on some aggregations like Sum with >> external time batch windows.. Queries can be found in [1] >> >> [1] >> https://github.com/wso2/analytics-is/blob/master/features/org.wso2.carbon.analytics.is.feature/src/main/capp/AuthenticationAnalyticsExecutionPlan/IsAnalytics-ExecutionPlan-AuthenticationData.siddhiql >> >> Thanks, >> Mohan >> >> >>> >>> Cheers~ >>> >>> On Mon, Jun 27, 2016 at 5:50 PM, Mohanadarshan Vivekanandalingam < >>> [email protected]> wrote: >>> >>>> >>>> >>>> On Mon, Jun 27, 2016 at 8:03 PM, Seshika Fernando <[email protected]> >>>> wrote: >>>> >>>>> Iranga, >>>>> The optimization applies to siddhi queries that were written for IS >>>>> analytics. Other product analytics would have different queries so this is >>>>> not applicable there. >>>>> This is not a siddhi level change, rather an optimization of the >>>>> previously written queries. >>>>> >>>> Yes, above per-second and per-minute design is applied to some other >>>> analytics product as well but it is not generic and too specific for the >>>> usecase and query as mentioned by Seshi.. >>>> >>>> Thanks, >>>> Mohan >>>> >>>> >>>>> Seshi >>>>> On 27 Jun 2016 16:48, "Iranga Muthuthanthri" <[email protected]> wrote: >>>>> >>>>>> >>>>>> >>>>>> On Mon, Jun 27, 2016 at 4:35 PM, Mohanadarshan Vivekanandalingam < >>>>>> [email protected]> wrote: >>>>>> >>>>>>> >>>>>>> >>>>>>> On Mon, Jun 27, 2016 at 4:30 PM, Seshika Fernando <[email protected]> >>>>>>> wrote: >>>>>>> >>>>>>>> Hey saith, >>>>>>>> >>>>>>>> This is great. So when you removed the duplicate windows, there >>>>>>>> were no OOM issues? >>>>>>>> >>>>>>> Yes, we have introduced per-second windows and eventually per-minute >>>>>>> event count get reduced. Then, there will not be an OOM.. >>>>>>> >>>>>> >>>>>> is this specific for IS-Analytics or common to all other 'product >>>>>> analytics'. >>>>>> >>>>>> >>>>>>> Latest number is also seems good.. SajithR, can share those numbers.. >>>>>>> >>>>>>> Thanks, >>>>>>> Mohan >>>>>>> >>>>>>> >>>>>>>> Seshi >>>>>>>> On 24 Jun 2016 14:28, "Sajith Ravindra" <[email protected]> wrote: >>>>>>>> >>>>>>>>> Hi Malith, >>>>>>>>> >>>>>>>>> Thanks for the explanation. >>>>>>>>>> >>>>>>>>>> I would expect some variation in the throughput. The aim should >>>>>>>>>> be minimize the variation in the throughput (while maintaining the >>>>>>>>>> throughput at its highest level). >>>>>>>>>> >>>>>>>>> Agreed. Actually, our expectation is to minimize the fluctuation >>>>>>>>> and to increase the throughput. >>>>>>>>> >>>>>>>>>> >>>>>>>>>> It is possible to measure the latency as well? >>>>>>>>>> >>>>>>>>> In IS-Analytics we don't generate any output events or alerts >>>>>>>>> currently. Therefore, we don't calculate the latency. What we do is >>>>>>>>> summarize events using siddhi queries and persist them in a DB and >>>>>>>>> further >>>>>>>>> summarize persisted data using Spark and then displayed through >>>>>>>>> dashbaords. >>>>>>>>> >>>>>>>>> >>>>>>>>> Thanks >>>>>>>>> *,Sajith Ravindra* >>>>>>>>> Senior Software Engineer >>>>>>>>> WSO2 Inc.; http://wso2.com >>>>>>>>> lean.enterprise.middleware >>>>>>>>> >>>>>>>>> mobile: +94 77 2273550 >>>>>>>>> blog: http://sajithr.blogspot.com/ >>>>>>>>> <http://lk.linkedin.com/pub/shani-ranasinghe/34/111/ab> >>>>>>>>> >>>>>>>>> On Fri, Jun 24, 2016 at 1:57 PM, Malith Jayasinghe < >>>>>>>>> [email protected]> wrote: >>>>>>>>> >>>>>>>>>> Hi Sajith, >>>>>>>>>> >>>>>>>>>> Thanks for the explanation. >>>>>>>>>> >>>>>>>>>> I would expect some variation in the throughput. The aim should >>>>>>>>>> be minimize the variation in the throughput (while maintaining the >>>>>>>>>> throughput at its highest level). >>>>>>>>>> >>>>>>>>>> It is possible to measure the latency as well? >>>>>>>>>> >>>>>>>>>> regards >>>>>>>>>> >>>>>>>>>> Malith >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Thu, Jun 23, 2016 at 11:13 PM, Sajith Ravindra < >>>>>>>>>> [email protected]> wrote: >>>>>>>>>> >>>>>>>>>>> Hi Malith, >>>>>>>>>>> >>>>>>>>>>> The deployment is just a standalone DAS server, and we are >>>>>>>>>>> planning to do a test for HA deployment in recent future. >>>>>>>>>>> >>>>>>>>>>> The workload is generated by a .csv data file which has 100K >>>>>>>>>>> sample events, 10M events are generated by iterating through the >>>>>>>>>>> same data >>>>>>>>>>> set 100 times. But we keep increasing the timestamp. A simple >>>>>>>>>>> thrift client >>>>>>>>>>> is used to publish data. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Thanks >>>>>>>>>>> *,Sajith Ravindra* >>>>>>>>>>> Senior Software Engineer >>>>>>>>>>> WSO2 Inc.; http://wso2.com >>>>>>>>>>> lean.enterprise.middleware >>>>>>>>>>> >>>>>>>>>>> mobile: +94 77 2273550 >>>>>>>>>>> blog: http://sajithr.blogspot.com/ >>>>>>>>>>> <http://lk.linkedin.com/pub/shani-ranasinghe/34/111/ab> >>>>>>>>>>> >>>>>>>>>>> On Fri, Jun 24, 2016 at 10:34 AM, Malith Jayasinghe < >>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>> >>>>>>>>>>>> Hi Sajith, >>>>>>>>>>>> >>>>>>>>>>>> Could you please provide some details about how you are >>>>>>>>>>>> actually doing these performance tests. For example, what is >>>>>>>>>>>> deployment >>>>>>>>>>>> model? How are you generating these workloads/events? >>>>>>>>>>>> >>>>>>>>>>>> Thanks >>>>>>>>>>>> >>>>>>>>>>>> Malith >>>>>>>>>>>> >>>>>>>>>>>> On Thu, Jun 23, 2016 at 11:28 AM, Sajith Ravindra < >>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> Hi all, >>>>>>>>>>>>> >>>>>>>>>>>>> This is to give an update on the performance study we >>>>>>>>>>>>> conducted on is-analytics server on last. The idea of this test >>>>>>>>>>>>> round was >>>>>>>>>>>>> to evaluate the performance of Siddhi queries used for >>>>>>>>>>>>> is-analytics, >>>>>>>>>>>>> therefore we disabled event stream persistence and spark for this >>>>>>>>>>>>> test. >>>>>>>>>>>>> >>>>>>>>>>>>> This test was conducted on a standalone DAS server with Xms2g >>>>>>>>>>>>> and Xmx4g. >>>>>>>>>>>>> >>>>>>>>>>>>> On the initial round when input TPS reaches ~20K, the server >>>>>>>>>>>>> went OOM after consuming around 1M events. The reason for this >>>>>>>>>>>>> was the >>>>>>>>>>>>> events accumulated inside 7 1min time batch windows used inside. >>>>>>>>>>>>> To >>>>>>>>>>>>> overcome this we implemented an extension to siddhi which allows >>>>>>>>>>>>> us to >>>>>>>>>>>>> avoid duplicating the window. >>>>>>>>>>>>> >>>>>>>>>>>>> After removing duplicate windows the server was able to >>>>>>>>>>>>> consume events at a rate of ~22K, but there were fluctuations >>>>>>>>>>>>> (see the >>>>>>>>>>>>> graph bellow) of the throughput. With analysis, we found that >>>>>>>>>>>>> intense GC >>>>>>>>>>>>> causes this. We suspect that this intense GC is caused when >>>>>>>>>>>>> expiring a >>>>>>>>>>>>> large number of events accumulated inside 1-minute window. To >>>>>>>>>>>>> overcome >>>>>>>>>>>>> this we are planning to batch events in 1-second windows and then >>>>>>>>>>>>> accumulate 1second batches in 1 min window in order to stop >>>>>>>>>>>>> accumulating a >>>>>>>>>>>>> large number of events. >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> As the next steps, we are planning to test the performance >>>>>>>>>>>>> with event stream persistence and then move on to check the >>>>>>>>>>>>> performance in >>>>>>>>>>>>> DAS minimum HA mode. >>>>>>>>>>>>> >>>>>>>>>>>>> We will keep updating this thread with our findings. >>>>>>>>>>>>> >>>>>>>>>>>>> Please share your thought, suggestions on this. >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks >>>>>>>>>>>>> *,Sajith Ravindra* >>>>>>>>>>>>> Senior Software Engineer >>>>>>>>>>>>> WSO2 Inc.; http://wso2.com >>>>>>>>>>>>> lean.enterprise.middleware >>>>>>>>>>>>> >>>>>>>>>>>>> mobile: +94 77 2273550 >>>>>>>>>>>>> blog: http://sajithr.blogspot.com/ >>>>>>>>>>>>> <http://lk.linkedin.com/pub/shani-ranasinghe/34/111/ab> >>>>>>>>>>>>> >>>>>>>>>>>>> _______________________________________________ >>>>>>>>>>>>> Architecture mailing list >>>>>>>>>>>>> [email protected] >>>>>>>>>>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> -- >>>>>>>>>>>> Malith Jayasinghe >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> WSO2, Inc. (http://wso2.com) >>>>>>>>>>>> Email : [email protected] >>>>>>>>>>>> Mobile : 0770704040 >>>>>>>>>>>> Lean . Enterprise . Middleware >>>>>>>>>>>> >>>>>>>>>>>> _______________________________________________ >>>>>>>>>>>> Architecture mailing list >>>>>>>>>>>> [email protected] >>>>>>>>>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> _______________________________________________ >>>>>>>>>>> Architecture mailing list >>>>>>>>>>> [email protected] >>>>>>>>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> Malith Jayasinghe >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> WSO2, Inc. (http://wso2.com) >>>>>>>>>> Email : [email protected] >>>>>>>>>> Mobile : 0770704040 >>>>>>>>>> Lean . Enterprise . Middleware >>>>>>>>>> >>>>>>>>>> _______________________________________________ >>>>>>>>>> Architecture mailing list >>>>>>>>>> [email protected] >>>>>>>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>> _______________________________________________ >>>>>>>>> Architecture mailing list >>>>>>>>> [email protected] >>>>>>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>>>>>>> >>>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> Architecture mailing list >>>>>>>> [email protected] >>>>>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> *V. Mohanadarshan* >>>>>>> *Associate Tech Lead,* >>>>>>> *Data Technologies Team,* >>>>>>> *WSO2, Inc. http://wso2.com <http://wso2.com> * >>>>>>> *lean.enterprise.middleware.* >>>>>>> >>>>>>> email: [email protected] >>>>>>> phone:(+94) 771117673 >>>>>>> >>>>>>> _______________________________________________ >>>>>>> Architecture mailing list >>>>>>> [email protected] >>>>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Thanks & Regards >>>>>> >>>>>> Iranga Muthuthanthri >>>>>> (M) -0777-255773 >>>>>> Team Product Management >>>>>> >>>>>> >>>>>> _______________________________________________ >>>>>> Architecture mailing list >>>>>> [email protected] >>>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>>>> >>>>>> >>>>> _______________________________________________ >>>>> Architecture mailing list >>>>> [email protected] >>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>>> >>>>> >>>> >>>> >>>> -- >>>> *V. Mohanadarshan* >>>> *Associate Tech Lead,* >>>> *Data Technologies Team,* >>>> *WSO2, Inc. http://wso2.com <http://wso2.com> * >>>> *lean.enterprise.middleware.* >>>> >>>> email: [email protected] >>>> phone:(+94) 771117673 >>>> >>>> _______________________________________________ >>>> Architecture mailing list >>>> [email protected] >>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>> >>>> >>> >>> >>> -- >>> Dulitha Wijewantha (Chan) >>> Software Engineer - Mobile Development >>> WSO2 Inc >>> Lean.Enterprise.Middleware >>> * ~Email [email protected] <[email protected]>* >>> * ~Mobile +94712112165 <%2B94712112165>* >>> * ~Website dulitha.me <http://dulitha.me>* >>> * ~Twitter @dulitharw <https://twitter.com/dulitharw>* >>> *~Github @dulichan <https://github.com/dulichan>* >>> *~SO @chan <http://stackoverflow.com/users/813471/chan>* >>> >>> _______________________________________________ >>> Architecture mailing list >>> [email protected] >>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>> >>> >> >> >> -- >> *V. Mohanadarshan* >> *Associate Tech Lead,* >> *Data Technologies Team,* >> *WSO2, Inc. http://wso2.com <http://wso2.com> * >> *lean.enterprise.middleware.* >> >> email: [email protected] >> phone:(+94) 771117673 >> >> _______________________________________________ >> Architecture mailing list >> [email protected] >> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >> >> > > _______________________________________________ > Architecture mailing list > [email protected] > https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture > > -- Malith Jayasinghe WSO2, Inc. (http://wso2.com) Email : [email protected] Mobile : 0770704040 Lean . Enterprise . Middleware
_______________________________________________ Architecture mailing list [email protected] https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
