Hi Malith, Thanks for the explanation. > > I would expect some variation in the throughput. The aim should be > minimize the variation in the throughput (while maintaining the throughput > at its highest level). > Agreed. Actually, our expectation is to minimize the fluctuation and to increase the throughput.
> > It is possible to measure the latency as well? > In IS-Analytics we don't generate any output events or alerts currently. Therefore, we don't calculate the latency. What we do is summarize events using siddhi queries and persist them in a DB and further summarize persisted data using Spark and then displayed through dashbaords. Thanks *,Sajith Ravindra* Senior Software Engineer WSO2 Inc.; http://wso2.com lean.enterprise.middleware mobile: +94 77 2273550 blog: http://sajithr.blogspot.com/ <http://lk.linkedin.com/pub/shani-ranasinghe/34/111/ab> On Fri, Jun 24, 2016 at 1:57 PM, Malith Jayasinghe <[email protected]> wrote: > Hi Sajith, > > Thanks for the explanation. > > I would expect some variation in the throughput. The aim should be > minimize the variation in the throughput (while maintaining the throughput > at its highest level). > > It is possible to measure the latency as well? > > regards > > Malith > > > > On Thu, Jun 23, 2016 at 11:13 PM, Sajith Ravindra <[email protected]> > wrote: > >> Hi Malith, >> >> The deployment is just a standalone DAS server, and we are planning to do >> a test for HA deployment in recent future. >> >> The workload is generated by a .csv data file which has 100K sample >> events, 10M events are generated by iterating through the same data set 100 >> times. But we keep increasing the timestamp. A simple thrift client is used >> to publish data. >> >> >> >> Thanks >> *,Sajith Ravindra* >> Senior Software Engineer >> WSO2 Inc.; http://wso2.com >> lean.enterprise.middleware >> >> mobile: +94 77 2273550 >> blog: http://sajithr.blogspot.com/ >> <http://lk.linkedin.com/pub/shani-ranasinghe/34/111/ab> >> >> On Fri, Jun 24, 2016 at 10:34 AM, Malith Jayasinghe <[email protected]> >> wrote: >> >>> Hi Sajith, >>> >>> Could you please provide some details about how you are actually doing >>> these performance tests. For example, what is deployment model? How are >>> you generating these workloads/events? >>> >>> Thanks >>> >>> Malith >>> >>> On Thu, Jun 23, 2016 at 11:28 AM, Sajith Ravindra <[email protected]> >>> wrote: >>> >>>> Hi all, >>>> >>>> This is to give an update on the performance study we conducted on >>>> is-analytics server on last. The idea of this test round was to evaluate >>>> the performance of Siddhi queries used for is-analytics, therefore we >>>> disabled event stream persistence and spark for this test. >>>> >>>> This test was conducted on a standalone DAS server with Xms2g and >>>> Xmx4g. >>>> >>>> On the initial round when input TPS reaches ~20K, the server went OOM >>>> after consuming around 1M events. The reason for this was the events >>>> accumulated inside 7 1min time batch windows used inside. To overcome this >>>> we implemented an extension to siddhi which allows us to avoid duplicating >>>> the window. >>>> >>>> After removing duplicate windows the server was able to consume events >>>> at a rate of ~22K, but there were fluctuations (see the graph bellow) of >>>> the throughput. With analysis, we found that intense GC causes this. We >>>> suspect that this intense GC is caused when expiring a large number of >>>> events accumulated inside 1-minute window. To overcome this we are >>>> planning to batch events in 1-second windows and then accumulate 1second >>>> batches in 1 min window in order to stop accumulating a large number of >>>> events. >>>> >>>> >>>> As the next steps, we are planning to test the performance with event >>>> stream persistence and then move on to check the performance in DAS minimum >>>> HA mode. >>>> >>>> We will keep updating this thread with our findings. >>>> >>>> Please share your thought, suggestions on this. >>>> >>>> Thanks >>>> *,Sajith Ravindra* >>>> Senior Software Engineer >>>> WSO2 Inc.; http://wso2.com >>>> lean.enterprise.middleware >>>> >>>> mobile: +94 77 2273550 >>>> blog: http://sajithr.blogspot.com/ >>>> <http://lk.linkedin.com/pub/shani-ranasinghe/34/111/ab> >>>> >>>> _______________________________________________ >>>> Architecture mailing list >>>> [email protected] >>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>> >>>> >>> >>> >>> -- >>> Malith Jayasinghe >>> >>> >>> WSO2, Inc. (http://wso2.com) >>> Email : [email protected] >>> Mobile : 0770704040 >>> Lean . Enterprise . Middleware >>> >>> _______________________________________________ >>> Architecture mailing list >>> [email protected] >>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>> >>> >> >> _______________________________________________ >> Architecture mailing list >> [email protected] >> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >> >> > > > -- > Malith Jayasinghe > > > WSO2, Inc. (http://wso2.com) > Email : [email protected] > Mobile : 0770704040 > Lean . Enterprise . Middleware > > _______________________________________________ > Architecture mailing list > [email protected] > https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture > >
_______________________________________________ Architecture mailing list [email protected] https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
