Hi Sajith, Thanks for the explanation.
I would expect some variation in the throughput. The aim should be minimize the variation in the throughput (while maintaining the throughput at its highest level). It is possible to measure the latency as well? regards Malith On Thu, Jun 23, 2016 at 11:13 PM, Sajith Ravindra <[email protected]> wrote: > Hi Malith, > > The deployment is just a standalone DAS server, and we are planning to do > a test for HA deployment in recent future. > > The workload is generated by a .csv data file which has 100K sample > events, 10M events are generated by iterating through the same data set 100 > times. But we keep increasing the timestamp. A simple thrift client is used > to publish data. > > > > Thanks > *,Sajith Ravindra* > Senior Software Engineer > WSO2 Inc.; http://wso2.com > lean.enterprise.middleware > > mobile: +94 77 2273550 > blog: http://sajithr.blogspot.com/ > <http://lk.linkedin.com/pub/shani-ranasinghe/34/111/ab> > > On Fri, Jun 24, 2016 at 10:34 AM, Malith Jayasinghe <[email protected]> > wrote: > >> Hi Sajith, >> >> Could you please provide some details about how you are actually doing >> these performance tests. For example, what is deployment model? How are >> you generating these workloads/events? >> >> Thanks >> >> Malith >> >> On Thu, Jun 23, 2016 at 11:28 AM, Sajith Ravindra <[email protected]> >> wrote: >> >>> Hi all, >>> >>> This is to give an update on the performance study we conducted on >>> is-analytics server on last. The idea of this test round was to evaluate >>> the performance of Siddhi queries used for is-analytics, therefore we >>> disabled event stream persistence and spark for this test. >>> >>> This test was conducted on a standalone DAS server with Xms2g and Xmx4g. >>> >>> On the initial round when input TPS reaches ~20K, the server went OOM >>> after consuming around 1M events. The reason for this was the events >>> accumulated inside 7 1min time batch windows used inside. To overcome this >>> we implemented an extension to siddhi which allows us to avoid duplicating >>> the window. >>> >>> After removing duplicate windows the server was able to consume events >>> at a rate of ~22K, but there were fluctuations (see the graph bellow) of >>> the throughput. With analysis, we found that intense GC causes this. We >>> suspect that this intense GC is caused when expiring a large number of >>> events accumulated inside 1-minute window. To overcome this we are >>> planning to batch events in 1-second windows and then accumulate 1second >>> batches in 1 min window in order to stop accumulating a large number of >>> events. >>> >>> >>> As the next steps, we are planning to test the performance with event >>> stream persistence and then move on to check the performance in DAS minimum >>> HA mode. >>> >>> We will keep updating this thread with our findings. >>> >>> Please share your thought, suggestions on this. >>> >>> Thanks >>> *,Sajith Ravindra* >>> Senior Software Engineer >>> WSO2 Inc.; http://wso2.com >>> lean.enterprise.middleware >>> >>> mobile: +94 77 2273550 >>> blog: http://sajithr.blogspot.com/ >>> <http://lk.linkedin.com/pub/shani-ranasinghe/34/111/ab> >>> >>> _______________________________________________ >>> Architecture mailing list >>> [email protected] >>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>> >>> >> >> >> -- >> Malith Jayasinghe >> >> >> WSO2, Inc. (http://wso2.com) >> Email : [email protected] >> Mobile : 0770704040 >> Lean . Enterprise . Middleware >> >> _______________________________________________ >> Architecture mailing list >> [email protected] >> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >> >> > > _______________________________________________ > Architecture mailing list > [email protected] > https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture > > -- Malith Jayasinghe WSO2, Inc. (http://wso2.com) Email : [email protected] Mobile : 0770704040 Lean . Enterprise . Middleware
_______________________________________________ Architecture mailing list [email protected] https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
