Hi Chamil , Great , Seems there is no impact on performance by integrating carbon-metrics and it may operates in asynchronous manner .Shall we look in to how we can visualize this data and how we can publish those to different Services like DAS .
thanks On Mon, Feb 8, 2016 at 7:06 AM, Chamil Elladeniya <[email protected]> wrote: > Hi all, > > As the previous test results were inconsistent, I did the test again using > wrk to test the above scenario. > > Remote environment testing Using wrk > Original GW > Integrated GW > # of threads Connections Requests per second [#/sec] Time per request > [ms] > # of threads Connections Requests per second [#/sec] Time per request > [ms] > 10 100 13721.18 7.98 > 10 100 14802.58 8.55 > 10 200 11884.68 18.24 > 10 200 14997.99 15.74 > 10 300 11735.99 26.98 > 10 300 13769.9 26.99 > 10 400 13257.39 32.68 > 10 400 14569.7 33.23 > 10 500 12124.66 44.14 > 10 500 14016.72 42.47 > 10 600 13575.55 50.63 > 10 600 13117.01 55.74 > 10 700 12891.08 66.33 > 10 700 12848.27 66.64 > 10 800 13045.18 71.42 > 10 800 13391.08 70.43 > 10 900 12990.17 86.4 > 10 900 13607.67 88.85 > 10 1000 7631.25 87.14 > 10 1000 8390.08 91.55 > > Above average values are based on these individual test values[1]. > > [1] > https://docs.google.com/a/wso2.com/spreadsheets/d/1LhOjuaVFlv3AVDN-gYOW0S9rhv5kyEqczPhQ106bTZE/edit?usp=sharing > > On Fri, Feb 5, 2016 at 8:07 AM, Nadeeshaan Gunasinghe <[email protected] > > wrote: > >> Hi Chamil, >> >> It would be better if we can run another round of perf testing before we >> come to a conclusion here. >> >> >> *Nadeeshaan Gunasinghe* >> Software Engineer, WSO2 Inc. http://wso2.com >> +94770596754 | [email protected] | Skype: nadeeshaan.gunasinghe >> <#-1155064647_1759133060_> >> <http://www.facebook.com/nadeeshaan.gunasinghe> >> <http://lk.linkedin.com/in/nadeeshaan> <http://twitter.com/Nadeeshaan> >> <http://nadeeshaan.blogspot.com/> >> Get a signature like this: Click here! >> <http://ws-promos.appspot.com/r?rdata=eyJydXJsIjogImh0dHA6Ly93d3cud2lzZXN0YW1wLmNvbS9lbWFpbC1pbnN0YWxsP3dzX25jaWQ9NjcyMjk0MDA4JnV0bV9zb3VyY2U9ZXh0ZW5zaW9uJnV0bV9tZWRpdW09ZW1haWwmdXRtX2NhbXBhaWduPXByb21vXzU3MzI1Njg1NDg3Njk3OTIiLCAiZSI6ICI1NzMyNTY4NTQ4NzY5NzkyIn0=&ws_random_number=> >> >> On Fri, Feb 5, 2016 at 7:47 AM, Isuru Ranawaka <[email protected]> wrote: >> >>> Hi Chamila, >>> >>> Results should be consisted and we have seen consisted results while >>> doing perf testing . Here we need to ensure whether there are other >>> processes running on server which can lead to degrade performance while >>> doing load testing and as Kasun mentioned and we can run perf testing by >>> enabling JFR. >>> >>> On Thu, Feb 4, 2016 at 10:20 PM, Kasun Indrasiri <[email protected]> wrote: >>> >>>> Yes. We need to run this on perf testing environments and do a detailed >>>> analysis of JFRs. Also, we better use 'wrk' instead of ab. >>>> >>>> I guess the results were quite consistent with GW's performance test >>>> suite. >>>> @Ranawaka can you please comment on the behavior of perf testing >>>> results. >>>> >>>> On Wed, Feb 3, 2016 at 8:32 PM, Isuru Perera <[email protected]> wrote: >>>> >>>>> The results are not consistent. For example, in Original GW, the RPS >>>>> is very low in 500 & 900 concurrency. Therefore we cannot exactly tell >>>>> whether there is a considerable performance impact when using Carbon >>>>> Metrics. >>>>> >>>>> We first need to make sure there are no issues in the performance >>>>> testing environment. >>>>> >>>>> During the previous Gateway testing, did anyone notice such >>>>> inconsistent behaviour in different concurrency levels? >>>>> >>>>> On Wed, Feb 3, 2016 at 7:02 PM, Chamil Elladeniya <[email protected]> >>>>> wrote: >>>>> >>>>>> Adding the team >>>>>> >>>>>> >>>>>> On Wed, Feb 3, 2016 at 6:59 PM, Chamil Elladeniya <[email protected]> >>>>>> wrote: >>>>>> >>>>>>> Hi all, >>>>>>> This is the perf test results which performed in order to compare >>>>>>> metrics integrated GW and the pure. >>>>>>> >>>>>>> Original GW >>>>>>> Integrated GW >>>>>>> # of requests Concurrency Requests per second [#/sec] Time per >>>>>>> request [ms] >>>>>>> # of requests Concurrency Requests per second [#/sec] Time per >>>>>>> request [ms] >>>>>>> 1000000 100 15543.07 6.434 >>>>>>> 1000000 100 15219.26 6.571 >>>>>>> 1000000 200 15633.71 12.793 >>>>>>> 1000000 200 12739.49 15.699 >>>>>>> 1000000 300 14406.71 20.824 >>>>>>> 1000000 300 15312.92 19.591 >>>>>>> 1000000 400 15116.58 26.461 >>>>>>> 1000000 400 12272.92 32.592 >>>>>>> 1000000 500 7053.54 70.886 >>>>>>> 1000000 500 6697.34 74.92 >>>>>>> 1000000 600 9497.65 63.173 >>>>>>> 1000000 600 11559.41 59.57 >>>>>>> 1000000 700 14322.05 48.876 >>>>>>> 1000000 700 10049.72 102.6 >>>>>>> 1000000 800 11374.63 70.332 >>>>>>> 1000000 800 10578.78 93.599 >>>>>>> 1000000 900 4838.48 186.009 >>>>>>> 1000000 900 3123.29 288.158 >>>>>>> 1000000 1000 9270.24 150.28 >>>>>>> 1000000 1000 9181.02 153.23 >>>>>>> 1000000 1100 13958.91 78.803 >>>>>>> 1000000 1100 13573.83 81.038 >>>>>>> 1000000 1300 14110.01 92.133 >>>>>>> 1000000 1300 5978.51 217.446 >>>>>>> 1000000 1400 13769.77 101.672 >>>>>>> 1000000 1400 timeout >>>>>>> 1000000 1500 5573.45 404.5 >>>>>>> 1000000 1500 timeout >>>>>>> >>>>>>> >>>>>>> Above values are taken as an average of tests per case. Integrated >>>>>>> GW results timeout when the concurrency level is above 1400 >>>>>>> >>>>>>> Thank you! >>>>>>> >>>>>>> On Tue, Jan 26, 2016 at 10:53 AM, Chamil Elladeniya < >>>>>>> [email protected]> wrote: >>>>>>> >>>>>>>> Hi all, >>>>>>>> >>>>>>>> Currently I'm tasked with implementing latency metrics calculation >>>>>>>> feature according to the proposed architecture [1]. So far I have >>>>>>>> integrated carbon-metrics and working on load testing to check if >>>>>>>> there is >>>>>>>> any performance degradation of GW. >>>>>>>> >>>>>>>> [1] [Architecture] Implementing Latency Metrics Calculation >>>>>>>> Feature in GW >>>>>>>> >>>>>>>> Thank you! >>>>>>>> >>>>>>>> On Thu, Dec 17, 2015 at 10:46 AM, Viraj Senevirathne < >>>>>>>> [email protected]> wrote: >>>>>>>> >>>>>>>>> Hi All, >>>>>>>>> >>>>>>>>> In ESB 4.10.0 we are introducing new statistic feature which lets >>>>>>>>> user drill down service level statistics. >>>>>>>>> >>>>>>>>> So for higher level statistics we can include, >>>>>>>>> >>>>>>>>> - Avg,Min, Maximum Mediation times for each service >>>>>>>>> - Statistics of each endpoints >>>>>>>>> - Allow users to enable and disable statistics for each >>>>>>>>> components >>>>>>>>> - Faults encounters while mediation for each service >>>>>>>>> >>>>>>>>> These are some extra parameters that exists in current transport >>>>>>>>> latency parameters. I think it would be better to incorporate >>>>>>>>> following >>>>>>>>> parameters too. >>>>>>>>> >>>>>>>>> - Parameters >>>>>>>>> - Messages Received. >>>>>>>>> - Requests received. >>>>>>>>> - Responses sent >>>>>>>>> - Fault in Receiving >>>>>>>>> - Faults in Sending >>>>>>>>> - Min, Max, Avg message size sent >>>>>>>>> - Min, Max, Avg message size received >>>>>>>>> - Bytes Received >>>>>>>>> - Bytes received >>>>>>>>> - Timeouts in Receiving >>>>>>>>> - Timeouts in Sending >>>>>>>>> - Active Thread Count >>>>>>>>> - Last Reset Time >>>>>>>>> - Statistics Views for Daily, Hourly, by minutes ( This may be >>>>>>>>> optional) >>>>>>>>> >>>>>>>>> >>>>>>>>> *Operations* >>>>>>>>> >>>>>>>>> - Reset Statistics >>>>>>>>> >>>>>>>>> >>>>>>>>> Thank You, >>>>>>>>> >>>>>>>>> On Thu, Dec 17, 2015 at 10:20 AM, Kasun Indrasiri <[email protected]> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> We may also need a bit of high level stats too.. For instance >>>>>>>>>> things we have included in ESB 4.10. >>>>>>>>>> >>>>>>>>>> On Thu, Dec 17, 2015 at 10:17 AM, Nadeeshaan Gunasinghe < >>>>>>>>>> [email protected]> wrote: >>>>>>>>>> >>>>>>>>>>> Hi all, >>>>>>>>>>> It has been a requirement to implement a feature for keeping >>>>>>>>>>> track of the various types of latency metrics in WSO2 GW. At the >>>>>>>>>>> moment I >>>>>>>>>>> am involved in implementing this latency metrics calculation feature >>>>>>>>>>> according to the architecture proposed at [1]. >>>>>>>>>>> As the first step I am capturing the raw data required for >>>>>>>>>>> calculating various latency values. These raw data being collected >>>>>>>>>>> as >>>>>>>>>>> follows at the moment, >>>>>>>>>>> >>>>>>>>>>> *Server Side* >>>>>>>>>>> >>>>>>>>>>> - Source Connection Creation time >>>>>>>>>>> - Source Connection life time >>>>>>>>>>> - Request header read time >>>>>>>>>>> - Request body read time >>>>>>>>>>> - Request read time >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> *Client Side* >>>>>>>>>>> >>>>>>>>>>> - Client connection creation time >>>>>>>>>>> - Client Connection life time >>>>>>>>>>> - Response header read time >>>>>>>>>>> - Response body read time >>>>>>>>>>> - Response read time >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> I am going to keep track of these raw data and then transport >>>>>>>>>>> these data through the carbon message, as the initial step. Then a >>>>>>>>>>> latency >>>>>>>>>>> calculation engine is going to be implemented to calculate the >>>>>>>>>>> various >>>>>>>>>>> types of latency values such as, >>>>>>>>>>> >>>>>>>>>>> - Average Throughput of a connection >>>>>>>>>>> - Average Latency of a connection >>>>>>>>>>> - Average jitter of a connection >>>>>>>>>>> - Message build time >>>>>>>>>>> - Message encoding time >>>>>>>>>>> - Message mediation time >>>>>>>>>>> - etc >>>>>>>>>>> >>>>>>>>>>> Then a data publisher component is going to be implemented for >>>>>>>>>>> publishing data to JMX and DAS. >>>>>>>>>>> >>>>>>>>>>> During the implementation additional raw data will be needed to >>>>>>>>>>> be captured according to the type of metrics we are going to >>>>>>>>>>> calculate. In >>>>>>>>>>> such situation, will update with the latest status and findings. >>>>>>>>>>> >>>>>>>>>>> [1] [Architecture] Implementing Latency Metrics Calculation >>>>>>>>>>> Feature in GW >>>>>>>>>>> >>>>>>>>>>> Regards >>>>>>>>>>> >>>>>>>>>>> *Nadeeshaan Gunasinghe* >>>>>>>>>>> Software Engineer, WSO2 Inc. http://wso2.com >>>>>>>>>>> +94770596754 | [email protected] | Skype: >>>>>>>>>>> nadeeshaan.gunasinghe >>>>>>>>>>> <#-1155064647_1759133060_737110862_-711576374_-417006362_-1084696516_112020634_-442828947_1663256302_151ae46f9e826311_151ae43483a6d1f0_> >>>>>>>>>>> <http://www.facebook.com/nadeeshaan.gunasinghe> >>>>>>>>>>> <http://lk.linkedin.com/in/nadeeshaan> >>>>>>>>>>> <http://twitter.com/Nadeeshaan> >>>>>>>>>>> <http://nadeeshaan.blogspot.com/> >>>>>>>>>>> Get a signature like this: Click here! >>>>>>>>>>> <http://ws-promos.appspot.com/r?rdata=eyJydXJsIjogImh0dHA6Ly93d3cud2lzZXN0YW1wLmNvbS9lbWFpbC1pbnN0YWxsP3dzX25jaWQ9NjcyMjk0MDA4JnV0bV9zb3VyY2U9ZXh0ZW5zaW9uJnV0bV9tZWRpdW09ZW1haWwmdXRtX2NhbXBhaWduPXByb21vXzU3MzI1Njg1NDg3Njk3OTIiLCAiZSI6ICI1NzMyNTY4NTQ4NzY5NzkyIn0=> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> Kasun Indrasiri >>>>>>>>>> Software Architect >>>>>>>>>> WSO2, Inc.; http://wso2.com >>>>>>>>>> lean.enterprise.middleware >>>>>>>>>> >>>>>>>>>> cell: +94 77 556 5206 >>>>>>>>>> Blog : http://kasunpanorama.blogspot.com/ >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> Viraj Senevirathne >>>>>>>>> Software Engineer; WSO2, Inc. >>>>>>>>> >>>>>>>>> Mobile : +94 71 958 0269 >>>>>>>>> Email : [email protected] >>>>>>>>> >>>>>>>>> _______________________________________________ >>>>>>>>> Dev mailing list >>>>>>>>> [email protected] >>>>>>>>> http://wso2.org/cgi-bin/mailman/listinfo/dev >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> Chamil Elladeniya >>>>>>>> *Software Engineering **Intern* >>>>>>>> Mobile : +94 71 6181154 <%2B94%20%280%29%20773%20451194> >>>>>>>> [email protected] >>>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Chamil Elladeniya >>>>>>> *Software Engineering **Intern* >>>>>>> Mobile : +94 71 6181154 <%2B94%20%280%29%20773%20451194> >>>>>>> [email protected] >>>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Chamil Elladeniya >>>>>> *Software Engineering **Intern* >>>>>> Mobile : +94 71 6181154 <%2B94%20%280%29%20773%20451194> >>>>>> [email protected] >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> Isuru Perera >>>>> Associate Technical Lead | WSO2, Inc. | http://wso2.com/ >>>>> Lean . Enterprise . Middleware >>>>> >>>>> about.me/chrishantha >>>>> Contact: +IsuruPereraWSO2 >>>>> <https://www.google.com/+IsuruPereraWSO2/about> >>>>> >>>> >>>> >>>> >>>> -- >>>> Kasun Indrasiri >>>> Software Architect >>>> WSO2, Inc.; http://wso2.com >>>> lean.enterprise.middleware >>>> >>>> cell: +94 77 556 5206 >>>> Blog : http://kasunpanorama.blogspot.com/ >>>> >>> >>> >>> >>> -- >>> Best Regards >>> Isuru Ranawaka >>> M: +94714629880 >>> Blog : http://isurur.blogspot.com/ >>> >>> _______________________________________________ >>> Dev mailing list >>> [email protected] >>> http://wso2.org/cgi-bin/mailman/listinfo/dev >>> >>> >> >> _______________________________________________ >> Dev mailing list >> [email protected] >> http://wso2.org/cgi-bin/mailman/listinfo/dev >> >> > > > -- > Chamil Elladeniya > *Software Engineering **Intern* > Mobile : +94 71 6181154 <%2B94%20%280%29%20773%20451194> > [email protected] > -- Best Regards Isuru Ranawaka M: +94714629880 Blog : http://isurur.blogspot.com/
_______________________________________________ Dev mailing list [email protected] http://wso2.org/cgi-bin/mailman/listinfo/dev
