Hi All, In yesterday's analytics team meeting we discussed about the performance results obtained through Log Analyzer for API-M performance tests. I had an offline discussion with Nuwan as well on the performance numbers. The summary of the above discussions was that hardly there will be any logs being generated when the API Manager installation is running in the proper conditions. The role of the log analyzer becomes important when the system starts emitting logs. For example, if someone enabled DEBUG logs on the Gateway there will be logs get printed for each API invocation. However, we may not be able to significantly improve performance to handle all that load (especially with the given test setup). We have decided to finalize the performance tests at this stage.
-- Thanks, Miyuru Dayarathna Senior Technical Lead Mobile: +94713527783 Blog: http://miyurublog.blogspot.com On Mon, Sep 12, 2016 at 11:36 AM, Ruwan Abeykoon <[email protected]> wrote: > Hi Tishan, > Yes, In production there is less number of logs in healthy system. > However, we have to think of the worst case scenario too. We have to make > sure that APIM does not become unstable even worst external service outage. > > Cheers, > Ruwan > > On Mon, Sep 12, 2016 at 10:56 AM, Tishan Dahanayakage <[email protected]> > wrote: > >> Hi Ruwan, >> >> Thanks for the comments and suggestions. >> >> On Mon, Sep 12, 2016 at 8:01 AM, Ruwan Abeykoon <[email protected]> wrote: >> >>> Hi All, >>> Thanks Tishan for the performance test. >>> >>> The test machines looks having average configuration for the purpose. >>> Hence I think the hardware is realistic. >>> >>> What we notice here is that 10,000 events per second being the >>> sustainable throughput. Since it seems at this rate CPU is loaded 60%, and >>> plenty of GC happening. >>> >>> Lets assume that on average an API invocation prints four(4) log lines. >>> This means we can sustain only 2500 API invocations on API manager when Log >>> Analyzer is turned on. This looks pretty low and not acceptable figure IMO. >>> >> AFAIK in a production environment we don't print any log per API >> invocation. Scenario which I can think logs can flood is the scenario of BE >> unavailability which will lead to timeouts logs. But in that case also we >> will get 1-2 log lines per API. Another situation is where the user has >> enabled debug logs. Given this what is the throughput level that you see as >> reasonable? >> >>> >>> We have to investigate where the bottlenecks are, and optimize them so >>> that the log publisher performance closely matches that of APIM/ESB >>> advertised throughput. >>> >> Bottleneck is at the persisting layer. We are writing each and every >> message to DB(DAS of-course write this in batches) and then run scripts on >> that. So we are bounded by that limitation in this case. These numbers are >> after we tune DB parameters. Earlier it was in the range of 700-1000. After >> tuning DB level we were able to increase throughput to this level. >> I can certainly look into code level and see whether we can optimize >> this more. Or else we need to re-think 'persist everything' strategy and >> look into option of persisting only the summary. But does that effort >> reasonable with our future road map? >> >> Thanks >> /Tishan >> >>> >>> Cheers, >>> Ruwan >>> >>> On Fri, Sep 9, 2016 at 6:09 PM, Tishan Dahanayakage <[email protected]> >>> wrote: >>> >>>> HI all, >>>> >>>> Please find the below document which contains log analyzer performance >>>> analysis. Initially we could not achieve desired performance numbers. After >>>> analyzing the JVM recording figured out the limitation was with Database >>>> access. After tuning DB access we were able to get better throughput. >>>> Analysis correlates throughput results with system resource utilization. >>>> >>>> Thanks, >>>> /Tishan >>>> >>>> -- >>>> Tishan Dahanayakage >>>> Senior Software Engineer >>>> WSO2, Inc. >>>> Mobile:+94 716481328 >>>> >>>> Disclaimer: This communication may contain privileged or other >>>> confidential information and is intended exclusively for the addressee/s. >>>> If you are not the intended recipient/s, or believe that you may have >>>> received this communication in error, please reply to the sender indicating >>>> that fact and delete the copy you received and in addition, you should not >>>> print, copy, re-transmit, disseminate, or otherwise use the information >>>> contained in this communication. Internet communications cannot be >>>> guaranteed to be timely, secure, error or virus-free. The sender does not >>>> accept liability for any errors or omissions. >>>> >>> >>> >>> >>> -- >>> >>> *Ruwan Abeykoon* >>> *Associate Director/Architect**,* >>> *WSO2, Inc. http://wso2.com <https://wso2.com/signature> * >>> *lean.enterprise.middleware.* >>> >>> >> >> >> -- >> Tishan Dahanayakage >> Senior Software Engineer >> WSO2, Inc. >> Mobile:+94 716481328 >> >> Disclaimer: This communication may contain privileged or other >> confidential information and is intended exclusively for the addressee/s. >> If you are not the intended recipient/s, or believe that you may have >> received this communication in error, please reply to the sender indicating >> that fact and delete the copy you received and in addition, you should not >> print, copy, re-transmit, disseminate, or otherwise use the information >> contained in this communication. Internet communications cannot be >> guaranteed to be timely, secure, error or virus-free. The sender does not >> accept liability for any errors or omissions. >> > > > > -- > > *Ruwan Abeykoon* > *Associate Director/Architect**,* > *WSO2, Inc. http://wso2.com <https://wso2.com/signature> * > *lean.enterprise.middleware.* > > -- Thanks, Miyuru Dayarathna Senior Technical Lead Mobile: +94713527783 Blog: http://miyurublog.blogspot.com
_______________________________________________ Architecture mailing list [email protected] https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
