Hey guys Rohit that sounds great about the performance improvement. Regarding the questions
root admin, domain admin and/or user accounts Yes, for all users/projects/accounts etc multiple management servers 2 management servers, but our front is pointing to only one +-------------------------+-----------+ | Variable_name | Value | +-------------------------+-----------+ | innodb_buffer_pool_size | 134217728 | +-------------------------+-----------+ BR, Ricardo Pertuz On 25 Aug 2023, 10:56 AM -0500, Rohit Yadav <rohit.ya...@shapeblue.com>, wrote: > Hi Ricardo, > > Thanks for sharing, in addition to what Simon has asked; can you also share > if this behaviour is consistent across different types of account (root > admin, domain admin and/or user accounts) and domains, was there any change > in the infrastructure after which this was observed? > > We've introduced a small optimisation with > https://github.com/apache/cloudstack/pull/7911 that you've tested, this is > coming in 4.18.1.0 and later which calls the listVirtualMachinesMetrics API > with a limited details view, this is used to render the instance list page in > the UI. I think most users could see 5-10x improvements in the instance page > load times. > > > Regards. > > ________________________________ > From: Simon Weller <siwelle...@gmail.com> > Sent: Friday, August 25, 2023 19:18 > To: users@cloudstack.apache.org <users@cloudstack.apache.org> > Subject: Re: Query to Cloudstack API is degraded > > Kuasar, > > How big is your installation, and what do you have your MySQL > innodb_buffer_pool_size set to? > > Are you running a single, or multiple management servers? > > -Si > > > On Fri, Aug 25, 2023 at 8:43 AM Kuasar <ricardo.per...@kuasar.co.invalid> > wrote: > > > Hi Rohit, > > > > Thanks for you prompt reply. > > > > Could you say try with a MySQL 8.x? > > > > We are having this problem in production which is bigger and changing the > > db engine right now will take us weeks > > > > you could try Ubuntu 22.04/20.04 > > > > Yes, we are using Ubuntu 20.04 > > > > Can you measure (say using dev tools of your browser) > > > > List-vms always takes around 5.6s checking on the timing tab of the > > browser and > > > > http://<cloudstack mgmt host domain or IP>:8080/client/#/vm?details=min > > http://<cloudstack mgmt host domain or > > IP>:8080/client/#/vm?details=servoff,tmpl,nics > > > > Only 1 second, much better > > > > > > BR, > > > > Ricardo Pertuz > > > > > > On 25 Aug 2023, 6:41 AM -0500, Rohit Yadav <rohit.ya...@shapeblue.com>, > > wrote: > > > Hi Ricardo, > > > > > > Thanks for sharing. I can't say without more details on how to reproduce > > the env/condition, or look at the DB server. It might be statistics or > > something else, you could try to log mysql queries and see if they're > > slow(ing down). > > > > > > Could you say try with a MySQL 8.x server and use a different host > > server (for example, there is an issue reportedly affecting EL9 - > > https://github.com/apache/cloudstack/issues/7910; you could try Ubuntu > > 22.04/20.04 or EL8 to rule out the DB server). > > > > > > Can you measure (say using dev tools of your browser) the time it takes > > to render the list VM API call (specifically the API would be called > > listVirtualMachinesMetrics API), share the outcomes with us. > > > > > > Can you try the following URLs for the list instances page in the UI, > > see if any makes difference in loading times for you: > > > > > > > > > http://<cloudstack mgmt host domain or IP>:8080/client/#/vm?details=min > > > http://<cloudstack mgmt host domain or > > IP>:8080/client/#/vm?details=servoff,tmpl,nics > > > > > > Compared to the following that you've by default: > > > > > > > > > http://<cloudstack mgmt host domain or IP>8080/client/#/vm > > > > > > I've raised a PR to explore an idea further: > > > https://github.com/apache/cloudstack/pull/7911 > > > > > > > > > Regards. > > > > > > ________________________________ > > > From: Kuasar <ricardo.per...@kuasar.co.INVALID> > > > Sent: Thursday, August 24, 2023 19:21 > > > To: users@cloudstack.apache.org <users@cloudstack.apache.org>; > > users@cloudstack.apache.org <users@cloudstack.apache.org> > > > Subject: Re: Query to Cloudstack API is degraded > > > > > > Hi Rohit, > > > > > > Thanks for asking, my reply below > > > > > > - which APIs you think have degraded -> > > > listing instances > > > > > > - how have you concluded that they are degrading? -> > > > we have an upper app that has a configured timeout and it wasn’t firing, > > we had to change the value, and it is visible as well, it takes 5.6s > > bringing 5 vms. > > > > > > - are those list API or other async/action oriented APIs? -> > > > yes, it it async > > > > > > - which version of CloudStack and MySQL server is this, other details > > about your env (hypervisor, scale/size of resources etc). > > > ACS 4.18.0. —> 4GB 2Cores > > > Mariadb 10.5.x -> 4GB 2Cores > > > Hyp KVM > > > > > > - does restarting mgmt server help? any other information on how to > > reproduce this? > > > no change > > > > > > We are guessing it is related to database performance/index as it not > > happening in QA > > > > > > > > > Atte, > > > > > > Ricardo Pertuz > > > > > > > > > On 24 Aug 2023, 6:33 AM -0500, Rohit Yadav <rohit.ya...@shapeblue.com>, > > wrote: > > > > Hi Ricardo, > > > > > > > > Can you share more details, such as; > > > > > > > > - which APIs you think have degraded > > > > - how have you concluded that they are degrading? > > > > - are those list API or other async/action oriented APIs? > > > > - which version of CloudStack and MySQL server is this, other details > > about your env (hypervisor, scale/size of resources etc) > > > > - does restarting mgmt server help? any other information on how to > > reproduce this? > > > > > > > > > > > > Regards. > > > > > > > > ________________________________ > > > > From: Kuasar <ricardo.per...@kuasar.co.INVALID> > > > > Sent: Wednesday, August 23, 2023 22:00 > > > > To: users@cloudstack.apache.org <users@cloudstack.apache.org> > > > > Subject: Query to Cloudstack API is degraded > > > > > > > > During the last weeks we have been experiencing a degrading > > performance in API queries to Cloudstack, is there a procedure you > > recommend to check the possible bottleneck?, we are checking database > > indices. > > > > > > > > > > > > BR, > > > > > > > > Ricardo Pertuz > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
