Maybe in some places of the code but not everywhere. The problem is when there is a large load all 3 servers get very slow on every page. I think it is the DB layer as we have 90 tables in one database 45 in another. I am also using connection pooling which I think is causing problems. Because the DAL is loaded with every request, wouldn't that means the pool is open for each request? Or should there only ever be 10 connections open even if I have 1000 concurrent connections?
On Fri, May 11, 2012 at 6:51 AM, Richard Vézina <[email protected] > wrote: > Bruce, > > Are you building dict with query for represent or other use? > > Recently I solve speed problem I had by caching dict building query. > > I never thought that building a dictionary could be that expensive in term > of cpu load. > > Richard > > > On Thu, May 10, 2012 at 5:13 PM, Bruce Wade <[email protected]> wrote: > >> Yes there are a LOT of wait state on the web2py nodes and high CPU >> >> I will try your suggestions. >> >> Thanks, >> Bruce >> >> >> On Thu, May 10, 2012 at 2:10 PM, Michele Comitini < >> [email protected]> wrote: >> >>> The high load on web2py nodes seems to point to code in web2py. If it >>> were a problem with postgres you would have a high load on postgresql >>> and a lot of wait state and little CPU time resulting in little uptime >>> on web2py nodes but long page rendering times. >>> I suggest to try to convert some logic to use raw resultsets using >>> executesql instead of DAL Row objects. But before doing that try the >>> query on postgres directly: you can use the _select() method to obtain >>> the query generated by the DAL. If postgresql answers slowly try >>> adding indexes on columns as requested by EXPLAIN. >>> If postgresql answers fast try the guilty query with the DAL in a >>> python shell (i.e. python web2py.py -M -S <appname>). If it slow than >>> you have found the cause. >>> >>> Else keep using top to find if other processes are infesting the CPU >>> maybe it is a simple problem of "ping pong" or swappiness. Simple >>> tuning of uWSGI could suffice. As a rule of thumb you should not have >>> the number of web2py processes be more than twice the number of cores. >>> >>> mic >>> >>> >>> 2012/5/10 Bruce Wade <[email protected]>: >>> > Web2py is on 3 different servers/nodes, postgresql is on it's own node >>> with >>> > 8GB ram. >>> > >>> > CPU is being used by uwsgi so web2py. The slowness I think is from DB >>> > queries as when you load a page without the DB involved much it loads >>> > quickly >>> > >>> > The serving ads part is not a problem it is the other pages on the >>> website. >>> > At least not the adviewer the banner ads are new. The adviewer has >>> served >>> > over 29 million ads. >>> > >>> > I will try disabling the banner ads for now and set them so they are >>> > querying from a completely different server, maybe using mongodb and >>> node.js >>> > >>> > >>> > On Thu, May 10, 2012 at 11:28 AM, pbreit <[email protected]> >>> wrote: >>> >> >>> >> Is your traffic from serving ads or users coming to your web site? >>> Have >>> >> you exhausted caching opportunities? >>> > >>> > >>> > >>> > >>> > -- >>> > -- >>> > Regards, >>> > Bruce Wade >>> > http://ca.linkedin.com/in/brucelwade >>> > http://www.wadecybertech.com >>> > http://www.fittraineronline.com - Fitness Personal Trainers Online >>> > http://www.warplydesigned.com >>> > >>> >> >> >> >> -- >> -- >> Regards, >> Bruce Wade >> http://ca.linkedin.com/in/brucelwade >> http://www.wadecybertech.com >> http://www.fittraineronline.com - Fitness Personal Trainers Online >> http://www.warplydesigned.com >> >> > -- -- Regards, Bruce Wade http://ca.linkedin.com/in/brucelwade http://www.wadecybertech.com http://www.fittraineronline.com - Fitness Personal Trainers Online http://www.warplydesigned.com
