Hi, thanks for all your responses, it help to guide me.
I removed the use of gevent and used waitress, but this ended up in a slowdown. I think in my case, I'm more cpu bound then IO bound. I tried using multiple instance that nginx load balance from this page and it worked! If I use only 1 process, I have a small slowdown, but now I can scale to multiple core. http://docs.pylonsproject.org/projects/pyramid_cookbook/en/latest/deployment/nginx.html thanks for your help. Frédéric On Tue, Nov 6, 2012 at 3:33 AM, Carsten Senger <[email protected]> wrote: > Hi Frédéric, > > Am 05.11.2012 22:13, schrieb Frédéric Bastien: > > Hi, > > > > I inherited a project that is done with pyramid 1.2. Currently, it can > > process aroud 100-200 requests/s for dynamic page generated. The > bottleneck > > seam to be the GPU, as the paster process run close to 100%. We need to > be > > able to scale that up. > > > > As we have multi-cores CPU, is there a way to make use of them to get > > faster results? Each view have been done to scale between server. So > there > > is no concurency problem. > > > > I didn't found any documentation on this subject and google didn't helped > > me. That surprise me. So I suspect I missed the right keyword. Do you > know > > where there the documentation is? > > Do you start the server on localhost and use apache/nginx/whatever as a > proxy in front? Then you can start several servers and use e.g. apache + > mod_proxy_balancer to distribute the requests to the backend servers. > > http://httpd.apache.org/docs/2.4/mod/mod_proxy_balancer.html > > I use that with paster and it works really well. In the ini file I > configure the backends…:: > > [server:main] > use = egg:Paste#http > host = 127.0.0.1 > port = 20011 > use_threadpool = True > threadpool_workers = 20 > > [server:main2] > use = egg:Paste#http > host = 127.0.0.1 > port = 20012 > use_threadpool = True > threadpool_workers = 20 > > … and start all of them with their server name:: > > paster serve <inifile> --server-name=main2 > > Then I configure the Apache VirtualHost with mod_proxy_balancer. In this > case it is done with a generic cookie to makes the balancer selection > sticky. You might be able to avoid the overhead.:: > > ## Add generic cookie to store a route on the client > ## for mod_proxy_balancer > Header add Set-Cookie "ROUTEID=.%{BALANCER_WORKER_ROUTE}e; path=/" > env=BALANCER_ROUTE_CHANGED > > ## Set up the balancer to use our 4 paster processes. Use the cookie > to elect the backends > <Proxy balancer://adhocracy/> > BalancerMember http://127.0.0.1:20011 route=1 > BalancerMember http://127.0.0.1:20012 route=2 > BalancerMember http://127.0.0.1:20013 route=3 > BalancerMember http://127.0.0.1:20014 route=4 > ProxySet stickysession=ROUTEID > </Proxy> > > > For testing the balance-manager is useful.:: > > ## Make balancer-manager available to monitore it > <Location /balancer-manager> > SetHandler balancer-manager > Order Deny,Allow > Deny from all > </Location> > > > ..Carsten > -- > Carsten Senger - Schumannstr. 38 - 65193 Wiesbaden > [email protected] - (0611) 5324176 > PGP: gpg --recv-keys --keyserver hkp://subkeys.pgp.net 0xE374C75A > > -- You received this message because you are subscribed to the Google Groups "pylons-discuss" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/pylons-discuss?hl=en.
