Re: [Zope] Zope + Apache on Quad Debian machine
Hi Hugo, I've got a similar setup, Compaq DL580 on a Quad Xeon with 3gb of ram on Debian. To overcome the GIL and avoid dealing with CPU affiinity and to optimize server utilization, I started using Xen virtualization to create multiple virtual servers. It's open source and works great. Check out http://www.cl.cam.ac.uk/Research/SRG/netos/xen/ or http://www.xensource.com. For performance see http://www.cl.cam.ac.uk/Research/SRG/netos/xen/performance.html This tutorial worked for me: http://www.debian-administration.org/articles/320 I also recommend installing xen-tools http://www.steve.org.uk/Software/xen-tools/ It's very easy to create new virtual servers: xen-create-image --host=apache xen-create-image --host=zclient1 xen-create-image --host=zclient2 xen-create-image --host=zeo To create and attach to a virtual server, just do xm create zclient2.cfg -c log in and install python, Zope and whatever else you need. So instead of running 1 plone instance at 22pps, you can have serveral plone instances running at 22pps each and no GIL problem. HTH, Kevin Smith -- View this message in context: http://www.nabble.com/Zope-%2B-Apache-on-Quad-Debian-machine-t1287183.html#a3540420 Sent from the Zope - General forum at Nabble.com. ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Zope + Apache on Quad Debian machine
Thanks Kevin, It's a very good architecture option. I'm afraid a bit too late to test.. I'm already in production with the new architecture and CPU affinity. It's running smoothly though.. :-) Regards Hugo On 3/22/06, ksmith99 [EMAIL PROTECTED] wrote: Hi Hugo, I've got a similar setup, Compaq DL580 on a Quad Xeon with 3gb of ram on Debian. To overcome the GIL and avoid dealing with CPU affiinity and to optimize server utilization, I started using Xen virtualization to create multiple virtual servers. It's open source and works great. Check out http://www.cl.cam.ac.uk/Research/SRG/netos/xen/ or http://www.xensource.com. For performance see http://www.cl.cam.ac.uk/Research/SRG/netos/xen/performance.html This tutorial worked for me: http://www.debian-administration.org/articles/320 I also recommend installing xen-tools http://www.steve.org.uk/Software/xen-tools/ It's very easy to create new virtual servers: xen-create-image --host=apache xen-create-image --host=zclient1 xen-create-image --host=zclient2 xen-create-image --host=zeo To create and attach to a virtual server, just do xm create zclient2.cfg -c log in and install python, Zope and whatever else you need. So instead of running 1 plone instance at 22pps, you can have serveral plone instances running at 22pps each and no GIL problem. HTH, Kevin Smith -- View this message in context: http://www.nabble.com/Zope-%2B-Apache-on-Quad-Debian-machine-t1287183.html#a3540420 Sent from the Zope - General forum at Nabble.com. ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev ) -- Hugo Ramos - [EMAIL PROTECTED] http://www.orkut.com/Profile.aspx?uid=10082105466310142690 http://otugga.blogspot.com/ http://otuggapoesia.blogspot.com/ ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Zope + Apache on Quad Debian machine
On Wed, Mar 15, 2006 at 05:45:23PM -0300, Hugo Ramos wrote: Yellow, I'm using Zope+Apache on a 4 xeon's/4GB ram machine running Debian. I've noticed that the CPU's never go beyond 30% top occupation... but on rush hours the site takes too long to load... I've been reading about process affinity and how it could speed up everything by making zope run on 1 CPU, Apache on another and so on... Has any1 tried this before? Can you point me to some documentation? What's your experience? is it true that not doing this the 4 cpu's will not be 100% used? Have you considered that your setup might be IO bound? -- __ Nothing is as subjective as reality Reinoud van Leeuwen[EMAIL PROTECTED] http://www.xs4all.nl/~reinoud __ ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
RE: [Zope] Zope + Apache on Quad Debian machine
Why do you want to do this? Do you believe that this is more effective or efficient than the scheduling algorithm that Linux uses? Binding an application to a single processor makes very little sense at all. If Zope really is your limiting factor, then you need to spread that load over as many processors as possible, not lock it down to a single cpu - if you do that, then the rest of the processors sit idle while waiting for Zope to respond. Read up on ZEO. Then set yourself a ZEO server up and a couple of ZEO clients, balance your requests between the two using pound, apache or squid. I can guarantee you that if your system is Zope limited, then this will greatly increase responsiveness and the number of requests you can process simultaneously - I run over a dozen dual processor machines hosting approx 60 Zope applications across them, all using ZEO - an average of 5 or 6 instances per system, we have no problem with Zope only using one processor; the applications that are across most machines (up to 10 in our largest application) are very, very fast indeed and can cope with all the concurrent users we can throw at it. Your problem is not Zope itself... you definitely need to investigate further before coming to the conclusion that binding a Zope instance to a single processor is a solution - if you'll excuse me, that's a rather simplistic view to take. I was also suggest reading up on Apache, Linux, Proxying/Caching, Clustering and Load Balancing - they're all factors to consider when developing and deploying a web application. John Snowdon - IT Support Specialist -==- School of Medical Education Development Faculty of Medical Sciences Computing University of Newcastle Email : [EMAIL PROTECTED] -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Hugo Ramos Sent: 15 March 2006 20:45 To: [Zope] Subject: [Zope] Zope + Apache on Quad Debian machine Yellow, I'm using Zope+Apache on a 4 xeon's/4GB ram machine running Debian. I've noticed that the CPU's never go beyond 30% top occupation... but on rush hours the site takes too long to load... I've been reading about process affinity and how it could speed up everything by making zope run on 1 CPU, Apache on another and so on... Has any1 tried this before? Can you point me to some documentation? What's your experience? is it true that not doing this the 4 cpu's will not be 100% used? Cheers -- Hugo Ramos - [EMAIL PROTECTED] http://www.orkut.com/Profile.aspx?uid=10082105466310142690 ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev ) ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Zope + Apache on Quad Debian machine
a) Your system may be I/O bound - not CPU bound - so you will never see the CPU max out because the limiting factors are memory and disk/ network access. b) Make sure to tune Python's checkinterval. While you should *always* do that, it is especially important on multi-processor/multi- core hardware. The rule of thumb is to run the pystone benchmark, divide the pystones your machine benchmarks at by 50, and put the result into etc/zope.conf. e.g: python-check-interval 1000 Once you get the checkinterval right, playing with processor affinity becomes a pointless exercise, in my experience. c) Always use profiling tools for performance analysis. Guessing rarely leads to effective optimizations. ZopeProfiler and PTProfiler come to mind. HTH, Stefan On 15. Mär 2006, at 21:45, Hugo Ramos wrote: Yellow, I'm using Zope+Apache on a 4 xeon's/4GB ram machine running Debian. I've noticed that the CPU's never go beyond 30% top occupation... but on rush hours the site takes too long to load... I've been reading about process affinity and how it could speed up everything by making zope run on 1 CPU, Apache on another and so on... Has any1 tried this before? Can you point me to some documentation? What's your experience? is it true that not doing this the 4 cpu's will not be 100% used? -- Anything that happens, happens. --Douglas Adams ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Zope + Apache on Quad Debian machine
Thank you all !! (very insane multiple exclamation marks) I've been trying different scenarios and Zope performance increased 10 to 11 times faster... Cheers Hugo On 3/16/06, Stefan H. Holek [EMAIL PROTECTED] wrote: a) Your system may be I/O bound - not CPU bound - so you will never see the CPU max out because the limiting factors are memory and disk/ network access. b) Make sure to tune Python's checkinterval. While you should *always* do that, it is especially important on multi-processor/multi- core hardware. The rule of thumb is to run the pystone benchmark, divide the pystones your machine benchmarks at by 50, and put the result into etc/zope.conf. e.g: python-check-interval 1000 Once you get the checkinterval right, playing with processor affinity becomes a pointless exercise, in my experience. c) Always use profiling tools for performance analysis. Guessing rarely leads to effective optimizations. ZopeProfiler and PTProfiler come to mind. HTH, Stefan On 15. Mär 2006, at 21:45, Hugo Ramos wrote: Yellow, I'm using Zope+Apache on a 4 xeon's/4GB ram machine running Debian. I've noticed that the CPU's never go beyond 30% top occupation... but on rush hours the site takes too long to load... I've been reading about process affinity and how it could speed up everything by making zope run on 1 CPU, Apache on another and so on... Has any1 tried this before? Can you point me to some documentation? What's your experience? is it true that not doing this the 4 cpu's will not be 100% used? -- Anything that happens, happens. --Douglas Adams -- Hugo Ramos - [EMAIL PROTECTED] http://www.orkut.com/Profile.aspx?uid=10082105466310142690 http://otugga.blogspot.com/ http://otuggapoesia.blogspot.com/ ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Zope + Apache on Quad Debian machine
On Thu, Mar 16, 2006 at 02:38:04PM +0100, Stefan H. Holek wrote: a) Your system may be I/O bound - not CPU bound - so you will never see the CPU max out because the limiting factors are memory and disk/ network access. On that note, if running linux on a box with IDE disks, make sure that the disks have dma turned on. This can have a big impact on overall system responsiveness. man hdparm for details. -PW -- Paul Winkler http://www.slinkp.com ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Zope + Apache on Quad Debian machine
Hugo Ramos wrote at 2006-3-15 17:45 -0300: I'm using Zope+Apache on a 4 xeon's/4GB ram machine running Debian. I've noticed that the CPU's never go beyond 30% top occupation... but on rush hours the site takes too long to load... I've been reading about process affinity and how it could speed up everything by making zope run on 1 CPU, Apache on another and so on... A further, more promissing option, is to run several Zope processes on the same machine (using different ports) with a load balancer in front feeding them with work. -- Dieter ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
