Re: [Zope] Scaling problems, or something else?

2007-05-22 Thread Chris Withers
Dieter Maurer wrote: I had to enhance ZEO to report which transactions are blocking in order to quickly resolve problems. It would be nice if you could feed those enhancements back into the source tree... cheers, Chris -- Simplistix - Content Management, Zope & Python Consulting

Re: [Zope] Scaling problems, or something else?

2007-05-21 Thread Dieter Maurer
Gerhard Schmidt wrote at 2007-5-21 08:42 +0200: > ... >We have expirienced some problem with ZEO when requests take to long. > >Sometimes we have Problems like this >2007-05-21T03:36:22 INFO ZEO.StorageServer (56016/10.152.64.29:52569) >Transaction blocked waiting for storage. Clients waiting: 1

Re: [Zope] Scaling problems, or something else?

2007-05-21 Thread Gerhard Schmidt
On Wed, May 16, 2007 at 05:52:16PM -0400, Paul Winkler wrote: > On Wed, May 16, 2007 at 04:24:56PM -0500, Jens Vagelpohl wrote: > > There's a difference between scaling and making something faster. ZEO > > makes a single instance slower, right. But you can deal with more > > requests concurrent

Re: [Zope] Scaling problems, or something else?

2007-05-17 Thread Dieter Maurer
Paul Winkler wrote at 2007-5-16 12:04 -0400: > ... >- multiple ZEO clients does allow you better responsiveness with high >concurrency, but you can also try increasing the number of threads to >get the same effect without the added network overhead. If you have several CPUs (and a CPU bound activi

Re: [Zope] Scaling problems, or something else?

2007-05-16 Thread Jens Vagelpohl
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 16 May 2007, at 16:52, Paul Winkler wrote: On Wed, May 16, 2007 at 04:24:56PM -0500, Jens Vagelpohl wrote: There's a difference between scaling and making something faster. ZEO makes a single instance slower, right. But you can deal with more r

Re: [Zope] Scaling problems, or something else?

2007-05-16 Thread Paul Winkler
On Wed, May 16, 2007 at 05:13:27PM +0200, Gaute Amundsen wrote: > On Wednesday 16 May 2007 07:52, Gerhard Schmidt wrote: > > All our frontendservers have 8gig ram. Zope gets major performance Problems > > when it reaches the limit of physical memory. Check your system if the. > > So having 2 zope P

Re: [Zope] Scaling problems, or something else?

2007-05-16 Thread Paul Winkler
On Wed, May 16, 2007 at 04:24:56PM -0500, Jens Vagelpohl wrote: > There's a difference between scaling and making something faster. ZEO > makes a single instance slower, right. But you can deal with more > requests concurrently using ZEO. That's what I consider "scaling". Agreed. But at the s

Re: [Zope] Scaling problems, or something else?

2007-05-16 Thread Jens Vagelpohl
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 16 May 2007, at 11:04, Paul Winkler wrote: On Wed, May 16, 2007 at 10:47:11AM -0500, Jens Vagelpohl wrote: On 16 May 2007, at 10:13, Gaute Amundsen wrote: Hm.. yes, that's the way we have been inclined.. just skip the whole ZEO thing, and go s

Re: [Zope] Scaling problems, or something else?

2007-05-16 Thread Paul Winkler
On Wed, May 16, 2007 at 10:47:11AM -0500, Jens Vagelpohl wrote: > On 16 May 2007, at 10:13, Gaute Amundsen wrote: > >Hm.. yes, that's the way we have been inclined.. just skip the > >whole ZEO > >thing, and go straight for Varnish integration. Lotts of old sinns > >that have > >to be paid off f

Re: [Zope] Scaling problems, or something else?

2007-05-16 Thread Jens Vagelpohl
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 16 May 2007, at 10:13, Gaute Amundsen wrote: Zope 2.7 doesn't scale very well with the ZEO. The more Frontend server you get the more Read Conflicts you have. Migration to Zope2.8 reduced this problem. I have build a squid proxy in reverse m

Re: [Zope] Scaling problems, or something else?

2007-05-16 Thread Gaute Amundsen
On Wednesday 16 May 2007 07:52, Gerhard Schmidt wrote: > On Thu, May 10, 2007 at 02:10:40AM +0200, Gaute Amundsen wrote: > > So there is no other possible limit in a zope instance than IO or CPU? > > If cpu was the limiting factor I would see the 2 python processes running > > 90% and dozens of htt

Re: [Zope] Scaling problems, or something else?

2007-05-16 Thread Gaute Amundsen
On Wednesday 16 May 2007 00:13, Paul Winkler wrote: > On Tue, May 15, 2007 at 12:00:52PM +0200, Gaute Amundsen wrote: > > I set cache-size to 1 last night, up from the default. > > I felt I had to try that before I had a grap in place, so I don't have > > good numbers to estimate tha change, bu

Re: [Zope] Scaling problems, or something else?

2007-05-16 Thread Gaute Amundsen
On Tuesday 15 May 2007 15:20, Chris Withers wrote: > Gaute Amundsen wrote: > > On Thursday 10 May 2007 17:57, Paul Winkler wrote: > >> On Thu, May 10, 2007 at 02:10:40AM +0200, Gaute Amundsen wrote: > >>> On Wednesday 09 May 2007 16:41, Paul Winkler wrote: > On Wed, May 09, 2007 at 12:07:54PM

Re: [Zope] Scaling problems, or something else?

2007-05-15 Thread Gerhard Schmidt
On Thu, May 10, 2007 at 02:10:40AM +0200, Gaute Amundsen wrote: > So there is no other possible limit in a zope instance than IO or CPU? > If cpu was the limiting factor I would see the 2 python processes running 90% > and dozens of httpd's taking up the rest? I'm running a zope site with up 70k

Re: [Zope] Scaling problems, or something else?

2007-05-15 Thread Paul Winkler
On Tue, May 15, 2007 at 12:00:52PM +0200, Gaute Amundsen wrote: > I set cache-size to 1 last night, up from the default. > I felt I had to try that before I had a grap in place, so I don't have good > numbers to estimate tha change, but with about 20 hits a sec on apache, there > was close to

Re: [Zope] Scaling problems, or something else?

2007-05-15 Thread Chris Withers
Gaute Amundsen wrote: On Thursday 10 May 2007 17:57, Paul Winkler wrote: On Thu, May 10, 2007 at 02:10:40AM +0200, Gaute Amundsen wrote: On Wednesday 09 May 2007 16:41, Paul Winkler wrote: On Wed, May 09, 2007 at 12:07:54PM +0200, Gaute Amundsen wrote: So there is no other possible limit in a

Re: [Zope] Scaling problems, or something else?

2007-05-15 Thread Gaute Amundsen
On Thursday 10 May 2007 17:57, Paul Winkler wrote: > On Thu, May 10, 2007 at 02:10:40AM +0200, Gaute Amundsen wrote: > > On Wednesday 09 May 2007 16:41, Paul Winkler wrote: > > > On Wed, May 09, 2007 at 12:07:54PM +0200, Gaute Amundsen wrote: > > > > So there is no other possible limit in a zope in

Re: [Zope] Scaling problems, or something else?

2007-05-10 Thread Paul Winkler
On Thu, May 10, 2007 at 02:10:40AM +0200, Gaute Amundsen wrote: > On Wednesday 09 May 2007 16:41, Paul Winkler wrote: > > On Wed, May 09, 2007 at 12:07:54PM +0200, Gaute Amundsen wrote: > So there is no other possible limit in a zope instance than IO or CPU? Well, there's RAM of course. If you ru

Re: [Zope] Scaling problems, or something else?

2007-05-09 Thread Gaute Amundsen
On Wednesday 09 May 2007 16:41, Paul Winkler wrote: > On Wed, May 09, 2007 at 12:07:54PM +0200, Gaute Amundsen wrote: > > Just a quick call for ideas on this problem we have... > > > > Setup: > > Zope 2.7.5 (~180 sites) -> apache (-> varnish for 1 high profile site) > > > > Most noticeable symtoms:

Re: [Zope] Scaling problems, or something else?

2007-05-09 Thread Paul Winkler
On Wed, May 09, 2007 at 12:07:54PM +0200, Gaute Amundsen wrote: > Just a quick call for ideas on this problem we have... > > Setup: > Zope 2.7.5 (~180 sites) -> apache (-> varnish for 1 high profile site) > > Most noticeable symtoms: > Takes 30 sec. or more to serve a page, or times out. > Spor

[Zope] Scaling problems, or something else?

2007-05-09 Thread Gaute Amundsen
Just a quick call for ideas on this problem we have... Setup: Zope 2.7.5 (~180 sites) -> apache (-> varnish for 1 high profile site) Most noticeable symtoms: Takes 30 sec. or more to serve a page, or times out. Sporadic problem, but allways during general high load. Lasts less than 1 hour. Res