Here is some of the information I can provide, some I'll have to go check. Database: In the past hour we had 1204 Object Stores and 159433 Loads and 465 connections.
-----Original Message----- From: Andrew Sawyers [mailto:[EMAIL PROTECTED] Sent: Thursday, September 01, 2005 1:30 PM To: Kennamore, Matthew G [NTK]; 'Chris McDonough' Cc: zope@zope.org Subject: RE: [Zope] Zope scalabilty and problems Sorry, I missed the beginning of this thread, I've got some questions that I didn't see answered or asked: Did Matt indicate if he was running multiple zeo app servers? It might help to be spreading the load. 1,0000 writes a day is not outrageous....so it's possible that really what you need to spread the load across a load balanced pool of zeo app servers. How many users are authenticated at any given time actually using the system for writing? Are you running a persistent or non-persistent cache? If so, what size it is? Check the control panel, under database management and choose your database, then look at the activity tab. Do you have a high volume of object loads? What is the volume of object stores? There is also an in-memory zeo object cache - click the cache parameters tab and see the number of objects in your in-memory cache. If your zeo client(s) have lots of ram (they definitely should), you can crank this up as well. What is the 'horespower' of your app server(s)? Andrew -- Zope Managed Hosting Software Engineer Zope Corporation (540) 361-1700 > -----Original Message----- > From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of > Kennamore, Matthew G [NTK] > Sent: Thursday, September 01, 2005 1:19 PM > To: Chris McDonough > Cc: zope@zope.org > Subject: RE: [Zope] Zope scalabilty and problems > > Thanks everyone for their suggestions, bought a squid book today and I > know what I'll be doing all weekend <grin> > > -----Original Message----- > From: Chris McDonough [mailto:[EMAIL PROTECTED] > Sent: Thursday, September 01, 2005 1:13 PM > To: Kennamore, Matthew G [NTK] > Cc: zope@zope.org > Subject: RE: [Zope] Zope scalabilty and problems > > What you've got now is a pretty reasonable setup. Moving to 2.7.7 will > likely not get you any "free" performance increases and it doesn't > appear that there are any "critical" ZODB bugfixes relating to your > problems since 2.7.3. FWIW, also, I'd venture a guess that moving "off > Data.fs" (as you mentioned in your original email) may be quite costly, > requiring a lot of application recoding. It also might not help solve > the scaling problem you're having. It's a "high risk, potentially low > reward" solution. You need something that is low-risk and potentially > high reward, I'd suspect. > > Others have said this, but it bears repeating. The biggest > "administrative" win you can get for the cheapest cost is to add an HTTP > cache (like Squid or even Apache's mod_proxy) in front of your ZEO > clients and to cause your Zope application to set the proper response > headers which allow the cache server to cache (and thus serve) those > pages on subsequent requests. > > This can be tricky when most of your traffic is authenticated because > there is usually very little win and a lot of danger to serving up > cached pages when a page in the cache represents a view that a > privileged user sees when visiting the site. You typically don't want > other people to see the same thing he does. As a result, typically > people will say "pages that require authentication can't be cached" > which isn't entirely true (you can cache them but it's often useless). > > However, even serving up fully static content that doesn't require any > authentication like CSS, images, and so on can be a win. You might > start there. > > HTH, > > - C > > > On Thu, 2005-09-01 at 11:54 -0500, Kennamore, Matthew G [NTK] wrote: > > Zope version 2.7.3 (planning to goto 2.7.7 soon with ZODB 3.2.9) > > Pyhton is 2.3.4 > > Apache 1.3 > > > > We have 943 users as of this minute with a bout 1000 objects being > created a day (Lots of creates) > > > > -----Original Message----- > > From: Chris McDonough [mailto:[EMAIL PROTECTED] > > Sent: Thursday, September 01, 2005 12:51 PM > > To: Kennamore, Matthew G [NTK] > > Cc: zope@zope.org > > Subject: Re: [Zope] Zope scalabilty and problems > > > > Your "tranaaction blocked" error messages seem to imply that either or > > both of the following is true: > > > > - some transactions are taking a "long time" > > (any more than, say, 200 milliseconds is a long time) > > > > - you have a very high transaction volume. > > > > That said, these messages are informational rather than signifying an > > error condition. Some transaction blockage is expected since the ZEO > > server can only deal with one transaction at a time. > > > > What version of Zope are you using? > > > > Is this an "intranet" application? Are there lots of users creating > > content? Can you venture a guess as to how many users are "creating" > > content vs. "viewing" content at any given time? Are the users that are > > creating content logged in? The users that are viewing content? > > > > On Wed, 2005-08-31 at 11:03 -0500, Kennamore, Matthew G [NTK] wrote: > > > Here is our situation, we originally launched zope with a small user > > > base in mind and like most IT implementations it got away from us > > > quickly and we have over 1000 users now. We are experiencing huge > > > slowdowns and lots of complaints of performance. > > > > > > Let me outline our system: > > > > > > I have a load balancer that round robins incoming request to 3 zeo > > > servers > > > > > > I have 1 zope server on the back en that severs the data, this is a > > > large single processor machine with 4 gigs of ram > > > > > > Each zeo server is 2 gigs of ram, single processor 3.2 gighz > > > > > > The throughput doesn't appear to be the problem as ram/processors are. > > > > > > Short term I plan to change all zeo's to 4 gigs ram and add 3 > > > additional servers. > > > > > > > > > The Technologies we are using are Zope, Zeo, plone and so forth. Why > > > are we having these large issues? I see sites with hundreds of > > > thousands of users and they don't seem to have these issues? We plan > > > to recode our site to get away from the Data.fs as soon as we can but > > > that is a ways off yet. What am I doing wrong can anyone help? > > > > > > _______________________________________________ > > > Zope maillist - Zope@zope.org > > > http://mail.zope.org/mailman/listinfo/zope > > > ** No cross posts or HTML encoding! ** > > > (Related lists - > > > http://mail.zope.org/mailman/listinfo/zope-announce > > > http://mail.zope.org/mailman/listinfo/zope-dev ) > > > > > > > > _______________________________________________ > Zope maillist - Zope@zope.org > http://mail.zope.org/mailman/listinfo/zope > ** No cross posts or HTML encoding! ** > (Related lists - > http://mail.zope.org/mailman/listinfo/zope-announce > http://mail.zope.org/mailman/listinfo/zope-dev ) _______________________________________________ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )