[Zope] Scalability
This is probably falls under the catagory of FAQ but I couldn't find a good answer through searching, so I'll ask. Are there any size limitations on file system storages? At what point can things become very slow? I'm wondering as much about the number of objects stored as well as file size for a Data.fs. And what about zeo storages. Is there on the number of zeo storages that can be connected to zope clients? As far as number of objects, I'm thinking in terms of 10,000,000 or more folderish and content objects. While I'm talking about it, are there any performance limitations on the zcatalog? Will it perform well even with this many objects? Thank you in advance, Kevin ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
RE: [Zope] Zope Scalability
>-Original Message- >From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On >Behalf Of Jens Vagelpohl >Sent: 07 October 2005 09:04 >To: Zope ML >Subject: Re: [Zope] Zope Scalability > > >> As an aside, we find management of ZEO clients much easier >if each ZEO >> client of a particular system shares the same products and external >> methods via an NFS share. That way we can untar one product and >> push it >> out to all of the clients simultaneously. > >I'd be a little afraid of creating a single point of failure with >NFS. I have used setups like that before, but personally prefer some >simple distribution mechanism instead. > >If you use CVS or SVN for your software you could write simple SSH >scripts to visit hosts and do a cvs/svn up in the right place and >then restart the clients. rsync is a good candidate as well. > >jens > We don't have anything that complex :-) Most sites that we host probably have less than 20 external python methods (a lot have none at all!), with common products shared by all the sites installed on each system in the master zope lib/python/products directory structure. The majority of complexity is in each systems ZODB. All of our boxes are on a private network, on the same gbit switch as the ZEO server, so performance/service interruption, network timeouts etc.. is never a factor. The NFS shares (products, extensions, file upload areas[uploads via zope, and these areas are then served back out via Apache] and user home directories[again, using our ldap systems for authentication]) are from the same ZEO server (primary storage is handled by a multi-terabyte SATA). We're running about ~50 individual Zope systems at the moment (all as ZEO clients, many have multiple ZEO clients across several nodes, with incoming requests load balanced via Pound).. and I must say, this setup is way better than our old Sun E3500 that used to (try) to handle all of this! ZEO is the way to go for sure though we are now hitting problems when trying to integrate the Internet2's Shibboleth single sign on authentication system into a load-balanced setup... but that's a story for another time :-) ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Zope Scalability
As an aside, we find management of ZEO clients much easier if each ZEO client of a particular system shares the same products and external methods via an NFS share. That way we can untar one product and push it out to all of the clients simultaneously. I'd be a little afraid of creating a single point of failure with NFS. I have used setups like that before, but personally prefer some simple distribution mechanism instead. If you use CVS or SVN for your software you could write simple SSH scripts to visit hosts and do a cvs/svn up in the right place and then restart the clients. rsync is a good candidate as well. jens ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
RE: [Zope] Zope Scalability
>--On 5. Oktober 2005 17:37:08 +0100 Tom Wilde ><[EMAIL PROTECTED]> >wrote: > >> Hi folks, >> >> I've been using Zope for a while now and found it to be an excellent >> solution however I've got a few enterprise questions about the >> scalability of Zope that just need to be answered. >> >> [ nb: I'm using V2.72 at the moment, not having moved over to V3 ] >> >> ExtFile >> Managed to serve 500,000 files through extfile - anybody >come across an >> upper limit here? Planning to serve larger sites off a SAN >hopefully - >> has anyone tried this? > >no idea For static content we still use Apache. >> >> Zeo and Cataloging >> If I have my site on one server and ZCatalog (using TextIndexNG) on >> another server (connected by ZEO) where does the actual >index operation >> occur? Do just the results get propogated to the catalog? > >The indexing work is always happens on the ZEO clients, not on >the server. >Just the results are transferred to the ZEO Server. > >> >> LDAP Authentication >> I understand that can use the LDAPUserFolder Product to provide >> authentication against an ldap server. Could I throw,say, >5+ users >> at zope using this system? > >This should work in general. Possibly it requires some >tweaking here and >there depending on your usecases. We use LDAPUserFolder as the backend authentication store for our online learning environment here at newcastle. It copes rather well with several thousand students and many groups - I'd imagine it will scale as well as your LDAP setup can. (Our LDAP server is a pretty low-end Sun V60 - single Xeon 2.8GHz with 2Gb of ram and is very rarely under any significant load). >> >> Load Balancing and Cookie Based Sessions >> Currently we use cookie based sessions via cookie crumbler >presumably in >> a load balanced server setup we'd have to host acl_users ona ZEO'd >> izope instance somewhere on the backend? > > >I don't see what sessions have to do with acl_users. In your >ZEO setup you >have identical data except sessions on every ZEO client. You >just have to >ensure somehow that users get to the same ZEO client for the >lifetime of >the session. How this is done depends basically on the load balancer. We initially tied users sessions to individual ZEO clients (6 x dual Opteron nodes) and it worked well. However we have since migrated the session information to be served via the ZEO server in the same manner as the data.fs; we now get the benefit of much finer grained load balancing and are very pleased with the performance we get. As a side effect, it means we can host multiple ZEO clients from different systems on one box without fear of a particular ZEO client bringing the box to it's knees. As an aside, we find management of ZEO clients much easier if each ZEO client of a particular system shares the same products and external methods via an NFS share. That way we can untar one product and push it out to all of the clients simultaneously. -John -- Faculty of Medical Sciences Computing University of Newcastle upon Tyne ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Zope Scalability
On 10/5/05, Andreas Jung <[EMAIL PROTECTED]> wrote: > This would require how much RAM for the index? :-) It can hold 16 billion billion pointers, with each pointer being 48 bits, thats 96 exabytes (100 million terabytes). Just for the index. :) As an absolute minimum. Indexes normally being btrees or something like that, you would surely need several times that in practice. :) Sorry for any calculation errors, but I can be rather a lot off without it making a difference, so I can't be bothered to check. ;) -- Lennart Regebro, Nuxeo http://www.nuxeo.com/ CPS Content Management http://www.cps-project.org/ ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Zope Scalability
[Tim Peters] >> If you're using FileStorage, a technical detail in the implementation >> of the FileStorage index limits the maximum file offset that can be >> used to 2**48-1, or about 281 terabytes. Object IDs are effectively >> 64-bit integers (masquerading as 8-byte strings). [Andreas Jung] > This would require how much RAM for the index? :-) Unfortunately, that's a complicated question -- the index is an OOBTree mapping 6-byte strings to a specialized kind of BTree mapping 2-byte strings to 6-byte strings. The complications add up. The good news is that if there are only two objects, say each consuming 128 terabytes, the index has only two entries and is actually very small . ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Zope Scalability
On 5 Oct 2005, at 20:57, Tim Peters wrote: [Jens Vagelpohl] Well, the question was actually "What is the maximum size of this file and/or maximum object ID? => just how many objects can the zodb hold?". There is only a theoretical limit that has to do with the largest index key for the ZODB index I believe, and that is some ludicrously high number that no one has ever reached. If you're using FileStorage, a technical detail in the implementation of the FileStorage index limits the maximum file offset that can be used to 2**48-1, or about 281 terabytes. Object IDs are effectively 64-bit integers (masquerading as 8-byte strings). If that's not falling in the "ludicrous" category, I don't know what is ;) I suppose we'll all be safe for a while. jens ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Zope Scalability
--On 5. Oktober 2005 15:57:14 -0400 Tim Peters <[EMAIL PROTECTED]> wrote: [Jens Vagelpohl] Well, the question was actually "What is the maximum size of this file and/or maximum object ID? => just how many objects can the zodb hold?". There is only a theoretical limit that has to do with the largest index key for the ZODB index I believe, and that is some ludicrously high number that no one has ever reached. If you're using FileStorage, a technical detail in the implementation of the FileStorage index limits the maximum file offset that can be used to 2**48-1, or about 281 terabytes. Object IDs are effectively 64-bit integers (masquerading as 8-byte strings). This would require how much RAM for the index? :-) -aj pgp75Yl8tx91D.pgp Description: PGP signature ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Zope Scalability
[Jens Vagelpohl] > Well, the question was actually "What is the maximum size of this > file and/or maximum object ID? => just how many objects can the zodb > hold?". There is only a theoretical limit that has to do with the > largest index key for the ZODB index I believe, and that is some > ludicrously high number that no one has ever reached. If you're using FileStorage, a technical detail in the implementation of the FileStorage index limits the maximum file offset that can be used to 2**48-1, or about 281 terabytes. Object IDs are effectively 64-bit integers (masquerading as 8-byte strings). ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Zope Scalability
On 5 Oct 2005, at 20:35, Matthew X. Economou wrote: There is no answer to this question because it cannot be answered. No one has come against a "limit" yet as far as I know. Hitting RAM limits depends on the settings you apply to the Zope/ZEO instance(s) that serve(s) the content and how much RAM you put in, you can control ZODB memory cache size via zope.conf. No one sets up Zope so that the whole ZODB is loaded into memory, if that's what you mean. I got the impression that he was asking about the maximum object size. Well, the question was actually "What is the maximum size of this file and/or maximum object ID? => just how many objects can the zodb hold?". There is only a theoretical limit that has to do with the largest index key for the ZODB index I believe, and that is some ludicrously high number that no one has ever reached. jens ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
RE: [Zope] Zope Scalability
Jens wrote: > There is no answer to this question because it cannot be answered. > No one has come against a "limit" yet as far as I know. Hitting RAM > limits depends on the settings you apply to the Zope/ZEO instance(s) > that serve(s) the content and how much RAM you put in, you can > control ZODB memory cache size via zope.conf. No one sets up Zope so > that the whole ZODB is loaded into memory, if that's what you mean. I got the impression that he was asking about the maximum object size. I don't know if there is a limit in Zope itself, but I've run into problems with PloneExFile when uploading files larger than about 25 MB. Who knows - it might be a bug in PloneExFile, AttachmentField, or Archetypes. I lack the expertise required to troubleshoot the error, so take this with the proverbial grain of salt. BTW: I'm using LDAPUserFolder in a 10,000-user Active Directory environment. It works great. Best wishes, Matthew -- jsoffron: I'm generally pretty high on national defense... Mr. Bad Example: Careful...it's a gateway policy. Before you know it, you'll be mainlining the hard stuff like trade agreements. jsoffron: Too late...I've been freebasing Nafta all day... Sweet, sweet NAFTA. - As seen on Slashdot ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Zope Scalability
ZODB size What is the maximum size of this file and/or maximum object ID? => just how many objects can the zodb hold? millions? tens of millions..?? billions...??? I have a feeling we'll run out of ram long before actually hitting the limit but it'd be nice to know the theoretical top limit. There is no answer to this question because it cannot be answered. No one has come against a "limit" yet as far as I know. Hitting RAM limits depends on the settings you apply to the Zope/ZEO instance(s) that serve(s) the content and how much RAM you put in, you can control ZODB memory cache size via zope.conf. No one sets up Zope so that the whole ZODB is loaded into memory, if that's what you mean. Zeo and Cataloging If I have my site on one server and ZCatalog (using TextIndexNG) on another server (connected by ZEO) where does the actual index operation occur? Do just the results get propogated to the catalog? The actual indexing operation happens on the Zope/ZEO instance, and the resulting database changes are applied to the respective storage where the catalog resides when the transaction commits. LDAP Authentication I understand that can use the LDAPUserFolder Product to provide authentication against an ldap server. Could I throw,say, 5+ users at zope using this system? I see no problems with that. It doesn't matter one bit to the LDAPUserFolder (or the Zope security machinery) how many user records you have in the DIT. If there are any slowdowns as the number of user records increases it is most likely due to misconfigurations on the LDAP server itself, such as bad indexing or overcomplicateed/ unnecessary access control checks. Load Balancing and Cookie Based Sessions Currently we use cookie based sessions via cookie crumbler presumably in a load balanced server setup we'd have to host acl_users ona ZEO'd izope instance somewhere on the backend? Cookie crumbler has nothing to do with sessions. All it does is set/ read a cookie with credentials and then "fake" regular HTTP authentication with this data. There is no state information being stored on the server side, so in a ZEO setup it does not matter which ZEO client you hit. jens ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Zope Scalability
--On 5. Oktober 2005 17:37:08 +0100 Tom Wilde <[EMAIL PROTECTED]> wrote: Hi folks, I've been using Zope for a while now and found it to be an excellent solution however I've got a few enterprise questions about the scalability of Zope that just need to be answered. [ nb: I'm using V2.72 at the moment, not having moved over to V3 ] ZODB size What is the maximum size of this file and/or maximum object ID? => just how many objects can the zodb hold? millions? tens of millions..?? billions...??? I have a feeling we'll run out of ram long before actually hitting the limit but it'd be nice to know the theoretical top limit. Several million objects are common for large installations. The ZODB keeps an internal index of the objects (the size is proportional to the number of objects). In addition you have caches. Their size depends on the configuration. Running a ZEO client with 1-2 GB of RAM for large installation is common. ExtFile Managed to serve 500,000 files through extfile - anybody come across an upper limit here? Planning to serve larger sites off a SAN hopefully - has anyone tried this? no idea Zeo and Cataloging If I have my site on one server and ZCatalog (using TextIndexNG) on another server (connected by ZEO) where does the actual index operation occur? Do just the results get propogated to the catalog? The indexing work is always happens on the ZEO clients, not on the server. Just the results are transferred to the ZEO Server. LDAP Authentication I understand that can use the LDAPUserFolder Product to provide authentication against an ldap server. Could I throw,say, 5+ users at zope using this system? This should work in general. Possibly it requires some tweaking here and there depending on your usecases. Load Balancing and Cookie Based Sessions Currently we use cookie based sessions via cookie crumbler presumably in a load balanced server setup we'd have to host acl_users ona ZEO'd izope instance somewhere on the backend? I don't see what sessions have to do with acl_users. In your ZEO setup you have identical data except sessions on every ZEO client. You just have to ensure somehow that users get to the same ZEO client for the lifetime of the session. How this is done depends basically on the load balancer. -aj pgpP49qwCbWDd.pgp Description: PGP signature ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] Zope Scalability
Hi folks, I've been using Zope for a while now and found it to be an excellent solution however I've got a few enterprise questions about the scalability of Zope that just need to be answered. [ nb: I'm using V2.72 at the moment, not having moved over to V3 ] ZODB size What is the maximum size of this file and/or maximum object ID? => just how many objects can the zodb hold? millions? tens of millions..?? billions...??? I have a feeling we'll run out of ram long before actually hitting the limit but it'd be nice to know the theoretical top limit. ExtFile Managed to serve 500,000 files through extfile - anybody come across an upper limit here? Planning to serve larger sites off a SAN hopefully - has anyone tried this? Zeo and Cataloging If I have my site on one server and ZCatalog (using TextIndexNG) on another server (connected by ZEO) where does the actual index operation occur? Do just the results get propogated to the catalog? LDAP Authentication I understand that can use the LDAPUserFolder Product to provide authentication against an ldap server. Could I throw,say, 5+ users at zope using this system? Load Balancing and Cookie Based Sessions Currently we use cookie based sessions via cookie crumbler presumably in a load balanced server setup we'd have to host acl_users ona ZEO'd izope instance somewhere on the backend? thanks in advance, any advice welcome - in a brave new world here. Tom ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
RE: [Zope] Zope scalability / efficiency question
Any load balancer should be able to do this, google for your OS there. The load balancer I use LVS. See http://www.linuxvirtualserver.org/ for more details. Andrew Sawyers -- Zope Managed Hosting Software Engineer Zope Corporation (540) 361-1700 > -Original Message- > From: Fernando Lujan [mailto:[EMAIL PROTECTED] > Sent: Wednesday, September 21, 2005 5:35 PM > To: Andrew Sawyers; zope@zope.org > Subject: Re: [Zope] Zope scalability / efficiency question > > On 9/20/05, Andrew Sawyers <[EMAIL PROTECTED]> wrote: > > I did some tests a few years back, basically that came out such that you > run > > the number of zope instances as you have CPUs. I was able to get double > the > > performance of a single instance on a dual cpu server when adding a > second > > instance. On a single CPU server, I found that the number of requests > the > > server was able to render was cut by nearly 50% +/- when adding a second > > instance. > > > > My rule for building out ZEO Client servers is: > > Single CPU - Single Zope instance > > Multiple Processors - # CPUs == # of Zope instances > > Hi Andrew, > > And how can I distribute the load over these Instances, running on the > same machine? > > Thanks in advance. > > Fernando Lujan ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Zope scalability / efficiency question
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Fernando Lujan wrote: > On 9/20/05, Andrew Sawyers <[EMAIL PROTECTED]> wrote: > >>I did some tests a few years back, basically that came out such that you run >>the number of zope instances as you have CPUs. I was able to get double the >>performance of a single instance on a dual cpu server when adding a second >>instance. On a single CPU server, I found that the number of requests the >>server was able to render was cut by nearly 50% +/- when adding a second >>instance. >> >>My rule for building out ZEO Client servers is: >>Single CPU - Single Zope instance >>Multiple Processors - # CPUs == # of Zope instances > > > Hi Andrew, > > And how can I distribute the load over these Instances, running on the > same machine? > You may use load balancers such as : - pound : http://www.apsis.ch/pound/ - balance : http://www.inlab.de/balance.html (not sticky last time I used it) and probably others. Cheers, J. - -- Julien Anguenot | Nuxeo R&D (Paris, France) CPS Platform : http://www.cps-project.org Zope3 / ECM : http://www.z3lab.org mail: anguenot at nuxeo.com; tel: +33 (0) 6 72 57 57 66 -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.1 (GNU/Linux) Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org iD8DBQFDMrDEGhoG8MxZ/pIRAsKKAJ9qxWcxH/6Kujvmxnvq2VlVrff6RgCfRC0V 1ItmtQGQiAnS89h+7tP9XXY= =/2zA -END PGP SIGNATURE- ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Zope scalability / efficiency question
On 9/20/05, Andrew Sawyers <[EMAIL PROTECTED]> wrote: > I did some tests a few years back, basically that came out such that you run > the number of zope instances as you have CPUs. I was able to get double the > performance of a single instance on a dual cpu server when adding a second > instance. On a single CPU server, I found that the number of requests the > server was able to render was cut by nearly 50% +/- when adding a second > instance. > > My rule for building out ZEO Client servers is: > Single CPU - Single Zope instance > Multiple Processors - # CPUs == # of Zope instances Hi Andrew, And how can I distribute the load over these Instances, running on the same machine? Thanks in advance. Fernando Lujan ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
RE: [Zope] Zope scalability / efficiency question
I did some tests a few years back, basically that came out such that you run the number of zope instances as you have CPUs. I was able to get double the performance of a single instance on a dual cpu server when adding a second instance. On a single CPU server, I found that the number of requests the server was able to render was cut by nearly 50% +/- when adding a second instance. My rule for building out ZEO Client servers is: Single CPU - Single Zope instance Multiple Processors - # CPUs == # of Zope instances Make sure you have enough ram if you're running multiple instances. The number of sites you can run from a single instance without cache problems really takes some localized knowledge of how your instances are performing. I run *lots* of sites from a single Zope instance without splitting them off at the load balancer into different mini-pools. I do this because I can, but if I was performance challenged from, say, cache flipping, I would definitely do this. You could likely better off if you ran all of your sites within a single instance (spread across multiple multi-processor machines with enough ram (2 - 4 GBs should do?)). It's a tough question to answer with much precision without intimate knowledge of what you're doing and how your systems are performing. If they're in the same zope instance, managing them as separate entities is difficult. Your application and maintenance of these sites might be the best determination for how you should factor the instances across your hardware. Andrew Sawyers -- Zope Managed Hosting Software Engineer Zope Corporation (540) 361-1700 > -Original Message- > From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of J > Cameron Cooper > Sent: Tuesday, September 20, 2005 1:26 PM > To: [EMAIL PROTECTED] > Cc: zope@zope.org > Subject: Re: [Zope] Zope scalability / efficiency question > > mark hellewell wrote: > > Hi everyone, I've got a question I'd really appreciate some input on. > > > > If I'm running perhaps a dozen Zope sites, experiencing anywhere > > between several hits per second to a lot less, would it potentially > > be more efficient (decreased latency etc.) to run each site as its own > > Zope instance or have one Zope instance running muliple (CMF based) > > sites. > > > > I haven't even thought about adding ZEO into the mix yet. > > > > The basic aim of the exercise is to find out how many usable > > zope instances / sites I could get onto a typical mid-range > > modern server. > > > > I know it's a bit of a "how long are your pieces of string" sort > > of question, but any help you could offer would be gratefully > > accepted! If anyway has any experience of this sort of setup > > and can offer a little insight it might reduce the amount of > > testing I have to do myself. > > I don't think you'll see any improvement unless you have a > multi-processor machine. In such a case, running X processes (where C is > the number of cores) will fully load the machine, since Python threads > won't skip processors. > > On a single-core machine, multiple instances probably will just increase > your overhead, especially in memory. But I could be wrong: there's so > many variables the best way to know is to test. (Caches and usage > patters, for instance, can make a big difference.) > > If you do have several sites, you could be running into cache eviction > issues since there are a lot of objects that need to be accessed. Check > your cache fullness and watch your ZODB usage chart for excessive loads. > > --jcc > -- > "Building Websites with Plone" > http://plonebook.packtpub.com/ > > Enfold Systems, LLC > http://www.enfoldsystems.com > ___ > Zope maillist - Zope@zope.org > http://mail.zope.org/mailman/listinfo/zope > ** No cross posts or HTML encoding! ** > (Related lists - > http://mail.zope.org/mailman/listinfo/zope-announce > http://mail.zope.org/mailman/listinfo/zope-dev ) ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Zope scalability / efficiency question
mark hellewell wrote: Hi everyone, I've got a question I'd really appreciate some input on. If I'm running perhaps a dozen Zope sites, experiencing anywhere between several hits per second to a lot less, would it potentially be more efficient (decreased latency etc.) to run each site as its own Zope instance or have one Zope instance running muliple (CMF based) sites. I haven't even thought about adding ZEO into the mix yet. The basic aim of the exercise is to find out how many usable zope instances / sites I could get onto a typical mid-range modern server. I know it's a bit of a "how long are your pieces of string" sort of question, but any help you could offer would be gratefully accepted! If anyway has any experience of this sort of setup and can offer a little insight it might reduce the amount of testing I have to do myself. I don't think you'll see any improvement unless you have a multi-processor machine. In such a case, running X processes (where C is the number of cores) will fully load the machine, since Python threads won't skip processors. On a single-core machine, multiple instances probably will just increase your overhead, especially in memory. But I could be wrong: there's so many variables the best way to know is to test. (Caches and usage patters, for instance, can make a big difference.) If you do have several sites, you could be running into cache eviction issues since there are a lot of objects that need to be accessed. Check your cache fullness and watch your ZODB usage chart for excessive loads. --jcc -- "Building Websites with Plone" http://plonebook.packtpub.com/ Enfold Systems, LLC http://www.enfoldsystems.com ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] Zope scalability / efficiency question
Hi everyone, I've got a question I'd really appreciate some input on. If I'm running perhaps a dozen Zope sites, experiencing anywhere between several hits per second to a lot less, would it potentially be more efficient (decreased latency etc.) to run each site as its own Zope instance or have one Zope instance running muliple (CMF based) sites. I haven't even thought about adding ZEO into the mix yet. The basic aim of the exercise is to find out how many usable zope instances / sites I could get onto a typical mid-range modern server. I know it's a bit of a "how long are your pieces of string" sort of question, but any help you could offer would be gratefully accepted! If anyway has any experience of this sort of setup and can offer a little insight it might reduce the amount of testing I have to do myself. Many thanks, Mark ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
