Re: [Zope-dev] Zope reference counts
Ivo van der Wijk wrote at 2003-1-22 16:10 +0100: On Wed, Jan 22, 2003 at 11:57:20AM +, Toby Dickenson wrote: Is there a way to optimize this? 1. Obviously, use fewer threads. Do you have a component that otherwise benefits from many threads? No, except that the server is pretty heavy loaded sometimes, and multiple threads should improve responsiveness (right?) When you have both short and long requests, then multiple threads help that the short requests are processed faster. When all your requests take approximately the same time, you will not be more responsive but your average response time becomes much worser (provided you can only use a single processor which is normal for Python threads). Dieter ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Zope reference counts
On Tue, Jan 21, 2003 at 04:27:05PM +, Toby Dickenson wrote: On Tuesday 21 January 2003 4:14 pm, Ivo van der Wijk wrote: Does this mean each thread keeps it's own cache pool of objects? yes Do these references point to the same object instances, or are these copied as well? They are independant copies I have the impression the memoryconsumption grows with the number of threads - is this true? yes Cheers I hope this helps It clarifies alot. Why exactly is this? concurrency issues? Is there a way to optimize this? I assume it's an issue on the ZODB level? Cheers Ivo -- Drs. I.R. van der Wijk -=- Brouwersgracht 132 Amaze Internet Services V.O.F. 1013 HA Amsterdam, NL -=- T +31-20-4688336 F +31-20-4688337Linux/Web/Zope/SQL/MMBase W http://www.amaze.nlE [EMAIL PROTECTED] Network Solutions W http://vanderwijk.info E [EMAIL PROTECTED] Consultancy PGP http://vanderwijk.info/pgp -=- ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Zope reference counts
Toby Dickenson wrote: On Wednesday 22 January 2003 11:43 am, Ivo van der Wijk wrote: Do these references point to the same object instances, or are these copied as well? They are independant copies Is there a way to optimize this? 1. Obviously, use fewer threads. Do you have a component that otherwise benefits from many threads? Also note that it is almost always a better use of memory to increase the cache size than the number of threads (this is the size of each cache, not the total cache size). Are you aware of the activity and cache parameters tab in the database management part of the control panel? If not, you may not be using a 2.6x Zope - there are some significant improvements in cache usage (especially under load) in these versions. Look at the Help on the cache parameters page for more info. seb ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Zope reference counts
On Wed, Jan 22, 2003 at 12:14:27PM +, seb bacon wrote: Toby Dickenson wrote: On Wednesday 22 January 2003 11:43 am, Ivo van der Wijk wrote: Do these references point to the same object instances, or are these copied as well? They are independant copies Is there a way to optimize this? 1. Obviously, use fewer threads. Do you have a component that otherwise benefits from many threads? Also note that it is almost always a better use of memory to increase the cache size than the number of threads (this is the size of each cache, not the total cache size). Are you aware of the activity and cache parameters tab in the database management part of the control panel? If not, you may not be using a 2.6x Zope - there are some significant improvements in cache usage (especially under load) in these versions. I'm working on a 2.3.x instance which I have already upgraded to 2.4.4. It soudns asif a further upgrde is reasonable. Also, the 'activity' tab is new to me (looks interesting ;), and caching settins seem to be ignored mostly until 2.6, right? Cheers Ivo -- Drs. I.R. van der Wijk -=- Brouwersgracht 132 Amaze Internet Services V.O.F. 1013 HA Amsterdam, NL -=- T +31-20-4688336 F +31-20-4688337Linux/Web/Zope/SQL/MMBase W http://www.amaze.nlE [EMAIL PROTECTED] Network Solutions W http://vanderwijk.info E [EMAIL PROTECTED] Consultancy PGP http://vanderwijk.info/pgp -=- ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Zope reference counts
On Wed, Jan 22, 2003 at 11:57:20AM +, Toby Dickenson wrote: Is there a way to optimize this? 1. Obviously, use fewer threads. Do you have a component that otherwise benefits from many threads? No, except that the server is pretty heavy loaded sometimes, and multiple threads should improve responsiveness (right?) 2. If you have large immutable state that does not hold references to persistent objects, use application-specific code to share this state between threads. Not really. The bloat is caused by a large squishdot site (20.000 comments) Anyway, I'll give 2.6 a try. Thanks! Ivo -- Drs. I.R. van der Wijk -=- Brouwersgracht 132 Amaze Internet Services V.O.F. 1013 HA Amsterdam, NL -=- T +31-20-4688336 F +31-20-4688337Linux/Web/Zope/SQL/MMBase W http://www.amaze.nlE [EMAIL PROTECTED] Network Solutions W http://vanderwijk.info E [EMAIL PROTECTED] Consultancy PGP http://vanderwijk.info/pgp -=- ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Zope reference counts
On Wednesday 22 January 2003 3:01 pm, Ivo van der Wijk wrote: It soudns asif a further upgrde is reasonable. I think you will have much more joy with 2.6 Also, the 'activity' tab is new to me (looks interesting ;), and caching settins seem to be ignored mostly until 2.6, right? it worked in a way that lead to strange behavior. 2.6 is more predictable, and understandable. -- Toby Dickenson http://www.geminidataloggers.com/people/tdickenson ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
[Zope-dev] Zope reference counts
Hi, I'm currently optimizing/analyzing a Zope server and I noticed the following: The number of references to specific objects (in this case specifically Squishdot comments) increases almost linear with the number of threads. Does this mean each thread keeps it's own cache pool of objects? Do these references point to the same object instances, or are these copied as well? I have the impression the memoryconsumption grows with the number of threads - is this true? Cheers Ivo -- Drs. I.R. van der Wijk -=- Brouwersgracht 132 Amaze Internet Services V.O.F. 1013 HA Amsterdam, NL -=- T +31-20-4688336 F +31-20-4688337Linux/Web/Zope/SQL/MMBase W http://www.amaze.nlE [EMAIL PROTECTED] Network Solutions W http://vanderwijk.info E [EMAIL PROTECTED] Consultancy PGP http://vanderwijk.info/pgp -=- ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Zope reference counts
On Tuesday 21 January 2003 4:14 pm, Ivo van der Wijk wrote: Does this mean each thread keeps it's own cache pool of objects? yes Do these references point to the same object instances, or are these copied as well? They are independant copies I have the impression the memoryconsumption grows with the number of threads - is this true? yes Cheers I hope this helps -- Toby Dickenson http://www.geminidataloggers.com/people/tdickenson ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
