Re: [Zope-dev] bug tracker gardening for bugday
Hello Tres, You think we have a chance to get rid of old cruft? Like 5 years old nothing happened since then bugs. I think we should add a tag too old, add a note to warn subscribers, then set a status Won't fix after a month. Anyone unhappy with that should speak up or reopen the bug. Wednesday, April 21, 2010, 7:14:54 PM, you wrote: TS -BEGIN PGP SIGNED MESSAGE- TS Hash: SHA1 TS Tres Seaver wrote: Adam GROSZER wrote: Hello, I've added the tag bugday20100424 to some bugs for the bugday. Feel free to do the same with others. Here is the Launchpad tag page for it: https://bugs.launchpad.net/zopetoolkit/+bugs?field.tag=bugday20100424 I have updated the bugday wiki page with that link. TS I have also tagged a bunch of other bugs (mostly those with patches) TS with the tag -- it would be good to adopt a culture where bugs with TS patches (or branches) never languish. TS Tres. -- Best regards, Adam GROSZERmailto:agros...@gmail.com -- Quote of the day: Many people today don't want honest answers insofar as honest means unpleasant or disturbing. They want a soft answer that turneth away anxiety. - Louis Kronenberger ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] bug tracker gardening for bugday
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 4/22/10 09:24 , Adam GROSZER wrote: Hello Tres, You think we have a chance to get rid of old cruft? Like 5 years old nothing happened since then bugs. I think we should add a tag too old, add a note to warn subscribers, then set a status Won't fix after a month. Anyone unhappy with that should speak up or reopen the bug. +lots jens -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.8 (Darwin) iEYEARECAAYFAkvQBp4ACgkQRAx5nvEhZLI0rwCeNk1bRYU3KIyyVDulUThtu+Tq glsAoJvy1qIqMrHKwyRpTTPeHGDQSCIq =YplW -END PGP SIGNATURE- ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] bug tracker gardening for bugday
Hello, JV On 4/22/10 09:24 , Adam GROSZER wrote: Hello Tres, You think we have a chance to get rid of old cruft? Like 5 years old nothing happened since then bugs. I think we should add a tag too old, add a note to warn subscribers, then set a status Won't fix after a month. Anyone unhappy with that should speak up or reopen the bug. JV +lots JV jens What should be the timeframe? 5 years? 3 years? 2 years? -- Best regards, Adam GROSZERmailto:agros...@gmail.com -- Quote of the day: Life is short and we have never too much time for gladdening the hearts of those who are traveling the dark journey with us. Oh be swift to love, make haste to be kind. - Henri Frederick Amiel ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] bug tracker gardening for bugday
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 4/22/10 11:11 , Adam GROSZER wrote: Hello, JV On 4/22/10 09:24 , Adam GROSZER wrote: Hello Tres, You think we have a chance to get rid of old cruft? Like 5 years old nothing happened since then bugs. I think we should add a tag too old, add a note to warn subscribers, then set a status Won't fix after a month. Anyone unhappy with that should speak up or reopen the bug. JV +lots JV jens What should be the timeframe? 5 years? 3 years? 2 years? I'd say 3 years. However, I have no experience how other projects handle this. jens -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.8 (Darwin) iEYEARECAAYFAkvQFRwACgkQRAx5nvEhZLI9NACfbyMcUSaLfeqkABXM/m6vMyW9 W7sAnA8yVv11COfN8vbHikKDaPbL1wPY =Hyw9 -END PGP SIGNATURE- ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] bug tracker gardening for bugday
On Thu, Apr 22, 2010 at 09:24, Adam GROSZER agros...@gmail.com wrote: Hello Tres, You think we have a chance to get rid of old cruft? Like 5 years old nothing happened since then bugs. I think we should add a tag too old, add a note to warn subscribers, then set a status Won't fix after a month. There is a state not enough information or something like that, and we can close bugs that has been in that state for a period of time. If the bug can be reproduced, it should not be closed just because it's old, even if it's 8 years old. -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64 ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] RFC: Proposed new style for documenting and testing ZTK packages
Am 22.04.2010, 05:48 Uhr, schrieb Christian Theune c...@gocept.com: Could get a lint template (or whatever it's called) to help here? Not sure what that means, can you elaborate a bit? Lint is a tool that analyses code for all kinds of languages, including Python, and scores it against various definable metrics. The idea is to encourage code readability and best practices. I imagine that for more predictable parts of code like unit tests it might be possible to use it to encourage a uniform style such as TestMock seems to promise for VisualStudio http://site.typemock.com/test-lint/ I'll admit to not using PyLint (from Nuxeo) myself on a regular basis but this is as much down to not knowing how to configure it to work with Zope projects as much as anything else. But maybe we can add it to anything the repository policy checker? Charlie -- Charlie Clark Managing Director Clark Consulting Research German Office Helmholtzstr. 20 Düsseldorf D- 40215 Tel: +49-211-600-3657 Mobile: +49-178-782-6226 ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Zope Tests: 10 OK, 4 Failed
Summary of messages to the zope-tests list. Period Wed Apr 21 12:00:00 2010 UTC to Thu Apr 22 12:00:00 2010 UTC. There were 14 messages: 6 from Zope Tests, 7 from ccomb at free.fr, 1 from ct at gocept.com. Test failures - Subject: FAILED: Repository policy check found errors in 669 projects From: ct at gocept.com Date: Wed Apr 21 21:13:34 EDT 2010 URL: http://mail.zope.org/pipermail/zope-tests/2010-April/014055.html Subject: FAILED : ZTK 1.0dev / Python2.4.6 Linux 32bit From: ccomb at free.fr Date: Wed Apr 21 23:57:14 EDT 2010 URL: http://mail.zope.org/pipermail/zope-tests/2010-April/014066.html Subject: FAILED : ZTK 1.0dev / Python2.5.2 Linux 32bit From: ccomb at free.fr Date: Wed Apr 21 23:58:41 EDT 2010 URL: http://mail.zope.org/pipermail/zope-tests/2010-April/014067.html Subject: FAILED : ZTK 1.0dev / Python2.6.4 Linux 32bit From: ccomb at free.fr Date: Wed Apr 21 23:58:54 EDT 2010 URL: http://mail.zope.org/pipermail/zope-tests/2010-April/014068.html Tests passed OK --- Subject: OK : Zope-2.10 Python-2.4.6 : Linux From: Zope Tests Date: Wed Apr 21 21:30:50 EDT 2010 URL: http://mail.zope.org/pipermail/zope-tests/2010-April/014056.html Subject: OK : Zope-2.11 Python-2.4.6 : Linux From: Zope Tests Date: Wed Apr 21 21:32:50 EDT 2010 URL: http://mail.zope.org/pipermail/zope-tests/2010-April/014057.html Subject: OK : Zope-2.12 Python-2.6.4 : Linux From: Zope Tests Date: Wed Apr 21 21:34:50 EDT 2010 URL: http://mail.zope.org/pipermail/zope-tests/2010-April/014058.html Subject: OK : Zope-2.12-alltests Python-2.6.4 : Linux From: Zope Tests Date: Wed Apr 21 21:36:50 EDT 2010 URL: http://mail.zope.org/pipermail/zope-tests/2010-April/014059.html Subject: OK : Zope-trunk Python-2.6.4 : Linux From: Zope Tests Date: Wed Apr 21 21:38:50 EDT 2010 URL: http://mail.zope.org/pipermail/zope-tests/2010-April/014060.html Subject: OK : Zope-trunk-alltests Python-2.6.4 : Linux From: Zope Tests Date: Wed Apr 21 21:40:51 EDT 2010 URL: http://mail.zope.org/pipermail/zope-tests/2010-April/014061.html Subject: OK : BlueBream template / Python2.4.6 32bit linux From: ccomb at free.fr Date: Wed Apr 21 22:00:45 EDT 2010 URL: http://mail.zope.org/pipermail/zope-tests/2010-April/014063.html Subject: OK : BlueBream template / Python2.5.2 32bit linux From: ccomb at free.fr Date: Wed Apr 21 22:00:46 EDT 2010 URL: http://mail.zope.org/pipermail/zope-tests/2010-April/014062.html Subject: OK : BlueBream template / Python2.6.4 32bit linux From: ccomb at free.fr Date: Wed Apr 21 22:00:47 EDT 2010 URL: http://mail.zope.org/pipermail/zope-tests/2010-April/014064.html Subject: OK : BlueBream template / Python2.7b1 32bit linux From: ccomb at free.fr Date: Wed Apr 21 22:00:47 EDT 2010 URL: http://mail.zope.org/pipermail/zope-tests/2010-April/014065.html ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] bug tracker gardening for bugday
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Lennart Regebro wrote: On Thu, Apr 22, 2010 at 09:24, Adam GROSZER agros...@gmail.com wrote: Hello Tres, You think we have a chance to get rid of old cruft? Like 5 years old nothing happened since then bugs. I think we should add a tag too old, add a note to warn subscribers, then set a status Won't fix after a month. There is a state not enough information or something like that, and we can close bugs that has been in that state for a period of time. If the bug can be reproduced, it should not be closed just because it's old, even if it's 8 years old. Incomplete is the auto-expiring state: if nobody responds in 30 days, it automatically disappears. I agree that we should at least attempt to reproduce even really old bugs before assigning them 'wontfix' status (if we can't) or 'incomplete' (if we need more information). Tres. - -- === Tres Seaver +1 540-429-0999 tsea...@palladion.com Palladion Software Excellence by Designhttp://palladion.com -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkvQQvMACgkQ+gerLs4ltQ5akQCguLU/iVzTBp7FKb8u4zLcUyYY UiQAmgMOQKMasmg83GRG0JBdvZBtiu7H =z1bO -END PGP SIGNATURE- ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] zope2/zodb cache size question
hi there! today i did some sanity check calcs on a set of approx 15 objects in one of our zodb. for these calculations i had to iterate over all of these objects. during this loop memory footprint of the zope process grew to ~5gb and the cache summary showed for the zodb connection which was used: Connection at 24c10410263104 462834 -- XLhost.de - eXperts in Linux hosting ® XLhost.de GmbH Jürgen Herrmann, Geschäftsführer Boelckestrasse 21, 93051 Regensburg, Germany Geschäftsführer: Volker Geith, Jürgen Herrmann Registriert unter: HRB9918 Umsatzsteuer-Identifikationsnummer: DE245931218 Fon: +49 (0)800 XLHOSTDE [0800 95467833] Fax: +49 (0)800 95467830 WEB: http://www.XLhost.de IRC: #xlh...@irc.quakenet.org ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] zope2/zodb cache size question
hi there! today i ran a check script which iterates over approx 15 objects and does some sanity check calculations on these. during this loop i saw the zope process use up to about 4.5gb memory. the database has ~3.5million objects in it. i set the zodb cache size for the mount point in question to 1 objects. obviously this limit is not honoured during one transaction: connection active objects total objects ... Connection at 24c10410 263104 462834 ... so two questions here: - would the byte-limited zodb cache setting help here? - if no - how can i iterate over a big set of objects without forcing them to stay in the cache for the whole transaction? after all i just need each object once during the iteration. thanks in advance and best regards, jürgen herrmann -- XLhost.de - eXperts in Linux hosting ® XLhost.de GmbH Jürgen Herrmann, Geschäftsführer Boelckestrasse 21, 93051 Regensburg, Germany Geschäftsführer: Volker Geith, Jürgen Herrmann Registriert unter: HRB9918 Umsatzsteuer-Identifikationsnummer: DE245931218 Fon: +49 (0)800 XLHOSTDE [0800 95467833] Fax: +49 (0)800 95467830 WEB: http://www.XLhost.de IRC: #xlh...@irc.quakenet.org ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] zope2/zodb cache size question
On Thu, 22 Apr 2010 14:56:31 +0200, Jürgen Herrmann juergen.herrm...@xlhost.de wrote: hi there! today i did some sanity check calcs on a set of approx 15 objects in one of our zodb. for these calculations i had to iterate over all of these objects. during this loop memory footprint of the zope process grew to ~5gb and the cache summary showed for the zodb connection which was used: Connection at 24c10410263104 462834 please ignore this msg, i hit the send button and the mail was not finished yet. ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Bugday: Updated docs for contributors
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
I expanded and reorganized the developer docs yesterday:
http://docs.zope.org/developer/
In particular, I tried to pull in stuff which would aid new
contributors, including those who don't (yet) have commit access, or
maybe even don't want that responsibility.
The Sphinx docs for that site are in SVN:
http://svn.zope.org/developer_docs/trunk
Please help fix any errors or omissions you see. In particular, I ran
out of steam / time on two of the new pages ('reporting-bugs.rst' and
'sphinx-documentation.rst'): if somebody can help flesh them out before
bugday, that would be great! Adding something about bug triage and
workflow would be especially helpful.
Tres.
- --
===
Tres Seaver +1 540-429-0999 tsea...@palladion.com
Palladion Software Excellence by Designhttp://palladion.com
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
iEYEARECAAYFAkvQTAcACgkQ+gerLs4ltQ5lPwCfaYyl6YFNe2nbgz+3r8t9nM5u
gr4Anj9yyHrtpfRX7jiDzuFBjE3JM+XI
=Noq/
-END PGP SIGNATURE-
___
Zope-Dev maillist - Zope-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
https://mail.zope.org/mailman/listinfo/zope-announce
https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] bug tracker gardening for bugday
On Thu, Apr 22, 2010 at 2:37 PM, Tres Seaver tsea...@palladion.com wrote: Incomplete is the auto-expiring state: if nobody responds in 30 days, it automatically disappears. I agree that we should at least attempt to reproduce even really old bugs before assigning them 'wontfix' status (if we can't) or 'incomplete' (if we need more information). Speaking from my experience with cleaning up the Zope2 tracker: Zope is remarkably stable and there's some really old but still completely accurate tickets in the tracker. Just because anything is old, doesn't mean it's not valid anymore. I found some really good patches from 8 years back ;) I think you should use the incomplete status exactly when something is that: there's not enough information in the ticket to reproduce it. There's another category of tickets which describe problems with no longer maintained or outdated versions. Something along the lines of Zope X3.0b1 is missing a Windows binary installer. Those I'd close with an immediate won't fix. Hanno ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] zope2/zodb cache size question
On Thu, Apr 22, 2010 at 03:00:16PM +0200, Jürgen Herrmann wrote: today i ran a check script which iterates over approx 15 objects and does some sanity check calculations on these. during this loop i saw the zope process use up to about 4.5gb memory. the database has ~3.5million objects in it. i set the zodb cache size for the mount point in question to 1 objects. obviously this limit is not honoured during one transaction: connection active objects total objects ... Connection at 24c10410 263104 462834 ... so two questions here: - would the byte-limited zodb cache setting help here? - if no - how can i iterate over a big set of objects without forcing them to stay in the cache for the whole transaction? after all i just need each object once during the iteration. Use savepoints: for n, obj in enumerate(your_objects): perform_sanity_check(obj) if n % 1 == 0: transaction.savepoint() Real-world example: http://bazaar.launchpad.net/~schooltool-owners/schooltool/schooltool/annotate/head:/src/schooltool/generations/evolve26.py Marius Gedminas -- http://pov.lt/ -- Zope 3 consulting and development signature.asc Description: Digital signature ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Bugday: Updated docs for contributors
Hello Tres, Thursday, April 22, 2010, 3:15:56 PM, you wrote: TS -BEGIN PGP SIGNED MESSAGE- TS Hash: SHA1 TS I expanded and reorganized the developer docs yesterday: TS http://docs.zope.org/developer/ TS In particular, I tried to pull in stuff which would aid new TS contributors, including those who don't (yet) have commit access, or TS maybe even don't want that responsibility. Resources for Contributors * Zope IRC Channels * #zope e, I'm confused or missed something. How's that with #zope and #zope3-dev? -- Best regards, Adam GROSZERmailto:agros...@gmail.com -- Quote of the day: When you were born, you cried and the world rejoiced; live your life so that when you die, the world cries and you rejoice. - Cherokee Proverb ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Bugday: Updated docs for contributors
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Adam GROSZER wrote: Hello Tres, Thursday, April 22, 2010, 3:15:56 PM, you wrote: TS -BEGIN PGP SIGNED MESSAGE- TS Hash: SHA1 TS I expanded and reorganized the developer docs yesterday: TS http://docs.zope.org/developer/ TS In particular, I tried to pull in stuff which would aid new TS contributors, including those who don't (yet) have commit access, or TS maybe even don't want that responsibility. Resources for Contributors * Zope IRC Channels * #zope e, I'm confused or missed something. How's that with #zope and #zope3-dev? I thought #zope3-dev was dormant / disused, like the equivalent mailing lists. Tres. - -- === Tres Seaver +1 540-429-0999 tsea...@palladion.com Palladion Software Excellence by Designhttp://palladion.com -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkvQeRYACgkQ+gerLs4ltQ7k5gCfX5wCmvN03oEe561Iq1BeG7Px vUcAn0Und3e0J3HYWntAnVQl0m2ZvTYK =QfvH -END PGP SIGNATURE- ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Bugday: Updated docs for contributors
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Tres Seaver wrote: Adam GROSZER wrote: Hello Tres, Thursday, April 22, 2010, 3:15:56 PM, you wrote: TS -BEGIN PGP SIGNED MESSAGE- TS Hash: SHA1 TS I expanded and reorganized the developer docs yesterday: TS http://docs.zope.org/developer/ TS In particular, I tried to pull in stuff which would aid new TS contributors, including those who don't (yet) have commit access, or TS maybe even don't want that responsibility. Resources for Contributors * Zope IRC Channels * #zope e, I'm confused or missed something. How's that with #zope and #zope3-dev? I thought #zope3-dev was dormant / disused, like the equivalent mailing lists. It looks as though the Subversion typo brownies have been cleaning up my mistakes and oversights, including this one. Reminding-myself-to-set-out-a-bowl-of-milk'ly, Tres. - -- === Tres Seaver +1 540-429-0999 tsea...@palladion.com Palladion Software Excellence by Designhttp://palladion.com -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkvQelgACgkQ+gerLs4ltQ4xZwCeIgsJvFtS9u/cmG6Pw3pvXoa7 6G4AoJIuYJQA3vgiv7CUUDE7Ew53ismA =BYkk -END PGP SIGNATURE- ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] zope2/zodb cache size question
On Thu, Apr 22, 2010 at 9:00 AM, Jürgen Herrmann juergen.herrm...@xlhost.de wrote: hi there! today i ran a check script which iterates over approx 15 objects and does some sanity check calculations on these. during this loop i saw the zope process use up to about 4.5gb memory. the database has ~3.5million objects in it. i set the zodb cache size for the mount point in question to 1 objects. obviously this limit is not honoured during one transaction: Right. so two questions here: - would the byte-limited zodb cache setting help here? No. The limits aren't really limits. They are more like suggestions. :) In particular, they are only enforced at transaction (or subtransaction) boundaries, when a connection is opened or closed, or when applications call certain APIs explicitly. - if no - how can i iterate over a big set of objects without forcing them to stay in the cache for the whole transaction? after all i just need each object once during the iteration. As you're iterating through the objects, if you know you aren't going to need an object again or otherwise think it would be OK to free it from memory, you can call _p_deactivate on it. If the object hasn't been modified, then it's state will be released and it will become a ghost. Jim -- Jim Fulton ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] zope2/zodb cache size question
On Thu, 22 Apr 2010 12:54:55 -0400, Jim Fulton j...@zope.com wrote: On Thu, Apr 22, 2010 at 9:00 AM, Jürgen Herrmann juergen.herrm...@xlhost.de wrote: hi there! today i ran a check script which iterates over approx 15 objects and does some sanity check calculations on these. during this loop i saw the zope process use up to about 4.5gb memory. the database has ~3.5million objects in it. i set the zodb cache size for the mount point in question to 1 objects. obviously this limit is not honoured during one transaction: Right. so two questions here: - would the byte-limited zodb cache setting help here? No. The limits aren't really limits. They are more like suggestions. :) In particular, they are only enforced at transaction (or subtransaction) boundaries, when a connection is opened or closed, or when applications call certain APIs explicitly. are these apis exposed somewhere? calling these periodically sounds cleaner than the aproach below. - if no - how can i iterate over a big set of objects without forcing them to stay in the cache for the whole transaction? after all i just need each object once during the iteration. As you're iterating through the objects, if you know you aren't going to need an object again or otherwise think it would be OK to free it from memory, you can call _p_deactivate on it. If the object hasn't been modified, then it's state will be released and it will become a ghost. as my objects contain references to OOBTrees, do i also have to call _p_deactivate() on these explicitly, or is it enough to call it on the containing object? thanks for your help! best regards, jürgen -- XLhost.de - eXperts in Linux hosting ® XLhost.de GmbH Jürgen Herrmann, Geschäftsführer Boelckestrasse 21, 93051 Regensburg, Germany Geschäftsführer: Volker Geith, Jürgen Herrmann Registriert unter: HRB9918 Umsatzsteuer-Identifikationsnummer: DE245931218 Fon: +49 (0)800 XLHOSTDE [0800 95467833] Fax: +49 (0)800 95467830 WEB: http://www.XLhost.de IRC: #xlh...@irc.quakenet.org ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] zope2/zodb cache size question
On Thu, Apr 22, 2010 at 1:19 PM, Jürgen Herrmann juergen.herrm...@xlhost.de wrote: On Thu, 22 Apr 2010 12:54:55 -0400, Jim Fulton j...@zope.com wrote: On Thu, Apr 22, 2010 at 9:00 AM, Jürgen Herrmann juergen.herrm...@xlhost.de wrote: hi there! today i ran a check script which iterates over approx 15 objects and does some sanity check calculations on these. during this loop i saw the zope process use up to about 4.5gb memory. the database has ~3.5million objects in it. i set the zodb cache size for the mount point in question to 1 objects. obviously this limit is not honoured during one transaction: Right. so two questions here: - would the byte-limited zodb cache setting help here? No. The limits aren't really limits. They are more like suggestions. :) In particular, they are only enforced at transaction (or subtransaction) boundaries, when a connection is opened or closed, or when applications call certain APIs explicitly. are these apis exposed somewhere? calling these periodically sounds cleaner than the aproach below. - if no - how can i iterate over a big set of objects without forcing them to stay in the cache for the whole transaction? after all i just need each object once during the iteration. As you're iterating through the objects, if you know you aren't going to need an object again or otherwise think it would be OK to free it from memory, you can call _p_deactivate on it. If the object hasn't been modified, then it's state will be released and it will become a ghost. as my objects contain references to OOBTrees, do i also have to call _p_deactivate() on these explicitly, or is it enough to call it on the containing object? You need to call it on any objects you want to go away. Another option is to periodically call cacheGC() on the connection object: some_object._p_jar.cacheGC() This will apply the limits, to the degree it can. As with _p_deactivate, only unmodified objects will be removed from memory. I'm assuming in this discussion that you're only analyzing, not updating objects. Jim -- Jim Fulton ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
