RE: [ZODB-Dev] ZEO LoadEx?
I will try to find it as it was throw away code that we deleted since our investigations... But globally just a matter of importing zLOG and logging the ClientStorage loadEx calls to ._server.loadEx() (i.e. ClientStorage cache miss), using ZODB.utils for a readable representation of the oid. Pascal -Message d'origine- De : Chris Withers [mailto:[EMAIL PROTECTED] Envoyé : mercredi 21 juin 2006 11:19 À : Pascal Peregrina Cc : zodb-dev; Tim Peters Objet : Re: [ZODB-Dev] ZEO LoadEx? Pascal Peregrina wrote: We located it by hacking the ClientStorage code in order to display the real load operations (oids) from ZEO server (cache misses). This sounds like pretty useful code... Is it in a how-to somewhere or otherwise online? cheers, Chris -- Simplistix - Content Management, Zope Python Consulting - http://www.simplistix.co.uk ** This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error please notify the system manager. This footnote also confirms that this email message has been swept by MIMEsweeper for the presence of computer viruses. www.mimesweeper.com ** ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] ZEO LoadEx?
Chris Withers wrote at 2006-6-20 07:31 +0100: ... But, there also seems to be (to my untrained eye) random periods of slowness from then onwards, again, without any apparent signs of excess load, memory usage or disk i/o. I'm somewhat flumoxed as to where to go from here on the debugging front... I have seen such behaviour when we still used cosine ranking (with TextIndexNG): it needs excessive amounts of data and can flush all other objects from the caches. We dropped cosine ranking and the behaviour disappeared. An essential help for the analysis was an instrumentation (in our private Zope/ZODB version) of the ZEOServerStub code to log information about the fetched objects: their oid, their class, their size and how long fetching took. -- Dieter ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] ZEO LoadEx?
Tim Peters wrote: Sure, but no way to guess from here. The only thing I can really guess from the above is that your client is going to the server a lot to get data. Well, the client and the server are on the same machine, which isn't load or memory bound, and doesn't seem to be i/o bound either. revision in the ZEO client cache. If the sloth is acute for a period following client startup, but gets better over time, then the lack of a persistent client cache would be the obvious suspect. Right, that's true enough, and I'd also expect it. But, there also seems to be (to my untrained eye) random periods of slowness from then onwards, again, without any apparent signs of excess load, memory usage or disk i/o. I'm somewhat flumoxed as to where to go from here on the debugging front... Chris -- Simplistix - Content Management, Zope Python Consulting - http://www.simplistix.co.uk ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
RE: [ZODB-Dev] ZEO LoadEx?
Hi, This reminds me something I noticed when we migrated from 2.7 to 2.8 Our issue was a very big PersistentMapping based tree of objects, which was involved in a lot of RW and RO transactions from different Zope instances (we use ZEO of course). There was no miracle to solve the issue; we had to change the structure to BTree. Pascal -Message d'origine- De : [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] De la part de Chris Withers Envoyé : mardi 20 juin 2006 08:31 À : Tim Peters Cc : zodb-dev Objet : Re: [ZODB-Dev] ZEO LoadEx? Tim Peters wrote: Sure, but no way to guess from here. The only thing I can really guess from the above is that your client is going to the server a lot to get data. Well, the client and the server are on the same machine, which isn't load or memory bound, and doesn't seem to be i/o bound either. revision in the ZEO client cache. If the sloth is acute for a period following client startup, but gets better over time, then the lack of a persistent client cache would be the obvious suspect. Right, that's true enough, and I'd also expect it. But, there also seems to be (to my untrained eye) random periods of slowness from then onwards, again, without any apparent signs of excess load, memory usage or disk i/o. I'm somewhat flumoxed as to where to go from here on the debugging front... Chris -- Simplistix - Content Management, Zope Python Consulting - http://www.simplistix.co.uk ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev ** This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error please notify the system manager. This footnote also confirms that this email message has been swept by MIMEsweeper for the presence of computer viruses. www.mimesweeper.com ** ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] ZEO LoadEx?
Pascal Peregrina wrote: This reminds me something I noticed when we migrated from 2.7 to 2.8 Well, it's 2.7 to 2.9 here, but yeah, it's the same big jump ;-) Our issue was a very big PersistentMapping based tree of objects, which was involved in a lot of RW and RO transactions from different Zope instances (we use ZEO of course). There was no miracle to solve the issue; we had to change the structure to BTree. How come this wasn't a problem with 2.7? That said, AFAIK, I'm using BTrees everywhere already... Chris PS: Still, I'm curious, how did you locate the troublesome object? -- Simplistix - Content Management, Zope Python Consulting - http://www.simplistix.co.uk ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
RE: [ZODB-Dev] ZEO LoadEx?
We located it by hacking the ClientStorage code in order to display the real load operations (oids) from ZEO server (cache misses). I just read again this mail thread. The lack disk I/O surge looks different from the symptoms we saw here, so it must me a different issue. Sorry for the noise. Pascal -Message d'origine- De : Chris Withers [mailto:[EMAIL PROTECTED] Envoyé : mardi 20 juin 2006 10:32 À : Pascal Peregrina Cc : Tim Peters; zodb-dev Objet : Re: [ZODB-Dev] ZEO LoadEx? Pascal Peregrina wrote: This reminds me something I noticed when we migrated from 2.7 to 2.8 Well, it's 2.7 to 2.9 here, but yeah, it's the same big jump ;-) Our issue was a very big PersistentMapping based tree of objects, which was involved in a lot of RW and RO transactions from different Zope instances (we use ZEO of course). There was no miracle to solve the issue; we had to change the structure to BTree. How come this wasn't a problem with 2.7? That said, AFAIK, I'm using BTrees everywhere already... Chris PS: Still, I'm curious, how did you locate the troublesome object? -- Simplistix - Content Management, Zope Python Consulting - http://www.simplistix.co.uk ** This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error please notify the system manager. This footnote also confirms that this email message has been swept by MIMEsweeper for the presence of computer viruses. www.mimesweeper.com ** ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
