Re: Hooking persistent.Persistent.__setstate__ was Re: [ZODB-Dev] Analyzing a ZODB.
Alan Runyan wrote: - Customer has software on a remote machine. They are seeing unnecessary transaction commits. Just like the guy 'Analyzing a ZODB'. I'm that guy ;). BTW, we have related those unnecessary commits to CMFQuestions, an old plone product now superseded by PloneSurveys... We came to that not by inspecting the code, but by realizing there were too many conflicts related with CMFQuestionnaire. We removed it, and the commits vanished. Best regards, Manuel. ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: Hooking persistent.Persistent.__setstate__ was Re: [ZODB-Dev] Analyzing a ZODB.
Hello Alan, Sunday, April 6, 2008, 12:30:21 AM, you wrote: AR Question: Is it possible for ZODB 3.9 to have a pure python AR implementation of persistent.Persistent? Maybe this would be a good AR ZODB GSOC project? That would give some help to the Zope 3 components on Jython project too ;-) Maybe you should post that on [EMAIL PROTECTED] btw, there is also a proposal on the GSoC list to further elaborate the zodbbrowser. svn://svn.zope.org/repos/main/z3c.zodbbrowser/sandbox/src/z3c/zodbbrowser Any ideas welcome. -- Best regards, Adam GROSZERmailto:[EMAIL PROTECTED] -- Quote of the day: The price of success is perseverance. The price of failure comes cheaper. (Anonymous) ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] Analyzing a ZODB.
Manuel Vazquez Acosta wrote at 2008-4-5 11:49 -0400: ... I wonder if there's a way to actually see what objects (or object types) are modified by those transactions. So I can go directly to the source of the (surely innecesary) transaction. The ZODB utility fsdump generates a human readable view of your storage. Among others, you see which transactions have been committed (together with all transaction metadata) and which objects was modified (identified by the oid (and I think, their class)). -- Dieter ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] Analyzing a ZODB.
On Apr 5, 2008, at 11:49 AM, Manuel Vazquez Acosta wrote: Hi all, Recently I was appointed to improve a Plone's performance, and I have the following issue: I see too many transactions made by the anonymous user. I can reproduce this behaviour: just watching the home page produces a transaction at /index by None. I wonder if there's a way to actually see what objects (or object types) are modified by those transactions. So I can go directly to the source of the (surely innecesary) transaction. I tried analyze.py but I don't grasp the output accurately. In a development environment, set a breakpoint (e.g. add import pdb; pdb.set_trace() in ZODB.Connection.Connection.register. You'll be able to see exactly what is causing object changes. I recommend doing this in the zope debugger, which makes it easy to run one request at a time. Jim -- Jim Fulton Zope Corporation ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] Analyzing a ZODB.
In a development environment, set a breakpoint (e.g. add import pdb; pdb.set_trace() in ZODB.Connection.Connection.register. You'll be able to see exactly what is causing object changes. I recommend doing this in the zope debugger, which makes it easy to run one request at a time. Dear Mr. Fulton, Thanks for your quick response By the zope debugger you mean this: http://www.simplistix.co.uk/software/zope/zdb ?? Best regards, Manuel. ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] Analyzing a ZODB.
On Apr 5, 2008, at 12:09 PM, Manuel Vazquez Acosta wrote: In a development environment, set a breakpoint (e.g. add import pdb; pdb.set_trace() in ZODB.Connection.Connection.register. You'll be able to see exactly what is causing object changes. I recommend doing this in the zope debugger, which makes it easy to run one request at a time. Dear Mr. Fulton, Thanks for your quick response By the zope debugger you mean this: http://www.simplistix.co.uk/software/zope/zdb ?? No, I mean the built-in debugger you get when you run: zopectl debug Get someone familiar with Zope to explain it to you. Jim -- Jim Fulton Zope Corporation ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Hooking persistent.Persistent.__setstate__ was Re: [ZODB-Dev] Analyzing a ZODB.
Here is something more ZODB and less Zope related (kinda). I was talking with Benji a few weeks ago about a problem that should be easy to debug but was not. Here is the scenerio: - Customer has software on a remote machine. They are seeing unnecessary transaction commits. Just like the guy 'Analyzing a ZODB'. - Customer is completely incapable of doing anything other than putting a script on the filesystem. - Benji and I thought about it and he proposed 'the simplest thing that c/should work, monkey patch persistent.__setstate__' so that I could see what objects were being mutated. - Unfortunately ZODB 3.x does not have a Python fallback of persistent.Persistent -- its in C. The customer did not have a C compiler on their box. IIRC how I solved it was increase ZEO event log to see the oid's. Then I walked him through loading the oid up to see what object was being mutated. It was more painful than it should have been. Question: Is it possible for ZODB 3.9 to have a pure python implementation of persistent.Persistent? Maybe this would be a good ZODB GSOC project? Notes: - ZODB 4.x did have a pure python impl of persistent.Persistent but that project did not make it into production. - Seemingly Jim F. said there was a 'best practice' in writing C extensions in Python, first write reference impl in Python and to write tests around the impl. Then re-implement in C. ZODB 3.x, never had this 'best practice' put into play. - increasing zeo server log level and watching oid's being changed is sort-of the equivalent of turning on RDBMS logging to see SQL stmt's being executed. Unfortunately I believe without having a hook in persistent.Persistent we can never really get that level of granularity (i.e. __getattribute__ is only accessible in client) with only ZEO server logs. - We could monkey patch the subclasses that use persistent. But in the world of non-Zope2 applications usually there are very thin classes that may not have any common base class other than persistent.Persistent. This is a transparent plea for a new feature. But I believe it would significantly help people writing ZODB applications. Maybe people should always have a base class that you override those methods which delegate to persistent.Persistent. i.e. class MyMixin(persistent.Persistent) and mixin MyMixin instead of mixin persistent.Persistent directly? Then you can instrument the MyMixin with the logging? Another GSOC idea: Best Practices of ZODB Programming. cheers -- Alan Runyan Enfold Systems, Inc. http://www.enfoldsystems.com/ phone: +1.713.942.2377x111 fax: +1.832.201.8856 ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Hooking persistent.Persistent.__setstate__ was Re: [ZODB-Dev] Analyzing a ZODB.
Here is something more ZODB and less Zope related (kinda). I was talking with Benji a few weeks ago about a problem that should be easy to debug but was not. Here is the scenerio: - Customer has software on a remote machine. They are seeing unnecessary transaction commits. Just like the guy 'Analyzing a ZODB'. - Customer is completely incapable of doing anything other than putting a script on the filesystem. - Benji and I thought about it and he proposed 'the simplest thing that c/should work, monkey patch persistent.__setstate__' so that I could see what objects were being mutated. - Unfortunately ZODB 3.x does not have a Python fallback of persistent.Persistent -- its in C. The customer did not have a C compiler on their box. IIRC how I solved it was increase ZEO event log to see the oid's. Then I walked him through loading the oid up to see what object was being mutated. It was more painful than it should have been. Question: Is it possible for ZODB 3.9 to have a pure python implementation of persistent.Persistent? Maybe this would be a good ZODB GSOC project? Notes: - ZODB 4.x did have a pure python impl of persistent.Persistent but that project did not make it into production. - Seemingly Jim F. said there was a 'best practice' in writing C extensions in Python, first write reference impl in Python and to write tests around the impl. Then re-implement in C. ZODB 3.x, never had this 'best practice' put into play. - increasing zeo server log level and watching oid's being changed is sort-of the equivalent of turning on RDBMS logging to see SQL stmt's being executed. Unfortunately I believe without having a hook in persistent.Persistent we can never really get that level of granularity (i.e. __getattribute__ is only accessible in client) with only ZEO server logs. - We could monkey patch the subclasses that use persistent. But in the world of non-Zope2 applications usually there are very thin classes that may not have any common base class other than persistent.Persistent. This is a transparent plea for a new feature. But I believe it would significantly help people writing ZODB applications. Maybe people should always have a base class that you override those methods which delegate to persistent.Persistent. i.e. class MyMixin(persistent.Persistent) and mixin MyMixin instead of mixin persistent.Persistent directly? Then you can instrument the MyMixin with the logging? Another GSOC idea: Best Practices of ZODB Programming. cheers -- Alan Runyan Enfold Systems, Inc. http://www.enfoldsystems.com/ phone: +1.713.942.2377x111 fax: +1.832.201.8856 -- Alan Runyan Enfold Systems, Inc. http://www.enfoldsystems.com/ phone: +1.713.942.2377x111 fax: +1.832.201.8856 ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
