Re: [Zope-dev] Session concurrency problem
On Wednesday 30 January 2008, Dieter Maurer wrote: 2. Sometimes, the session package data does not have the data in it and fails with a key error. This happens maybe 1 in 50 times and I have not reliably reproduced this problem. What is session package data? A very simple object: class SessionPkgData(persistent.Persistent, IterableUserDict): See zope.session.interfaces.ISessionData session = SessionPkgData() ISessionPkgData.providedBy(session) True zope.interface.implements(ISessionPkgData) def __init__(self): self.data = OOBTree() (zope.session.session, line 517) Regards, Stephan -- Stephan Richter Web Software Design, Development and Training Google me. Zope Stephan Richter ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Session concurrency problem
Hi everyone, I have come across a situation, where temporarily the session data for a user is not available. Scenario: User 1 creates some session data. While a user 2 creates also some session data, user 1's data is not accessible. The problem plays out in two ways: 1. The data's __dict__ returns an empty dictionary. When you access an attribute directly, though, the data can be seen. This symptom can be reliably reproduced. 2. Sometimes, the session package data does not have the data in it and fails with a key error. This happens maybe 1 in 50 times and I have not reliably reproduced this problem. There are no conflict errors. And when I store the data to the root folder using the client id, I never see any of the two problems. This makes me believe that the issue is contained to the session code. The only two things that seem specific to the session code is the sweeping algorithm. The comment in zope.session.session, line 192 discusses some issues of sweep, but I am not sure it is relevant. I have attached a package that demonstrates the issue. (This is one of the major benefits of having buildout and eggs!) Here is how you get the problem reproduced: tar xvzf sessionIssue2.tgz cd sessionIssue2 python2.4 bootstrap.py ./bin/buildout -N ./bin/demo fg In a separate shell: cd sessionIssue2 for number in 1 2 3 4 5 6 7 8 ;do sh sessionTest.sh;done The before and after result should always be the same. You can change the demo code to not use sessions and see that it works. You can also see the differences between accessing __dict__ and an attribute directly. So, does anyone have an idea of what is happening here? Any hints are greatly appreciated! Regards, Stephan -- Stephan Richter Web Software Design, Development and Training Google me. Zope Stephan Richter sessionIssue2.tgz Description: application/tgz ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Session concurrency problem
Stephan Richter wrote at 2008-1-30 08:45 -0500: ... 1. The data's __dict__ returns an empty dictionary. When you access an attribute directly, though, the data can be seen. This symptom can be reliably reproduced. This is normal for persistent objects. When a persistent object is in Ghost state, its __dict__ appears to be empty (it is empty). When you access a non special attribute, the object is unghosted and then __dict__ is filled in. 2. Sometimes, the session package data does not have the data in it and fails with a key error. This happens maybe 1 in 50 times and I have not reliably reproduced this problem. What is session package data? -- Dieter ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] SESSION values only refreshing with Shift-reload
I have a basic DTML document that has several input fields on it. When the user submits the form, validation occurs and goes to an error page, if need be. On my error page (dtml-document), there is a link to go back to the previous page to correct your errors. However, when I click on that link, my SESSION values are not displaying. But if I press Shift-Reload, I see my values. Any ideas?Upon Submitting the first form, here is what happens:populate_session_vars (This sets the SESSION values based on what was entered on the form ie. dtml-call REQUEST.SESSION.set('variablename', REQUEST.form['variablename']) )validate_form (This validates that the required fields are filled in)If there's an error, the user goes to the error page and from there, clicks on a link to go back to the main form to fix their errors (dtml-document).If anyone knows what is going on, please respond quickly.
I am under a tight deadline.Thanks so much.
All-new Yahoo! Mail - Fire up a more powerful email and get things done faster.___
Zope-Dev maillist - Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] SESSION values only refreshing with Shift-reload
Tina Matter wrote at 2006-9-29 06:57 -0700: I have a basic DTML document that has several input fields on it. When the user submits the form, validation occurs and goes to an error page, if need be. On my error page (dtml-document), there is a link to go back to the previous page to correct your errors. However, when I click on that link, my SESSION values are not displaying. But if I press Shift-Reload, I see my values. Any ideas? Wheneven something bad happens and vanishes afte a Shift-Reload, you probably have a caching problem. Search for cache-control and the a corresponding response header such that you document will not be cached. -- Dieter ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] SESSION values only refreshing with Shift-reload
Hi Tina,
Tina Matter wrote:
I have a basic DTML document that has several input fields on it.
I'm pretty sure you're looking for the zope@zope.org list ;-)
On my error page (dtml-document),
Use ZPT instead, or Twiddler when it's out ;-)
However, when I click on that link, my SESSION values are not displaying. But
if I press Shift-Reload, I see my values. Any ideas?
Sounds like IE caching, have you tried using Firefox?
populate_session_vars (This sets the SESSION values based on what was entered on the
form ie. dtml-call REQUEST.SESSION.set('variablename',
REQUEST.form['variablename']) )
, you really need to learn to use Python Scripts ;-)
If anyone knows what is going on, please respond quickly. I am under a tight
deadline.
...then pay someone to fix it for you 0.5 wink
Chris
--
Simplistix - Content Management, Zope Python Consulting
- http://www.simplistix.co.uk
___
Zope-Dev maillist - Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] session cookies newbie question
Hi All, I would like to access the browser's session cookie values. By browser's (or clients) session cookies I mean those cookies that have the expiry time set 'at end of session' and are not stored in the cookie.txt. Is there something in Zope's API that can help me do it. I need the solution for Mozilla and Microsoft based browsers. Either Zope or general python API pointer would be much appreciated. Thanks Regards Serge ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] session error
A user got this error trying to edit content. It only happened once, and on second try she was ok. Using Zope 2.7.0: Traceback (innermost last): Module ZPublisher.Publish, line 100, in publish Module ZPublisher.mapply, line 88, in mapply Module ZPublisher.Publish, line 40, in call_object Module Products.CMFFormController.FSControllerPageTemplate, line 81, in __call__ Module Products.CMFFormController.BaseControllerPageTemplate, line 42, in _call Module Shared.DC.Scripts.Bindings, line 306, in __call__ Module Shared.DC.Scripts.Bindings, line 343, in _bindAndExec Module Products.CMFCore.FSPageTemplate, line 191, in _exec Module Products.CMFCore.FSPageTemplate, line 124, in pt_render Module Products.PageTemplates.PageTemplate, line 96, in pt_render - FSControllerPageTemplate at /saultdiocese/base_edit used for /saultdiocese/DioceseOffice.2004-04-26.0049447933/WorkspaceFolder.2004-04-26.0631977769/Workspace.2004-07-15.1703647729/News.2004-10-13.7108192668/en Module TAL.TALInterpreter, line 189, in __call__ [ some TALInterpreter lines snipped ] Module TAL.TALInterpreter, line 459, in do_setLocal_tal Module Products.PageTemplates.TALES, line 220, in evaluate Module Products.PageTemplates.ZRPythonExpr, line 47, in __call__ - __traceback_info__: widget.postback and request.get(fieldName, here.session_restore_value(fieldName, value)) or value Module Python expression widget.postback and request.get(fieldName, here.session_restore_value(fieldName, value)) or value, line 1, in expression Module Products.CMFCore.FSPythonScript, line 104, in __call__ Module Shared.DC.Scripts.Bindings, line 306, in __call__ Module Shared.DC.Scripts.Bindings, line 343, in _bindAndExec Module Products.CMFCore.FSPythonScript, line 160, in _exec Module None, line 2, in session_restore_value - FSPythonScript at /saultdiocese/session_restore_value used for /saultdiocese/DioceseOffice.2004-04-26.0049447933/WorkspaceFolder.2004-04-26.0631977769/Workspace.2004-07-15.1703647729/News.2004-10-13.7108192668/en - Line 2 Module ZPublisher.HTTPRequest, line 1218, in __getattr__ Module ZPublisher.HTTPRequest, line 1178, in get Module Products.Sessions.SessionDataManager, line 93, in getSessionData Module Products.Sessions.SessionDataManager, line 180, in _getSessionDataObject Module Products.Transience.Transience, line 176, in new_or_existing Module Products.Transience.Transience, line 799, in get Module Products.Transience.Transience, line 548, in _getCurrentBucket Module ZODB.Connection, line 559, in setstate Module tempstorage.TemporaryStorage, line 94, in load KeyError: '\x00\x00\x00\x00\x00\x00\x05B' ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] session error
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Wed, 20 Oct 2004 04:23 am, Gerry Kirk wrote: A user got this error trying to edit content. It only happened once, and on second try she was ok. Using Zope 2.7.0: 2.7.3 (currently beta, but live on our production server) fixes errors of this sort. Richard -BEGIN PGP SIGNATURE- Version: GnuPG v1.2.4 (GNU/Linux) iD8DBQFBdcksrGisBEHG6TARAvDvAJ4oHmYvLcZG+ltfW7+CIyemzVHimACfc3lP 0g5TiJFRGSB5LFEn2qK27hM= =KedN -END PGP SIGNATURE- ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] session error
--On Mittwoch, 20. Oktober 2004 12:10 Uhr +1000 Richard Jones [EMAIL PROTECTED] wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Wed, 20 Oct 2004 04:23 am, Gerry Kirk wrote: A user got this error trying to edit content. It only happened once, and on second try she was ok. Using Zope 2.7.0: 2.7.3 (currently beta, but live on our production server) fixes errors of this sort. nice to hear that this error went away (hopefully finally)...hail ChrisM. -aj ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] session problem again?
Matteo, I'm afraid that there's just no simple way turn Zope 2.6.X into a Zope that has all the fixes and patches of Zope 2.7.3. It's not an effort you can succeed at under the type of time pressure you seem to be under. A much better tact would be to make your application work under Zope 2.7.3. What prevents it from running under 2.7.3 now? - C On Tue, 2004-09-28 at 19:07, Matteo Memelli wrote: Hello I include the conversation that I had today with Chris: I'm using Zope 2.6b1+ with ZWarehouse e-commerce product (ZWarehouse-20040522 version 0.8) I tried upgrading to Zope 2.7 but ZWarehouse gave me many problems so I decided to try applying your patches for Sessioning problems that I have. Hmm.. I think that the error that zope gives to me is very known to you... I tried to copy the whole directories Transience, Session and ZPublisher from both Zope-2.7.3b1-0 and Zope 2.7.2 but without success I also copied the file startup.py from lib/python/Zope/App/ ... Am I missing anything? Yes, the ZODB directory also... although I reallly have no idea whether this will work under 2.6. Really, to be sure you've got all the right files, it'd be best if you could upgrade to 2.7.3... maybe we could help fix whatever in ZWarehouse doesn't work under there (move this conversation to the Zope list [EMAIL PROTECTED]) To be thorough, you will also need to delete and recreate any persistent session data container you've got laying around (unless it's in temp_folder, that's in RAM anyway and gets recreated on reboot). My customer is complaining so much and I really don't know what to do! Hoping in your answer...thank you in advance Matteo Memelli Site Error An error was encountered while publishing this resource. KeyError Sorry, a site error occurred. Traceback (innermost last): * Module ZPublisher.Publish, line 175, in publish_module_standard * Module Products.iHotfix, line 83, in new_publish * Module ZPublisher.Publish, line 132, in publish * Module Zope.App.startup, line 203, in zpublisher_exception_hook * Module ZPublisher.Publish, line 101, in publish * Module ZPublisher.mapply, line 88, in mapply * Module ZPublisher.Publish, line 39, in call_object * Module OFS.DTMLDocument, line 128, in __call__ DTMLDocument instance at 41725260 URL: http://www.arte-luce.com/catalog/html/catalog.html/manage_main Physical Path:/arteluce/catalog/html/catalog.html * Module DocumentTemplate.DT_String, line 474, in __call__ * Module DocumentTemplate.DT_Let, line 76, in render * Module OFS.DTMLDocument, line 121, in __call__ DTMLDocument instance at 4172b6b0 URL: http://www.arte-luce.com/catalog/html/zwarehouse_html_header/manage_main Physical Path:/arteluce/catalog/html/zwarehouse_html_header * Module DocumentTemplate.DT_String, line 474, in __call__ * Module DocumentTemplate.DT_Util, line 201, in eval __traceback_info__: cart_functions * Module string, line 1, in expression * Module Shared.DC.Scripts.Bindings, line 306, in __call__ * Module Shared.DC.Scripts.Bindings, line 343, in _bindAndExec * Module Products.PythonScripts.PythonScript, line 318, in _exec * Module None, line 16, in setSessionByRequest.py PythonScript at /arteluce/catalog/cart_functions/setSessionByRequest.py Line 16 * Module ZPublisher.HTTPRequest, line 1218, in __getattr__ * Module ZPublisher.HTTPRequest, line 1178, in get * Module Products.Sessions.SessionDataManager, line 94, in getSessionData * Module Products.Sessions.SessionDataManager, line 181, in _getSessionDataObject * Module Products.Transience.Transience, line 847, in new_or_existing * Module Products.Transience.Transience, line 450, in __setitem__ KeyError: 1096365080 (Also, an error occurred while attempting to render the standard error message.) ___ Troubleshooting Suggestions * The URL may be incorrect. * The parameters passed to this resource may be incorrect. * A resource that this resource relies on may be encountering an error. For more detailed information about the error, please refer to error log. If the error persists please contact the site maintainer. Thank you for your patience.
[Zope-dev] session problem again?
Hello I include the conversation that I had today with Chris: I'm using Zope 2.6b1+ with ZWarehouse e-commerce product (ZWarehouse-20040522 version 0.8) I tried upgrading to Zope 2.7 but ZWarehouse gave me many problems so I decided to try applying your patches for Sessioning problems that I have. Hmm.. I think that the error that zope gives to me is very known to you... I tried to copy the whole directories Transience, Session and ZPublisher from both Zope-2.7.3b1-0 and Zope 2.7.2 but without success I also copied the file startup.py from lib/python/Zope/App/ ... Am I missing anything? Yes, the ZODB directory also... although I reallly have no idea whether this will work under 2.6. Really, to be sure you've got all the right files, it'd be best if you could upgrade to 2.7.3... maybe we could help fix whatever in ZWarehouse doesn't work under there (move this conversation to the Zope list [EMAIL PROTECTED]) To be thorough, you will also need to delete and recreate any persistent session data container you've got laying around (unless it's in temp_folder, that's in RAM anyway and gets recreated on reboot). My customer is complaining so much and I really don't know what to do! Hoping in your answer...thank you in advance Matteo Memelli Site Error An error was encountered while publishing this resource. KeyError Sorry, a site error occurred. Traceback (innermost last): * Module ZPublisher.Publish, line 175, in publish_module_standard * Module Products.iHotfix, line 83, in new_publish * Module ZPublisher.Publish, line 132, in publish * Module Zope.App.startup, line 203, in zpublisher_exception_hook * Module ZPublisher.Publish, line 101, in publish * Module ZPublisher.mapply, line 88, in mapply * Module ZPublisher.Publish, line 39, in call_object * Module OFS.DTMLDocument, line 128, in __call__ DTMLDocument instance at 41725260 URL: http://www.arte-luce.com/catalog/html/catalog.html/manage_main Physical Path:/arteluce/catalog/html/catalog.html * Module DocumentTemplate.DT_String, line 474, in __call__ * Module DocumentTemplate.DT_Let, line 76, in render * Module OFS.DTMLDocument, line 121, in __call__ DTMLDocument instance at 4172b6b0 URL: http://www.arte-luce.com/catalog/html/zwarehouse_html_header/manage_main Physical Path:/arteluce/catalog/html/zwarehouse_html_header * Module DocumentTemplate.DT_String, line 474, in __call__ * Module DocumentTemplate.DT_Util, line 201, in eval __traceback_info__: cart_functions * Module string, line 1, in expression * Module Shared.DC.Scripts.Bindings, line 306, in __call__ * Module Shared.DC.Scripts.Bindings, line 343, in _bindAndExec * Module Products.PythonScripts.PythonScript, line 318, in _exec * Module None, line 16, in setSessionByRequest.py PythonScript at /arteluce/catalog/cart_functions/setSessionByRequest.py Line 16 * Module ZPublisher.HTTPRequest, line 1218, in __getattr__ * Module ZPublisher.HTTPRequest, line 1178, in get * Module Products.Sessions.SessionDataManager, line 94, in getSessionData * Module Products.Sessions.SessionDataManager, line 181, in _getSessionDataObject * Module Products.Transience.Transience, line 847, in new_or_existing * Module Products.Transience.Transience, line 450, in __setitem__ KeyError: 1096365080 (Also, an error occurred while attempting to render the standard error message.) ___ Troubleshooting Suggestions * The URL may be incorrect. * The parameters passed to this resource may be incorrect. * A resource that this resource relies on may be encountering an error. For more detailed information about the error, please refer to error log. If the error persists please contact the site maintainer. Thank you for your patience. Hmmm.. that's an odd place to get that error. I'm surprised. Does it only happen after some period of inactivity on the site? - C FOLLOWING Chris's advice first I tried to copy also ZODB directory from Zope 2.7.3 but still the same error... so I Installed Zope-2.7.3b1-0 (IS THIS THE RIGHT VERSION THAT SHOULD SOLVE SESSION PROBLEMS?) and reinstalled with success the following packages: ZWarehouse 0.8 iHotfix-0.4.0 itools-0.4.6 Localizer-1.1.0a4 ZMySQLDA-2.0.8 MySQL-python-1.0.0 TranslationService-0.4 THEN imported my customer's web site but I still
Re: [Zope-dev] session problem again?
On 29/09/2004, at 11:07 AM, Matteo Memelli wrote: Hello I include the conversation that I had today with Chris: I'm using Zope 2.6b1+ with ZWarehouse e-commerce product (ZWarehouse-20040522 version 0.8) I tried upgrading to Zope 2.7 but ZWarehouse gave me many problems so I decided to try applying your patches for Sessioning problems that I have. Hmm.. I think that the error that zope gives to me is very known to you... I tried to copy the whole directories Transience, Session and ZPublisher from both Zope-2.7.3b1-0 and Zope 2.7.2 but without success I also copied the file startup.py from lib/python/Zope/App/ ... Am I missing anything? Yes, the ZODB directory also... although I reallly have no idea whether this will work under 2.6. Really, to be sure you've got all the right files, it'd be best if you could upgrade to 2.7.3... maybe we could help fix whatever in ZWarehouse doesn't work under there (move this conversation to the Zope list [EMAIL PROTECTED]) To be thorough, you will also need to delete and recreate any persistent session data container you've got laying around (unless it's in temp_folder, that's in RAM anyway and gets recreated on reboot). My customer is complaining so much and I really don't know what to do! Hoping in your answer...thank you in advance Matteo Memelli Site Error An error was encountered while publishing this resource. KeyError Sorry, a site error occurred. Traceback (innermost last): * Module ZPublisher.Publish, line 175, in publish_module_standard * Module Products.iHotfix, line 83, in new_publish * Module ZPublisher.Publish, line 132, in publish * Module Zope.App.startup, line 203, in zpublisher_exception_hook * Module ZPublisher.Publish, line 101, in publish * Module ZPublisher.mapply, line 88, in mapply * Module ZPublisher.Publish, line 39, in call_object * Module OFS.DTMLDocument, line 128, in __call__ DTMLDocument instance at 41725260 URL: http://www.arte-luce.com/catalog/html/catalog.html/manage_main Physical Path:/arteluce/catalog/html/catalog.html * Module DocumentTemplate.DT_String, line 474, in __call__ * Module DocumentTemplate.DT_Let, line 76, in render * Module OFS.DTMLDocument, line 121, in __call__ DTMLDocument instance at 4172b6b0 URL: http://www.arte-luce.com/catalog/html/zwarehouse_html_header/ manage_main Physical Path:/arteluce/catalog/html/zwarehouse_html_header * Module DocumentTemplate.DT_String, line 474, in __call__ * Module DocumentTemplate.DT_Util, line 201, in eval __traceback_info__: cart_functions * Module string, line 1, in expression * Module Shared.DC.Scripts.Bindings, line 306, in __call__ * Module Shared.DC.Scripts.Bindings, line 343, in _bindAndExec * Module Products.PythonScripts.PythonScript, line 318, in _exec * Module None, line 16, in setSessionByRequest.py PythonScript at /arteluce/catalog/cart_functions/setSessionByRequest.py Line 16 * Module ZPublisher.HTTPRequest, line 1218, in __getattr__ * Module ZPublisher.HTTPRequest, line 1178, in get * Module Products.Sessions.SessionDataManager, line 94, in getSessionData * Module Products.Sessions.SessionDataManager, line 181, in _getSessionDataObject * Module Products.Transience.Transience, line 847, in new_or_existing * Module Products.Transience.Transience, line 450, in __setitem__ KeyError: 1096365080 (Also, an error occurred while attempting to render the standard error message.) ___ Troubleshooting Suggestions * The URL may be incorrect. * The parameters passed to this resource may be incorrect. * A resource that this resource relies on may be encountering an error. For more detailed information about the error, please refer to error log. If the error persists please contact the site maintainer. Thank you for your patience. Hmmm.. that's an odd place to get that error. I'm surprised. Does it only happen after some period of inactivity on the site? - C FOLLOWING Chris's advice first I tried to copy also ZODB directory from Zope 2.7.3 but still the same error... so I Installed Zope-2.7.3b1-0 (IS THIS THE RIGHT VERSION THAT SHOULD SOLVE SESSION PROBLEMS?) and reinstalled with success the following packages: ZWarehouse 0.8 iHotfix-0.4.0 itools-0.4.6 Localizer-1.1.0a4 ZMySQLDA-2.0.8 MySQL-python-1.0.0 TranslationService-0.4 THEN imported my customer's web site but I still have the same problem... whats going on... what am i missing? For every change I made I
Re: [Zope-dev] session concurrency
I have never seen disappearing sessions -- but I must admit that we use our own session implementation - deriving from Zope's session but getting rid of the over-complex session management code. I've not explained very well. Session object doesn't disappears, just some data in session object does it. (they occur non-deterministically, they are extremely difficult to reproduce and analyse). As said: The advertised behaviour when several requests modify the same session object is that all but the first committing request get a ConflictError and are retried. Everything else is a bug. I'm agree that, if it works fine, the retrieing request method is a good solution. My problem is quasi deterministic, it crashes most of times :-( I will try to provide a example.zexp when entering the bug so it could be easier to reproduce. Another note: it is known (though we do not yet fully understand why) that sessions behave non-deterministically (disappear, revert to old values) when they are read/written in standard_error_message (Zope's error handling hook for failing requests). I'm not using SESSION in standard_error_message, but symptoms are very similar. Santi Camps ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] session concurrency
It is unlikely that the the low conflict connection causes your problem. Low conflict affects the following situation: You try to load an object (because it is not in your ZODB cache) and get informed that is has been modified (by another commited transaction) since your transaction started. With a normal ZODB connection, you will get a ReadConflictError as adding the object to your ZODB cache would make it inconsistent: some objects would have the state from your transactions start and others from a later date. With a low conflict connection, the inconsistency is accepted and no ReadConflictError is raised. As you described, you loose the session object. This is not easily understood as the effect of a cache inconsistency. Sure. Disabling Low conflict hasn't solved anything. I'm not able to find where is the problem. I have not enought knowledge of Zope internals. I will try to store my data in persisten subobjects, as you said. One thing is true: Changing session object simultaniously in various frames doesn't work fine. Sometimes session data is not the espected becouse another frame has erased your changes in the session object. I think this is a serious inconvenient in front of other platforms. From my point of view, it would be better to have SESSION out of the transaction but with changes stored inmediatly. You don't? Perhaps some day I will be able to implement this for myself. Until that, do you think I should enter a bug, or this cann't be considered as a bug? Thanks for your help Santi Camps ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] session concurrency
I'm not able to find where is the problem. I have not enought knowledge of Zope internals. I will try to store my data in persisten subobjects, as you said. Finally I've found a better and easy partial workarround. I will explain it here so other users in similar situation can know. It consist in to have multiple Transitient Object Containers and multiple Session Data Managers. For instance, in my case: Folder1/some_scripts Folder1/session_data (a Transitient Object Container) Folder1/session_data_manager (a Session Data Manager using previous object and referenced in REQUEST as SESSION_FOLDER1) Folder2/some_scripts Folder2/session_data (a Transitient Object Container) Folder2/session_data_manager (a Session Data Manager using previous object and referenced in REQUEST as SESSION_FOLDER2) Then, scripts in Folder1 can use SESSION_FOLDER1 to store its session data, scripts in Folder2 can use SESSION_FOLDER2 to do the same. To use SESSSION_FOLDER1 from scripts in Folder2, a simple traversal will work fine. Using this solution, my simultanious frames are working using session with no problems. Cheers Santi Camps http://zetadb.sourceforge.net ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] session concurrency
Santi Camps wrote at 2003-11-27 17:41 +0100: ... One thing is true: Changing session object simultaniously in various frames doesn't work fine. Sometimes session data is not the espected becouse another frame has erased your changes in the session object. I think this is a serious inconvenient in front of other platforms. This is a general problem of concurrently updating the same object. Zope's Session mechanism is not that bad in this respect (it is in some way better than that for other plattforms). When it works as advertised (I see it working this way; you do not), then concurrent modification will result in a ConflictError with automatic request retrial. This provides for consistency despite concurrent updates. A *BIG* problem is the large granularity of a Zope session. It is essentially one big monolitic object (with respect to concurrent update). This makes ConflictError much more likely. I have never seen disappearing sessions -- but I must admit that we use our own session implementation - deriving from Zope's session but getting rid of the over-complex session management code. This code caused many inconsistency errors for the session management data structures; which is a bad sign. The complexity comes from the fact that it tries to scale well for large numbers of sessions. Our simplified code scales not that well but can easily cope with several thousand (locally maintained) sessions we sometimes see. From my point of view, it would be better to have SESSION out of the transaction but with changes stored inmediatly. You don't? No: you would face the general inconsistency problem caused by concurrent updates of an object. Analysing and fixing such problems is a nightmare (they occur non-deterministically, they are extremely difficult to reproduce and analyse). Perhaps some day I will be able to implement this for myself. No need. There is a product called ZSession. It maintains session data in RAM and (I think) allows threads to read/write sessions asynchronously. It is probably no longer maintained, now that Zope comes with Sessions in the core distribution. But that need not be a big problem. Should it break in some future Zope version (which may be possible), the fixes should be quite easy and straight forward. Until that, do you think I should enter a bug, or this cann't be considered as a bug? When you see session objects disappearing (and they have not expired), then this is a bug! As said: The advertised behaviour when several requests modify the same session object is that all but the first committing request get a ConflictError and are retried. Everything else is a bug. Another note: it is known (though we do not yet fully understand why) that sessions behave non-deterministically (disappear, revert to old values) when they are read/written in standard_error_message (Zope's error handling hook for failing requests). standard_error_message is called after the transaction has been aborted -- thus any (transactional) effect from the request has already been wiped out. Accessing Session there may read old values or create a new (empty) session. Apparently, and this is what we do not yet understand, such phony sessions seem to be able to spill over into later requests. One hypothesis was that it were caused by Zope forgetting to abort/commit the transaction implicitly created for standard_error_message execution (this definitely is a bug!). But the transaction *is* aborted later, before it is being used in another request -- maybe this happens then in the wrong context? I will not see problems caused by this failing to commit/abort error transaction bug because I fixed our Zope as soon as the bug became known. I am not sure whether the bug is fixed in Zope 2.6.2. That's a disadvantage of maintaining our own Zope version - I do not know and can not easily check the distributed code any longer. -- Dieter ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] session concurrency
Santi Camps wrote at 2003-11-24 23:14 +0100: I've a product, http://zetadb.sourceforge.net, that uses SESSION extensively. I've just discover, unhappiness, that there are some problems when SESSION object is changed at the same time in two diferent python scripts (using frames). The two frames are changing diferent keys of the SESSION object, but at the end the whole SESSION object is saved, I think, so one of the two objects losses its SESSION data It should *NOT* loose its session data but get a ConflictError and automatically retry the request. That's normal ZODB behaviour. Search for Application specific conflict resolution for a (complex, in your case) way to work around this. A partial workaround may be to put your various keys into persistent subobjects. I'm trying to understand how Conflict Resolution works. The first step will be change my DirectoryStorage for a FileStorage and try again. As you say, I seems that Zope retries the request 3 times when a ConflictError is raised, so it should work in my case. If not, I will try to debug and see how not, or perhaps disabling low conflict flag. If, at the end, it countinues crashing, I will store my data in persistent subobjects. Thanks a lot Santi Camps ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] session concurrency
[EMAIL PROTECTED] wrote at 2003-11-26 11:54 +0100: ... I'm trying to understand how Conflict Resolution works. The first step will be change my DirectoryStorage for a FileStorage and try again. This will have no influence to your session problems. Sessions are (by default) put into a RAM based storage independent from your main storage (DirectoryStorage or FileStorage). As you say, I seems that Zope retries the request 3 times when a ConflictError is raised, so it should work in my case. Indeed. If not, I will try to debug and see how not, or perhaps disabling low conflict flag. It is unlikely that the the low conflict connection causes your problem. Low conflict affects the following situation: You try to load an object (because it is not in your ZODB cache) and get informed that is has been modified (by another commited transaction) since your transaction started. With a normal ZODB connection, you will get a ReadConflictError as adding the object to your ZODB cache would make it inconsistent: some objects would have the state from your transactions start and others from a later date. With a low conflict connection, the inconsistency is accepted and no ReadConflictError is raised. As you described, you loose the session object. This is not easily understood as the effect of a cache inconsistency. -- Dieter ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] session concurrency
Hi Santi, I've a product, http://zetadb.sourceforge.net, that uses SESSION extensively. I've just discover, unhappiness, that there are some problems when SESSION object is changed at the same time in two diferent python scripts (using frames). The two frames are changing diferent keys of the SESSION object, but at the end the whole SESSION object is saved, I think, so one of the two objects losses its SESSION data Anybody knows any workarround about this? I guess this happens because the SESSION is stored with a low conflict resolution in the TempFolder. You should be able to switch the conflict resolution in the temp folder to something more strict, which may increase the number conflict errors a lot. There have been a thread which may tell You how to do this on this list in March(?) of this year; seaching the mailing list archive for LowConflictConnection or something similar may help. Cheers, Clemens ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] session concurrency
Santi Camps wrote at 2003-11-24 23:14 +0100: I've a product, http://zetadb.sourceforge.net, that uses SESSION extensively. I've just discover, unhappiness, that there are some problems when SESSION object is changed at the same time in two diferent python scripts (using frames). The two frames are changing diferent keys of the SESSION object, but at the end the whole SESSION object is saved, I think, so one of the two objects losses its SESSION data It should *NOT* loose its session data but get a ConflictError and automatically retry the request. That's normal ZODB behaviour. Search for Application specific conflict resolution for a (complex, in your case) way to work around this. A partial workaround may be to put your various keys into persistent subobjects. -- Dieter ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] session concurrency
Hi all, I've a product, http://zetadb.sourceforge.net, that uses SESSION extensively. I've just discover, unhappiness, that there are some problems when SESSION object is changed at the same time in two diferent python scripts (using frames). The two frames are changing diferent keys of the SESSION object, but at the end the whole SESSION object is saved, I think, so one of the two objects losses its SESSION data Anybody knows any workarround about this? Thanks in advance Santi Camps ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] SESSION loosing data when system has multiple users
We have been using Zope 2.7.0-beta2 and we have been having a bit of trouble with SESSION loosing data when we have more than one user on the system. These problems usually show up as KeyErrors when a particular data element in SESSION cannot be found. If I set the zserver threads to 1 in zope.conf, the problem seems to go away (but the system isn't really usable either). So I assumed it was a thread-safety problem in the SESSION code. I decided to be brave and take a look at the code. To my surprize, I found a test file called stresstestMultiThread.py in the lib/python/Products/Sessions/stresstests directory that seemed to be the exact thing I wanted to test. It wouldn't run without a couple of minor modifications, but once I got it running, with more than three or four threads, the tests fail (never complete). Is this test file still valid? It looks like it hasn't been modified for almost a year now. If it is valid, has anyone looked into getting the tests to pass. I will probably look into it more myself, but before I do, I just want to make sure I'm not headed down a dead-end street. Thanks, Steve P.S. - The tests failed in both the 2.7.0b2-src and the what is currently in cvs. ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] SESSION loosing data when system has multiple users
Chris, Sorry, I should have included more info, but my message was already a bit long. I have run the tests on both Red Hat 7.3 and Red Hat 9 with the same result. I am using python 2.2.3. Here is the step-by-step for what I did (I'm going to err on the side of too much info here): - check out latest Zope from cvs - cd Zope - ./configure - make - make instance - cd lib/python/Products/Sessions/stresstests - edit stresstestMultiThread.py - comment out line 38 - changed readiters and writeiters fro 100 to 10 on lines 117 and 118 - cp ~/Zope/lib/python/Products/Transience/tests/fauxtime.py . - python2 stresstestMultiThread.py This time, on Red Hat 7.3, I actually got an exception. I'll include the output below. Please let me know if there is ANYTHING I can do to help out with this one. I appreciate you looking into it. Steve [EMAIL PROTECTED] stresstests]$ python2 stresstestMultiThread.py testNonOverlappingBrowserIds (__main__.TestMultiThread) ... 4 values in toc waiting for 8 threads read conflict 4 values in toc read conflict read conflict read conflict waiting for 8 threads read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict 4 values in toc 4 values in toc 4 values in toc 4 values in toc 4 values in toc 4 values in toc 4 values in toc 4 values in toc read conflict read conflict read conflict read conflict read conflict read conflict waiting for 8 threads read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict waiting for 8 threads read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict 0 read conflict read conflict read conflict read conflict read conflict waiting for 7 threads read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict 2003-10-17T14:41:56 25 Exception in thread Thread-3: Traceback (most recent call last): File //usr/lib/python2.2/threading.py, line 414, in __bootstrap self.run() File stresstestMultiThread.py, line 164, in run self.run1() File stresstestMultiThread.py, line 187, in run1 data = session_data_manager.getSessionData(create=1) File /home/stevej/Zope/lib/python/Products/Sessions/SessionDataManager.py, line 93, in getSessionData return self._getSessionDataObject(key) File /home/stevej/Zope/lib/python/Products/Sessions/SessionDataManager.py, line 180, in _getSessionDataObject ob = container.new_or_existing(key) File /home/stevej/Zope/lib/python/Products/Transience/Transience.py, line 175, in new_or_existing item = self.get(key, notfound) File /home/stevej/Zope/lib/python/Products/Transience/Transience.py, line 807, in get v = self._data[b].get(k, notfound) KeyError: 0 read conflict read conflict read conflict 2003-10-17T14:41:56 17 Exception in thread Thread-2: Traceback (most recent call last): File //usr/lib/python2.2/threading.py, line 414, in __bootstrap self.run() File stresstestMultiThread.py, line 164, in run self.run1() File stresstestMultiThread.py, line 187, in run1 data = session_data_manager.getSessionData(create=1) File /home/stevej/Zope/lib/python/Products/Sessions/SessionDataManager.py, line 93, in getSessionData return self._getSessionDataObject(key) File /home/stevej/Zope/lib/python/Products/Sessions/SessionDataManager.py, line 180, in _getSessionDataObject ob = container.new_or_existing(key) File /home/stevej/Zope/lib/python/Products/Transience/Transience.py, line 175, in new_or_existing item = self.get(key, notfound) File /home/stevej/Zope/lib/python/Products/Transience/Transience.py, line 807, in get v = self._data[b].get(k, notfound) File /home/stevej/Zope/lib/python/ZODB/Connection.py, line 561, in setstate p, serial = self._storage.load(oid, self._version) File /home/stevej/Zope/lib/python/tempstorage/TemporaryStorage.py, line 94, in load s=self._index[oid] KeyError: 2003-10-17T14:41:56 18 Exception in thread Thread-4: Traceback (most recent call last): File //usr/lib/python2.2/threading.py, line 414, in __bootstrap self.run() File stresstestMultiThread.py, line
Re: [Zope-dev] SESSION loosing data when system has multiple users
I will take a look. Thanks! - C On Fri, 2003-10-17 at 17:04, Steve Jibson wrote: Chris, Sorry, I should have included more info, but my message was already a bit long. I have run the tests on both Red Hat 7.3 and Red Hat 9 with the same result. I am using python 2.2.3. Here is the step-by-step for what I did (I'm going to err on the side of too much info here): - check out latest Zope from cvs - cd Zope - ./configure - make - make instance - cd lib/python/Products/Sessions/stresstests - edit stresstestMultiThread.py - comment out line 38 - changed readiters and writeiters fro 100 to 10 on lines 117 and 118 - cp ~/Zope/lib/python/Products/Transience/tests/fauxtime.py . - python2 stresstestMultiThread.py This time, on Red Hat 7.3, I actually got an exception. I'll include the output below. Please let me know if there is ANYTHING I can do to help out with this one. I appreciate you looking into it. Steve [EMAIL PROTECTED] stresstests]$ python2 stresstestMultiThread.py testNonOverlappingBrowserIds (__main__.TestMultiThread) ... 4 values in toc waiting for 8 threads read conflict 4 values in toc read conflict read conflict read conflict waiting for 8 threads read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict 4 values in toc 4 values in toc 4 values in toc 4 values in toc 4 values in toc 4 values in toc 4 values in toc 4 values in toc read conflict read conflict read conflict read conflict read conflict read conflict waiting for 8 threads read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict waiting for 8 threads read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict 0 read conflict read conflict read conflict read conflict read conflict waiting for 7 threads read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict 2003-10-17T14:41:56 25 Exception in thread Thread-3: Traceback (most recent call last): File //usr/lib/python2.2/threading.py, line 414, in __bootstrap self.run() File stresstestMultiThread.py, line 164, in run self.run1() File stresstestMultiThread.py, line 187, in run1 data = session_data_manager.getSessionData(create=1) File /home/stevej/Zope/lib/python/Products/Sessions/SessionDataManager.py, line 93, in getSessionData return self._getSessionDataObject(key) File /home/stevej/Zope/lib/python/Products/Sessions/SessionDataManager.py, line 180, in _getSessionDataObject ob = container.new_or_existing(key) File /home/stevej/Zope/lib/python/Products/Transience/Transience.py, line 175, in new_or_existing item = self.get(key, notfound) File /home/stevej/Zope/lib/python/Products/Transience/Transience.py, line 807, in get v = self._data[b].get(k, notfound) KeyError: 0 read conflict read conflict read conflict 2003-10-17T14:41:56 17 Exception in thread Thread-2: Traceback (most recent call last): File //usr/lib/python2.2/threading.py, line 414, in __bootstrap self.run() File stresstestMultiThread.py, line 164, in run self.run1() File stresstestMultiThread.py, line 187, in run1 data = session_data_manager.getSessionData(create=1) File /home/stevej/Zope/lib/python/Products/Sessions/SessionDataManager.py, line 93, in getSessionData return self._getSessionDataObject(key) File /home/stevej/Zope/lib/python/Products/Sessions/SessionDataManager.py, line 180, in _getSessionDataObject ob = container.new_or_existing(key) File /home/stevej/Zope/lib/python/Products/Transience/Transience.py, line 175, in new_or_existing item = self.get(key, notfound) File /home/stevej/Zope/lib/python/Products/Transience/Transience.py, line 807, in get v = self._data[b].get(k, notfound) File /home/stevej/Zope/lib/python/ZODB/Connection.py, line 561, in setstate p, serial = self._storage.load(oid, self._version)
Re: [Zope-dev] Session Errors (read conflicts)
On Mon, 2003-03-17 at 20:34, John Eikenberry wrote: John Eikenberry wrote: Toby Dickenson wrote: Read conflicts occur if a change is committed in between the start of a transaction, and the transaction needing to load the object. A workaround to reduce the number of read conflicts is to touch the objects that are likely to change early in the transaction. Thanks for the tip. My sessions test script had a sleep() call right after the transaction commit, thus making it right at the beginning of the next transaction. Moving it to right before the commit (after all the session tests) pretty much eliminated all the read conflict errors. Just get a standard ConflictError occasionally now (not nearly as often). This turned out to be better than I thought. After nearly 3 hours of constant abuse I have yet to see a ReadConflictError and have yet to get either the load() or get() KeyErrors, previously I could force one of these in a 2-3 minutes of abuse. Our live sessions code uses the sessions about half to two-thirds of the way through the transaction. Given what can happen in that first half, there is easily plenty of time for read conflicts. I think I might be able to move our session use to the beginning of the transaction (just storing stuff in the REQUEST object for later usage) and hopefully fix our issue. Interesting, and I'm very glad you got it working reliably. But I'm not sure I understand why touching an object at the start of a transaction would fix the consistency problem. If a object A is read at the beginning of a transaction that uses the data from object A during write of object B and later object A changes and is invalidated by another transaction (before the first finishes), no conflict errors will be raised. Isn't there the potential for the first transaction to write inconsistent data into object B? In my mind, this is the same thing as using a low conflict connection (ie. you are essentially turning off read conflicts, albeit by circumstance rather than by code). I guess the question boils down to: why *don't* you see consistency problems now? I would expect to continue to see the get errors you reported earlier. I think this exact problem is being discussed on ZODB-Dev under the title Database Read Conflicts -- neither safe nor robust. The KeyErrors happen under similar circumstances to the ReadConflictErrors. The significant difference being that the KeyErrors happen after the transience timeout has occured. When I am running with the LowConflictConnection disabled the KeyErrors occur in the Connection.setstate method at line 509, before the ReadConflictError check. Is this the same KeyError you reported as coming out of TemporaryStorage.load? So say you have 2 threads; A and B. If A starts first, hits the session code and triggers a _housekeep() call during which time B has started but has not reached the Sessions code until after _housekeep() has finished. When it does reach the sessions code, you get the KeyErrors. Would you mind restating that? I think this is important, and I'm not sure I understand the second sentence above. Thanks! - C ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Session Errors
On Mon, 2003-03-17 at 15:02, Dieter Maurer wrote: You know that we use a Transience implementation with minimal (and not very essential) administrative data which is immune to inconsistencies and can be used across several ZEO clients (although we do not use that). We have good experience with this implementation. Does this package implement a similar API as the existing Transience package? Would you be willing to send it to me? The problem might also be the MountPoint -- with the connection to the TemporaryStorage not correctly synchronized with the transaction. Good point, thanks. - C ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Session Errors (read conflicts)
On Tuesday 18 March 2003 3:24 pm, Chris McDonough wrote: Our live sessions code uses the sessions about half to two-thirds of the way through the transaction. Given what can happen in that first half, there is easily plenty of time for read conflicts. I think I might be able to move our session use to the beginning of the transaction (just storing stuff in the REQUEST object for later usage) and hopefully fix our issue. Depending on what your sessioning involves, you might not need to change when you do all the work. Possibly just touching (an attribute of) the persistent session object will be enough. But I'm not sure I understand why touching an object at the start of a transaction would fix the consistency problem. If a object A is read at the beginning of a transaction that uses the data from object A during write of object B and later object A changes and is invalidated by another transaction (before the first finishes), no conflict errors will be raised. Isn't there the potential for the first transaction to write inconsistent data into object B? Today you are guaranteed that your transaction reads a consistent initial state of A and B. That is, there is no chance that you only see half the changes of a recent transaction that modified both. Yes, it is possible for one transaction to modify A and a concurrent transaction to modify B. This is what Deiter describes as not safe in that zodb-dev thread. IMO, it is only not safe if you are not aware of it. And it is critical for performance with concurrency - consider these two transactions occuring on different zeo nodes. In my mind, this is the same thing as using a low conflict connection (ie. you are essentially turning off read conflicts, albeit by circumstance rather than by code). There is a significant difference. The low conflict connection permits the possibility of seeing half the changes of a recent transaction that modified both objects. low conflict connection is a misleadnig name. The conflicts are still there - just not reported as exceptions. low consistency connection would be more accurate ;-) ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Session Errors (read conflicts)
Today you are guaranteed that your transaction reads a consistent initial state of A and B. That is, there is no chance that you only see half the changes of a recent transaction that modified both. Yes, it is possible for one transaction to modify A and a concurrent transaction to modify B. This is what Deiter describes as not safe in that zodb-dev thread. IMO, it is only not safe if you are not aware of it. And it is critical for performance with concurrency - consider these two transactions occuring on different zeo nodes. Right, I understand now, thank you. In my mind, this is the same thing as using a low conflict connection (ie. you are essentially turning off read conflicts, albeit by circumstance rather than by code). There is a significant difference. The low conflict connection permits the possibility of seeing half the changes of a recent transaction that modified both objects. low conflict connection is a misleadnig name. The conflicts are still there - just not reported as exceptions. low consistency connection would be more accurate ;-) Yup. Maybe I should rename it? ;-) So... I'm wondering what the hell I should do. ;-) Obviously I need to deactivate the low consistency connection in the default setup. I'm thinking that I also may just need to move the housekeeping duties to a separate scheduled thread that only happens when the system is not busy (e.g. when the asyncore poll select timeout is reached maybe) in order to reduce the potential for conflicts. But I'm not sure what else to do, and I still don't understand what is causing the KeyErrors in the storage code. Sigh. - C ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Session Errors (read conflicts)
On Tuesday 18 March 2003 6:12 pm, Chris McDonough wrote: I'm thinking that I also may just need to move the housekeeping duties to a separate scheduled thread that only happens when the system is not busy (e.g. when the asyncore poll select timeout is reached maybe) in order to reduce the potential for conflicts. Thats hard to define, when zeo is installed. But I'm not sure what else to do, and I still don't understand what is causing the KeyErrors in the storage code. Sigh. Ive never looked at nor used your Sessioning support, but I am interested in things that can cause (POS)KeyErrors in Storages. This might not be a storage bug. There are several known corner cases in zodb that mean semi-legitimate applications can cause KeyErrors. DirectoryStorage tries to guard against them, and will raise a DanglingReferenceError rather than commit a transaction that will cause a POSKeyError when read. Can the problem be reproduced using DirectoryStorage? (without the low consistency connection) -- Toby Dickenson http://www.geminidataloggers.com/people/tdickenson ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Session Errors (read conflicts)
On Tue, 2003-03-18 at 14:01, Toby Dickenson wrote: On Tuesday 18 March 2003 6:12 pm, Chris McDonough wrote: I'm thinking that I also may just need to move the housekeeping duties to a separate scheduled thread that only happens when the system is not busy (e.g. when the asyncore poll select timeout is reached maybe) in order to reduce the potential for conflicts. Thats hard to define, when zeo is installed. It would only make sense for shared session data if the housekeeping thread was enabled on a single ZEO client in a cluster. Of course this scenario has its own configuration and management difficulties. But I'm not sure what else to do, and I still don't understand what is causing the KeyErrors in the storage code. Sigh. Ive never looked at nor used your Sessioning support, but I am interested in things that can cause (POS)KeyErrors in Storages. This might not be a storage bug. There are several known corner cases in zodb that mean semi-legitimate applications can cause KeyErrors. DirectoryStorage tries to guard against them, and will raise a DanglingReferenceError rather than commit a transaction that will cause a POSKeyError when read. Can the problem be reproduced using DirectoryStorage? (without the low consistency connection) That's a fine question. ;-) John, did you see these Key Errors happen under a storage other than TemporaryStorage (e.g. if the session data container is in the main database)? I will check myself as well when possible. Dieter also observed that this may be a behavior specific to mounted storages, so I will need to try that as well. - C ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Session Errors (read conflicts)
Chris McDonough wrote: On Mon, 2003-03-17 at 20:34, John Eikenberry wrote: The KeyErrors happen under similar circumstances to the ReadConflictErrors. The significant difference being that the KeyErrors happen after the transience timeout has occured. When I am running with the LowConflictConnection disabled the KeyErrors occur in the Connection.setstate method at line 509, before the ReadConflictError check. Is this the same KeyError you reported as coming out of TemporaryStorage.load? Yes. I'm referring to the traceback I reported in a previous mail: http://mail.zope.org/pipermail/zope-dev/2003-March/019118.html The error occurs on the same line in TemporaryStorage.load(), whether it gets called from ZODB/Connection.py or TemporaryFolder/LowConflictConnection.py. So say you have 2 threads; A and B. If A starts first, hits the session code and triggers a _housekeep() call during which time B has started but has not reached the Sessions code until after _housekeep() has finished. When it does reach the sessions code, you get the KeyErrors. Would you mind restating that? I think this is important, and I'm not sure I understand the second sentence above. I'm working off the idea that the load() KeyErrors went away when the ReadConflictErrors did. So it seemed like they were probably triggered by similar scenarios. The difference being that for the former, the timeout had been reached and the _housekeep() code had been run (and possibly some additional code in _getCurrentBucket). Now given Toby's description of how ReadConflicts occur [1], it seems that instead of a change being committed to the object it is instead deleted (in _housekeep). Thus instead of getting the object and seeing it is marked as invalid, it cannot get the object at all when it expects to be able to... resulting in the KeyError in load(). [1] Read conflicts occur if a change is committed in between the start of a transaction, and the transaction needing to load the object. A workaround to reduce the number of read conflicts is to touch the objects that are likely to change early in the transaction. -- John Eikenberry [EMAIL PROTECTED] __ A society that will trade a little liberty for a little order will deserve neither and lose both. --B. Franklin ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Session Errors
On Saturday 15 March 2003 5:40 pm, Chris McDonough wrote: - The second is an unidentified (yet) bug in TemporaryStorage. As a cause of the KeyError exception? Could be I have seen equivalent exceptions with plain FileStorage in applications that do not use sessions, so I suspect there might be a hard-to-hit zodb bug that can cause this too. (In FileStorage the equivalent exception is POSKeyError, a subclass of KeyError) If I find and fix the TemporaryStorage bug I will let you know. In the meantime, you can probably work around this by using a different non-undoing storage to put your session data in (e.g. Berkeley, or maybe DirectoryStorage?). It would be interesting to hear if this exception is repeatable with the session state stored in a FileStorage too. You may see many more conflicts with this running. But maybe the data structures will not become desynchronized. You weren't kidding about the increase in conflict errors. Read conflicts occur if a change is committed in between the start of a transaction, and the transaction needing to load the object. A workaround to reduce the number of read conflicts is to touch the objects that are likely to change early in the transaction. -- Toby Dickenson http://www.geminidataloggers.com/people/tdickenson ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Session Errors
Hi Chris, Chris McDonough wrote at 2003-3-15 12:40 -0500: ... - The first is that turning off read conflicts allows for the potential for the session data bucket and the session data index to become out of sync. I admit that I knew this already, but haven't figured out a way around it yet without having massive numbers of conflicts. That said, it may be that we can't avoid them and I may turn this behavior off in the next revision. Another alternative is to use a nonpersistent index, but that has other challenges ( it could also prevent the shared use of session data between ZEO clients). You know that we use a Transience implementation with minimal (and not very essential) administrative data which is immune to inconsistencies and can be used across several ZEO clients (although we do not use that). We have good experience with this implementation. - The second is an unidentified (yet) bug in TemporaryStorage. Most of the code for TemporaryStorage came right out of the Berkeley Packless storage. I should probably try to replicate this under load (this should not be difficult).To verify the assertion that there really is a bug in TemporaryStorage without much trouble, you could temporarily put your transient object container (session_data) in the main database and point the session data manager at it and run for a while under this config. This is not recommended for production as it will cause main database bloat, however. The problem might also be the MountPoint -- with the connection to the TemporaryStorage not correctly synchronized with the transaction. Dieter ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Session Errors
Toby Dickenson wrote: Read conflicts occur if a change is committed in between the start of a transaction, and the transaction needing to load the object. A workaround to reduce the number of read conflicts is to touch the objects that are likely to change early in the transaction. Thanks for the tip. My sessions test script had a sleep() call right after the transaction commit, thus making it right at the beginning of the next transaction. Moving it to right before the commit (after all the session tests) pretty much eliminated all the read conflict errors. Just get a standard ConflictError occasionally now (not nearly as often). -- John Eikenberry [EMAIL PROTECTED] __ A society that will trade a little liberty for a little order will deserve neither and lose both. --B. Franklin ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Session Errors
John Eikenberry wrote: Toby Dickenson wrote: Read conflicts occur if a change is committed in between the start of a transaction, and the transaction needing to load the object. A workaround to reduce the number of read conflicts is to touch the objects that are likely to change early in the transaction. Thanks for the tip. My sessions test script had a sleep() call right after the transaction commit, thus making it right at the beginning of the next transaction. Moving it to right before the commit (after all the session tests) pretty much eliminated all the read conflict errors. Just get a standard ConflictError occasionally now (not nearly as often). This turned out to be better than I thought. After nearly 3 hours of constant abuse I have yet to see a ReadConflictError and have yet to get either the load() or get() KeyErrors, previously I could force one of these in a 2-3 minutes of abuse. Our live sessions code uses the sessions about half to two-thirds of the way through the transaction. Given what can happen in that first half, there is easily plenty of time for read conflicts. I think I might be able to move our session use to the beginning of the transaction (just storing stuff in the REQUEST object for later usage) and hopefully fix our issue. The KeyErrors happen under similar circumstances to the ReadConflictErrors. The significant difference being that the KeyErrors happen after the transience timeout has occured. When I am running with the LowConflictConnection disabled the KeyErrors occur in the Connection.setstate method at line 509, before the ReadConflictError check. So say you have 2 threads; A and B. If A starts first, hits the session code and triggers a _housekeep() call during which time B has started but has not reached the Sessions code until after _housekeep() has finished. When it does reach the sessions code, you get the KeyErrors. Sound reasonable? -- John Eikenberry [EMAIL PROTECTED] __ A society that will trade a little liberty for a little order will deserve neither and lose both. --B. Franklin ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Session Errors
Thanks so much for helping to pin this down. Thanks for the observation that the error occurs more frequently when the timeout is small. This is likely because the timeout value directly impacts the frequency of housekeeping operations, where old buckets are garbage collected and new buckets are added. The housekeeping operations must confuse the underlying storage. I don't know how this could happen, but it does. I suspect it's related somehow to conflict errors. So we seem to have two bugs: - The first is that turning off read conflicts allows for the potential for the session data bucket and the session data index to become out of sync. I admit that I knew this already, but haven't figured out a way around it yet without having massive numbers of conflicts. That said, it may be that we can't avoid them and I may turn this behavior off in the next revision. Another alternative is to use a nonpersistent index, but that has other challenges ( it could also prevent the shared use of session data between ZEO clients). - The second is an unidentified (yet) bug in TemporaryStorage. Most of the code for TemporaryStorage came right out of the Berkeley Packless storage. I should probably try to replicate this under load (this should not be difficult).To verify the assertion that there really is a bug in TemporaryStorage without much trouble, you could temporarily put your transient object container (session_data) in the main database and point the session data manager at it and run for a while under this config. This is not recommended for production as it will cause main database bloat, however. If I find and fix the TemporaryStorage bug I will let you know. In the meantime, you can probably work around this by using a different non-undoing storage to put your session data in (e.g. Berkeley, or maybe DirectoryStorage?). - C On Fri, 2003-03-14 at 20:28, John Eikenberry wrote: Sorry for the length of this one... but I'm trying to braindump to give you as much info about the problem as possible. To be sure it doesn't get lost in my below ramblings, there is probably important peice of information I haven't mentioned yet... that is that these errors seem to coincide with the session data timeout setting [1]. I don't get the errors at all until the timeout is reached or has passed. [1] The timeout setting I'm refering to is denoted by the label: Data object timeout value in minutes on the /temp_folder/session_data object. Chris McDonough wrote: OK, thanks John. Let's try one more thing... currently the mounted database used to store the session data uses a connection that ignores read conflicts. This is known to be bad because the machinery which deals with keeping the sessioning index data will also ignore read conflicts, which may create inconcstencies between two data structures (BTrees) that need to be kept in sync. I tried this and it seemed to help some. I haven't seen the get() error we've been dicussing yet, but a the load() error just occurred (line 94 in TemporaryStorage - this was error #1 in my original email). Though the traceback is a bit different from my original email, as the LowConflictConnection isn't being used. Here's the new Traceback: Error Type: KeyError Error Value: [non-ascii chars] Traceback (innermost last): * Module ZPublisher.Publish, line 98, in publish * Module ZPublisher.mapply, line 88, in mapply * Module ZPublisher.Publish, line 39, in call_object * Module Products.DotOrg.Pages.KPage, line 110, in testSession * Module Products.DotOrg.Utils.Spawn, line 42, in launchProcess * Module Products.DotOrg.Utils.Spawn, line 73, in storeArgs * Module Products.Sessions.SessionDataManager, line 180, in * _getSessionDataObject * Module Products.Transience.Transience, line 175, in new_or_existing * Module Products.Transience.Transience, line 797, in get * Module Products.Transience.Transience, line 546, in _getCurrentBucket * Module ZODB.Connection, line 509, in setstate * Module Products.TemporaryFolder.TemporaryStorage, line 94, in load Here's a patch to lib/python/Products/TemporaryFolder/TemporaryFolder.py that reenables read conflict generation on the database. Index: TemporaryFolder.py === RCS file: /cvs-repository/Zope/lib/python/Products/TemporaryFolder/TemporaryFolder.py,v retrieving revision 1.7 diff -r1.7 TemporaryFolder.py 72c72 db.klass = LowConflictConnection --- #db.klass = LowConflictConnection You may see many more conflicts with this running. But maybe the data structures will not become desynchronized. You weren't kidding about the increase in conflict errors. Another problem, still unexplained, experienced by Andrew Athan, is that if a reference is made to a session data object
Re: [Zope-dev] Session Errors
Chris McDonough wrote: OK, thanks John. Thank you for helping. I hate to ask this (I should have done this to start with), but would you be willing to use the following patch --against the original file, not your recently patched version-- and try again? I only checked one of the two BTrees that might be at the heart of the problem with the first patch, this patch checks the second as well. Put the patch in place and have a couple errors already. Doesn't look like its going to be much help though. Just a couple repetitions of this: -- 2003-03-14T03:35:53 PROBLEM(100) Transience KeyError raised in get, checking BTrees -- 2003-03-14T03:35:53 PROBLEM(100) Transience BTree check for data succeeded -- 2003-03-14T03:35:53 PROBLEM(100) Transience BTree check for index succeeded - C On Thu, 2003-03-13 at 18:18, John Eikenberry wrote: Patch applied and the first results are in... so far its a lot of these: 2003-03-13T15:18:07 PROBLEM(100) Transience KeyError raised in get, checking _data BTree -- 2003-03-13T15:18:07 PROBLEM(100) Transience BTree check succeeded Chris McDonough wrote: Hi John, Can you apply the attached diff to your Transience.py file and run with it in place for a couple of days? It will not fix the problem (the symptoms will remain) but it should print some diagnostic information to the Zope event log (the STUPID_LOG_FILE, hopefully you've got that going) that will help us track down what this might be. Once you notice it happen, send the relevant parts of your logfile to me and I will see if I can analyze it. - C On Thu, 2003-03-13 at 15:19, John Eikenberry wrote: Sorry, its Zope 2.6.1. Chris McDonough wrote: John, Which Zope 2.6? Zope 2.6.1? Here's what line 807 of the current Transience.py looks like: v = self._data[b].get(k, notfound) Does yours look like that? Yes. What is the value of the __version__ variable at the top of the Transience.py file? __version__='$Revision: 1.28.6.4 $'[11:-2] On Thu, 2003-03-13 at 07:11, John Eikenberry wrote: Since upgrading to Zope-2.6 we've been getting KeyErrors when using Sessions. They seem to happen more now that we've started using hasSessionData(), but I'm pretty sure they happened prior to that. Anyways, here are the 2 related tracebacks. Has anyone else seen these? Traceback #1 occurs most frequently. The KeyError's value is an unprintable string of non-ascii characters. * Module ZPublisher.Publish, line 150, in publish_module * Module ZPublisher.Publish, line 114, in publish * Module The application server.App.startup, line 182, in zpublisher_exception_hook * Module ZPublisher.Publish, line 98, in publish * Module ZPublisher.mapply, line 88, in mapply * Module ZPublisher.Publish, line 39, in call_object * Module App.special_dtml, line 61, in __call__ * Module DocumentTemplate.DT_String, line 474, in __call__ * Module Products.Transience.Transience, line 342, in nudge * Module Products.Transience.Transience, line 467, in _getCurrentBucket * Module Products.TemporaryFolder.LowConflictConnection, line 34, in setstate * Module Products.TemporaryFolder.TemporaryStorage, line 94, in load KeyError: Traceback #2 happens less frequently, though today it seemed like it was trying to catch up (3 of these today). * Module ZPublisher.Publish, line 98, in publish * Module ZPublisher.mapply, line 88, in mapply * Module ZPublisher.Publish, line 39, in call_object * Module OFS.DTMLMethod, line 126, in __call__ * Module DocumentTemplate.DT_String, line 474, in __call__ * Module Products.DotOrg.Pages.KContent, line 149, in __call__ * Module Products.DotOrg.Pages.KContent, line 194, in getEditInfo * Module Products.DotOrg.Pages.KContent, line 506, in hasSessionData * Module Products.Sessions.SessionDataManager, line 101, in hasSessionData * Module Products.Sessions.SessionDataManager, line 175, in _hasSessionDataObject * Module Products.Transience.Transience, line 838, in has_key * Module Products.Transience.Transience, line 807, in get KeyError: 1047409860 -- John Eikenberry [EMAIL PROTECTED] __ A society that will trade a little liberty for a little order will deserve neither and lose both. --B. Franklin ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or
Re: [Zope-dev] Session Errors
OK, thanks John. Let's try one more thing... currently the mounted database used to store the session data uses a connection that ignores read conflicts. This is known to be bad because the machinery which deals with keeping the sessioning index data will also ignore read conflicts, which may create inconcstencies between two data structures (BTrees) that need to be kept in sync. Here's a patch to lib/python/Products/TemporaryFolder/TemporaryFolder.py that reenables read conflict generation on the database. Index: TemporaryFolder.py === RCS file: /cvs-repository/Zope/lib/python/Products/TemporaryFolder/TemporaryFolder.py,v retrieving revision 1.7 diff -r1.7 TemporaryFolder.py 72c72 db.klass = LowConflictConnection --- #db.klass = LowConflictConnection You may see many more conflicts with this running. But maybe the data structures will not become desynchronized. Another problem, still unexplained, experienced by Andrew Athan, is that if a reference is made to a session data object from within the standard error message, somehow things get screwy under high load. If you're doing the same, please let me know. - C On Fri, 2003-03-14 at 06:48, John Eikenberry wrote: Chris McDonough wrote: OK, thanks John. Thank you for helping. I hate to ask this (I should have done this to start with), but would you be willing to use the following patch --against the original file, not your recently patched version-- and try again? I only checked one of the two BTrees that might be at the heart of the problem with the first patch, this patch checks the second as well. Put the patch in place and have a couple errors already. Doesn't look like its going to be much help though. Just a couple repetitions of this: -- 2003-03-14T03:35:53 PROBLEM(100) Transience KeyError raised in get, checking BTrees -- 2003-03-14T03:35:53 PROBLEM(100) Transience BTree check for data succeeded -- 2003-03-14T03:35:53 PROBLEM(100) Transience BTree check for index succeeded - C On Thu, 2003-03-13 at 18:18, John Eikenberry wrote: Patch applied and the first results are in... so far its a lot of these: 2003-03-13T15:18:07 PROBLEM(100) Transience KeyError raised in get, checking _data BTree -- 2003-03-13T15:18:07 PROBLEM(100) Transience BTree check succeeded Chris McDonough wrote: Hi John, Can you apply the attached diff to your Transience.py file and run with it in place for a couple of days? It will not fix the problem (the symptoms will remain) but it should print some diagnostic information to the Zope event log (the STUPID_LOG_FILE, hopefully you've got that going) that will help us track down what this might be. Once you notice it happen, send the relevant parts of your logfile to me and I will see if I can analyze it. - C On Thu, 2003-03-13 at 15:19, John Eikenberry wrote: Sorry, its Zope 2.6.1. Chris McDonough wrote: John, Which Zope 2.6? Zope 2.6.1? Here's what line 807 of the current Transience.py looks like: v = self._data[b].get(k, notfound) Does yours look like that? Yes. What is the value of the __version__ variable at the top of the Transience.py file? __version__='$Revision: 1.28.6.4 $'[11:-2] On Thu, 2003-03-13 at 07:11, John Eikenberry wrote: Since upgrading to Zope-2.6 we've been getting KeyErrors when using Sessions. They seem to happen more now that we've started using hasSessionData(), but I'm pretty sure they happened prior to that. Anyways, here are the 2 related tracebacks. Has anyone else seen these? Traceback #1 occurs most frequently. The KeyError's value is an unprintable string of non-ascii characters. * Module ZPublisher.Publish, line 150, in publish_module * Module ZPublisher.Publish, line 114, in publish * Module The application server.App.startup, line 182, in zpublisher_exception_hook * Module ZPublisher.Publish, line 98, in publish * Module ZPublisher.mapply, line 88, in mapply * Module ZPublisher.Publish, line 39, in call_object * Module App.special_dtml, line 61, in __call__ * Module DocumentTemplate.DT_String, line 474, in __call__ * Module Products.Transience.Transience, line 342, in nudge * Module Products.Transience.Transience, line 467, in _getCurrentBucket * Module Products.TemporaryFolder.LowConflictConnection, line 34, in setstate * Module Products.TemporaryFolder.TemporaryStorage, line 94, in load KeyError: Traceback #2 happens less frequently, though today it seemed like it was
Re: [Zope-dev] Session Errors
Sorry for the length of this one... but I'm trying to braindump to give you as much info about the problem as possible. To be sure it doesn't get lost in my below ramblings, there is probably important peice of information I haven't mentioned yet... that is that these errors seem to coincide with the session data timeout setting [1]. I don't get the errors at all until the timeout is reached or has passed. [1] The timeout setting I'm refering to is denoted by the label: Data object timeout value in minutes on the /temp_folder/session_data object. Chris McDonough wrote: OK, thanks John. Let's try one more thing... currently the mounted database used to store the session data uses a connection that ignores read conflicts. This is known to be bad because the machinery which deals with keeping the sessioning index data will also ignore read conflicts, which may create inconcstencies between two data structures (BTrees) that need to be kept in sync. I tried this and it seemed to help some. I haven't seen the get() error we've been dicussing yet, but a the load() error just occurred (line 94 in TemporaryStorage - this was error #1 in my original email). Though the traceback is a bit different from my original email, as the LowConflictConnection isn't being used. Here's the new Traceback: Error Type: KeyError Error Value: [non-ascii chars] Traceback (innermost last): * Module ZPublisher.Publish, line 98, in publish * Module ZPublisher.mapply, line 88, in mapply * Module ZPublisher.Publish, line 39, in call_object * Module Products.DotOrg.Pages.KPage, line 110, in testSession * Module Products.DotOrg.Utils.Spawn, line 42, in launchProcess * Module Products.DotOrg.Utils.Spawn, line 73, in storeArgs * Module Products.Sessions.SessionDataManager, line 180, in * _getSessionDataObject * Module Products.Transience.Transience, line 175, in new_or_existing * Module Products.Transience.Transience, line 797, in get * Module Products.Transience.Transience, line 546, in _getCurrentBucket * Module ZODB.Connection, line 509, in setstate * Module Products.TemporaryFolder.TemporaryStorage, line 94, in load Here's a patch to lib/python/Products/TemporaryFolder/TemporaryFolder.py that reenables read conflict generation on the database. Index: TemporaryFolder.py === RCS file: /cvs-repository/Zope/lib/python/Products/TemporaryFolder/TemporaryFolder.py,v retrieving revision 1.7 diff -r1.7 TemporaryFolder.py 72c72 db.klass = LowConflictConnection --- #db.klass = LowConflictConnection You may see many more conflicts with this running. But maybe the data structures will not become desynchronized. You weren't kidding about the increase in conflict errors. Another problem, still unexplained, experienced by Andrew Athan, is that if a reference is made to a session data object from within the standard error message, somehow things get screwy under high load. If you're doing the same, please let me know. Before this started happening there was a hasSessionData check getting called during standard error publishing, though we removed that early this week when this started happening. --- It might help you to better understand what might be causing the problem if you know where we're using sessions and how we can force this problem to occur. Not sure if this willl be of much help, but thought it couldn't hurt. We use sessions primarily as a sort of authenticated user marker. It just stored their username and a state field that get used in non-authenticated sections of our site to detect the user as having logged into the site (we can then raise an unautorized error to get the basic auth info for that user). Anyways, these calls happen on our basic Content class (subclassed from DTMLMethod) in its __call__() method. We use it a couple other places for small things, but this one sees the most use. I've figured out how to force these errors to happen to some extent. I've written a method that starts up a thread, which uses Client.call to call another method, which then basically just loops endlessly calling hasSessionData and getSessionData, incrementing a number in the session data and sleeping for a N number of seconds between loops. One of these guys will run forever without a problem. Once you start a second thread ReadConflictErrors start getting raised. Which thread gets the conflict and which one keeps working seems variable (probably just a timing thing). If I start enough of these threads I can cause the error to happen. But only once the session timeout is reached. Note that to help speed up getting the errors I either set the session time to 1 minute via _setTimeout() call or even manually tweak the appropriate session data managers attributes (_timeout_secs, _period and _timeout_slices) to very small values (ie. a few seconds). -- John Eikenberry
Re: [Zope-dev] Session Errors
John Eikenberry wrote: Once you start a second thread ReadConflictErrors start getting raised. Which thread gets the conflict and which one keeps working seems variable (probably just a timing thing). If I start enough of these threads I can cause the error to happen. But only once the session timeout is reached. Of course, once I finally post this I notice that the behaviour is not really this way. If the sleep delay is set higher and in a way such that the 2 threads won't run at the same time very often, they will both run fine. Its only if they try to access the sessions at the same time that the ReadConflictError gets raised. Oh, and in my threading code I catch/print/ignore these errors so my threads will keep going. The threads also do a full commit each time through (to help better simulate actual usage). -- John Eikenberry [EMAIL PROTECTED] __ A society that will trade a little liberty for a little order will deserve neither and lose both. --B. Franklin ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Session Errors
John, Which Zope 2.6? Zope 2.6.1? Here's what line 807 of the current Transience.py looks like: v = self._data[b].get(k, notfound) Does yours look like that? What is the value of the __version__ variable at the top of the Transience.py file? On Thu, 2003-03-13 at 07:11, John Eikenberry wrote: Since upgrading to Zope-2.6 we've been getting KeyErrors when using Sessions. They seem to happen more now that we've started using hasSessionData(), but I'm pretty sure they happened prior to that. Anyways, here are the 2 related tracebacks. Has anyone else seen these? Traceback #1 occurs most frequently. The KeyError's value is an unprintable string of non-ascii characters. * Module ZPublisher.Publish, line 150, in publish_module * Module ZPublisher.Publish, line 114, in publish * Module The application server.App.startup, line 182, in zpublisher_exception_hook * Module ZPublisher.Publish, line 98, in publish * Module ZPublisher.mapply, line 88, in mapply * Module ZPublisher.Publish, line 39, in call_object * Module App.special_dtml, line 61, in __call__ * Module DocumentTemplate.DT_String, line 474, in __call__ * Module Products.Transience.Transience, line 342, in nudge * Module Products.Transience.Transience, line 467, in _getCurrentBucket * Module Products.TemporaryFolder.LowConflictConnection, line 34, in setstate * Module Products.TemporaryFolder.TemporaryStorage, line 94, in load KeyError: Traceback #2 happens less frequently, though today it seemed like it was trying to catch up (3 of these today). * Module ZPublisher.Publish, line 98, in publish * Module ZPublisher.mapply, line 88, in mapply * Module ZPublisher.Publish, line 39, in call_object * Module OFS.DTMLMethod, line 126, in __call__ * Module DocumentTemplate.DT_String, line 474, in __call__ * Module Products.DotOrg.Pages.KContent, line 149, in __call__ * Module Products.DotOrg.Pages.KContent, line 194, in getEditInfo * Module Products.DotOrg.Pages.KContent, line 506, in hasSessionData * Module Products.Sessions.SessionDataManager, line 101, in hasSessionData * Module Products.Sessions.SessionDataManager, line 175, in _hasSessionDataObject * Module Products.Transience.Transience, line 838, in has_key * Module Products.Transience.Transience, line 807, in get KeyError: 1047409860 -- John Eikenberry [EMAIL PROTECTED] __ A society that will trade a little liberty for a little order will deserve neither and lose both. --B. Franklin ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope ) ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Session Errors
Sorry, its Zope 2.6.1. Chris McDonough wrote: John, Which Zope 2.6? Zope 2.6.1? Here's what line 807 of the current Transience.py looks like: v = self._data[b].get(k, notfound) Does yours look like that? Yes. What is the value of the __version__ variable at the top of the Transience.py file? __version__='$Revision: 1.28.6.4 $'[11:-2] On Thu, 2003-03-13 at 07:11, John Eikenberry wrote: Since upgrading to Zope-2.6 we've been getting KeyErrors when using Sessions. They seem to happen more now that we've started using hasSessionData(), but I'm pretty sure they happened prior to that. Anyways, here are the 2 related tracebacks. Has anyone else seen these? Traceback #1 occurs most frequently. The KeyError's value is an unprintable string of non-ascii characters. * Module ZPublisher.Publish, line 150, in publish_module * Module ZPublisher.Publish, line 114, in publish * Module The application server.App.startup, line 182, in zpublisher_exception_hook * Module ZPublisher.Publish, line 98, in publish * Module ZPublisher.mapply, line 88, in mapply * Module ZPublisher.Publish, line 39, in call_object * Module App.special_dtml, line 61, in __call__ * Module DocumentTemplate.DT_String, line 474, in __call__ * Module Products.Transience.Transience, line 342, in nudge * Module Products.Transience.Transience, line 467, in _getCurrentBucket * Module Products.TemporaryFolder.LowConflictConnection, line 34, in setstate * Module Products.TemporaryFolder.TemporaryStorage, line 94, in load KeyError: Traceback #2 happens less frequently, though today it seemed like it was trying to catch up (3 of these today). * Module ZPublisher.Publish, line 98, in publish * Module ZPublisher.mapply, line 88, in mapply * Module ZPublisher.Publish, line 39, in call_object * Module OFS.DTMLMethod, line 126, in __call__ * Module DocumentTemplate.DT_String, line 474, in __call__ * Module Products.DotOrg.Pages.KContent, line 149, in __call__ * Module Products.DotOrg.Pages.KContent, line 194, in getEditInfo * Module Products.DotOrg.Pages.KContent, line 506, in hasSessionData * Module Products.Sessions.SessionDataManager, line 101, in hasSessionData * Module Products.Sessions.SessionDataManager, line 175, in _hasSessionDataObject * Module Products.Transience.Transience, line 838, in has_key * Module Products.Transience.Transience, line 807, in get KeyError: 1047409860 -- John Eikenberry [EMAIL PROTECTED] __ A society that will trade a little liberty for a little order will deserve neither and lose both. --B. Franklin ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope ) -- John Eikenberry [EMAIL PROTECTED] __ A society that will trade a little liberty for a little order will deserve neither and lose both. --B. Franklin ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Session Errors
Hi John,
Can you apply the attached diff to your Transience.py file and run with
it in place for a couple of days? It will not fix the problem (the
symptoms will remain) but it should print some diagnostic information to
the Zope event log (the STUPID_LOG_FILE, hopefully you've got that
going) that will help us track down what this might be.
Once you notice it happen, send the relevant parts of your logfile to me
and I will see if I can analyze it.
- C
On Thu, 2003-03-13 at 15:19, John Eikenberry wrote:
Sorry, its Zope 2.6.1.
Chris McDonough wrote:
John,
Which Zope 2.6? Zope 2.6.1? Here's what line 807 of the current
Transience.py looks like:
v = self._data[b].get(k, notfound)
Does yours look like that?
Yes.
What is the value of the __version__ variable at the top of the
Transience.py file?
__version__='$Revision: 1.28.6.4 $'[11:-2]
On Thu, 2003-03-13 at 07:11, John Eikenberry wrote:
Since upgrading to Zope-2.6 we've been getting KeyErrors when using
Sessions. They seem to happen more now that we've started using
hasSessionData(), but I'm pretty sure they happened prior to that.
Anyways, here are the 2 related tracebacks. Has anyone else seen these?
Traceback #1 occurs most frequently. The KeyError's value is an unprintable
string of non-ascii characters.
* Module ZPublisher.Publish, line 150, in publish_module
* Module ZPublisher.Publish, line 114, in publish
* Module The application server.App.startup, line 182, in
zpublisher_exception_hook
* Module ZPublisher.Publish, line 98, in publish
* Module ZPublisher.mapply, line 88, in mapply
* Module ZPublisher.Publish, line 39, in call_object
* Module App.special_dtml, line 61, in __call__
* Module DocumentTemplate.DT_String, line 474, in __call__
* Module Products.Transience.Transience, line 342, in nudge
* Module Products.Transience.Transience, line 467, in _getCurrentBucket
* Module Products.TemporaryFolder.LowConflictConnection, line 34, in
setstate
* Module Products.TemporaryFolder.TemporaryStorage, line 94, in load
KeyError:
Traceback #2 happens less frequently, though today it seemed like it was
trying to catch up (3 of these today).
* Module ZPublisher.Publish, line 98, in publish
* Module ZPublisher.mapply, line 88, in mapply
* Module ZPublisher.Publish, line 39, in call_object
* Module OFS.DTMLMethod, line 126, in __call__
* Module DocumentTemplate.DT_String, line 474, in __call__
* Module Products.DotOrg.Pages.KContent, line 149, in __call__
* Module Products.DotOrg.Pages.KContent, line 194, in getEditInfo
* Module Products.DotOrg.Pages.KContent, line 506, in hasSessionData
* Module Products.Sessions.SessionDataManager, line 101, in hasSessionData
* Module Products.Sessions.SessionDataManager, line 175, in
_hasSessionDataObject
* Module Products.Transience.Transience, line 838, in has_key
* Module Products.Transience.Transience, line 807, in get
KeyError: 1047409860
--
John Eikenberry [EMAIL PROTECTED]
__
A society that will trade a little liberty for a little order
will deserve neither and lose both.
--B. Franklin
___
Zope-Dev maillist - [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )
--
John Eikenberry [EMAIL PROTECTED]
__
A society that will trade a little liberty for a little order
will deserve neither and lose both.
--B. Franklin
___
Zope-Dev maillist - [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )
? btreecheck.diff
? kedaipatch
Index: Transience.py
===
RCS file: /cvs-repository/Zope/lib/python/Products/Transience/Transience.py,v
retrieving revision 1.28.6.4
diff -r1.28.6.4 Transience.py
34a35
from BTrees.check import check, display
45a47
from cStringIO import StringIO
807c809,830
v = self._data[b].get(k, notfound)
---
try:
v = self._data[b].get(k, notfound)
except KeyError:
LOG('Transience', WARNING,
'KeyError raised in get, checking _data BTree')
try:
check(self._data)
Re: [Zope-dev] Session Errors
Patch applied and the first results are in... so far its a lot of these: 2003-03-13T15:18:07 PROBLEM(100) Transience KeyError raised in get, checking _data BTree -- 2003-03-13T15:18:07 PROBLEM(100) Transience BTree check succeeded Chris McDonough wrote: Hi John, Can you apply the attached diff to your Transience.py file and run with it in place for a couple of days? It will not fix the problem (the symptoms will remain) but it should print some diagnostic information to the Zope event log (the STUPID_LOG_FILE, hopefully you've got that going) that will help us track down what this might be. Once you notice it happen, send the relevant parts of your logfile to me and I will see if I can analyze it. - C On Thu, 2003-03-13 at 15:19, John Eikenberry wrote: Sorry, its Zope 2.6.1. Chris McDonough wrote: John, Which Zope 2.6? Zope 2.6.1? Here's what line 807 of the current Transience.py looks like: v = self._data[b].get(k, notfound) Does yours look like that? Yes. What is the value of the __version__ variable at the top of the Transience.py file? __version__='$Revision: 1.28.6.4 $'[11:-2] On Thu, 2003-03-13 at 07:11, John Eikenberry wrote: Since upgrading to Zope-2.6 we've been getting KeyErrors when using Sessions. They seem to happen more now that we've started using hasSessionData(), but I'm pretty sure they happened prior to that. Anyways, here are the 2 related tracebacks. Has anyone else seen these? Traceback #1 occurs most frequently. The KeyError's value is an unprintable string of non-ascii characters. * Module ZPublisher.Publish, line 150, in publish_module * Module ZPublisher.Publish, line 114, in publish * Module The application server.App.startup, line 182, in zpublisher_exception_hook * Module ZPublisher.Publish, line 98, in publish * Module ZPublisher.mapply, line 88, in mapply * Module ZPublisher.Publish, line 39, in call_object * Module App.special_dtml, line 61, in __call__ * Module DocumentTemplate.DT_String, line 474, in __call__ * Module Products.Transience.Transience, line 342, in nudge * Module Products.Transience.Transience, line 467, in _getCurrentBucket * Module Products.TemporaryFolder.LowConflictConnection, line 34, in setstate * Module Products.TemporaryFolder.TemporaryStorage, line 94, in load KeyError: Traceback #2 happens less frequently, though today it seemed like it was trying to catch up (3 of these today). * Module ZPublisher.Publish, line 98, in publish * Module ZPublisher.mapply, line 88, in mapply * Module ZPublisher.Publish, line 39, in call_object * Module OFS.DTMLMethod, line 126, in __call__ * Module DocumentTemplate.DT_String, line 474, in __call__ * Module Products.DotOrg.Pages.KContent, line 149, in __call__ * Module Products.DotOrg.Pages.KContent, line 194, in getEditInfo * Module Products.DotOrg.Pages.KContent, line 506, in hasSessionData * Module Products.Sessions.SessionDataManager, line 101, in hasSessionData * Module Products.Sessions.SessionDataManager, line 175, in _hasSessionDataObject * Module Products.Transience.Transience, line 838, in has_key * Module Products.Transience.Transience, line 807, in get KeyError: 1047409860 -- John Eikenberry [EMAIL PROTECTED] __ A society that will trade a little liberty for a little order will deserve neither and lose both. --B. Franklin ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope ) -- John Eikenberry [EMAIL PROTECTED] __ A society that will trade a little liberty for a little order will deserve neither and lose both. --B. Franklin ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope ) ? btreecheck.diff ? kedaipatch Index: Transience.py === RCS file: /cvs-repository/Zope/lib/python/Products/Transience/Transience.py,v retrieving revision 1.28.6.4 diff -r1.28.6.4 Transience.py 34a35 from BTrees.check import check, display 45a47 from cStringIO import
Re: [Zope-dev] Session Errors
OK, thanks John. I hate to ask this (I should have done this to start with), but would you be willing to use the following patch --against the original file, not your recently patched version-- and try again? I only checked one of the two BTrees that might be at the heart of the problem with the first patch, this patch checks the second as well. - C On Thu, 2003-03-13 at 18:18, John Eikenberry wrote: Patch applied and the first results are in... so far its a lot of these: 2003-03-13T15:18:07 PROBLEM(100) Transience KeyError raised in get, checking _data BTree -- 2003-03-13T15:18:07 PROBLEM(100) Transience BTree check succeeded Chris McDonough wrote: Hi John, Can you apply the attached diff to your Transience.py file and run with it in place for a couple of days? It will not fix the problem (the symptoms will remain) but it should print some diagnostic information to the Zope event log (the STUPID_LOG_FILE, hopefully you've got that going) that will help us track down what this might be. Once you notice it happen, send the relevant parts of your logfile to me and I will see if I can analyze it. - C On Thu, 2003-03-13 at 15:19, John Eikenberry wrote: Sorry, its Zope 2.6.1. Chris McDonough wrote: John, Which Zope 2.6? Zope 2.6.1? Here's what line 807 of the current Transience.py looks like: v = self._data[b].get(k, notfound) Does yours look like that? Yes. What is the value of the __version__ variable at the top of the Transience.py file? __version__='$Revision: 1.28.6.4 $'[11:-2] On Thu, 2003-03-13 at 07:11, John Eikenberry wrote: Since upgrading to Zope-2.6 we've been getting KeyErrors when using Sessions. They seem to happen more now that we've started using hasSessionData(), but I'm pretty sure they happened prior to that. Anyways, here are the 2 related tracebacks. Has anyone else seen these? Traceback #1 occurs most frequently. The KeyError's value is an unprintable string of non-ascii characters. * Module ZPublisher.Publish, line 150, in publish_module * Module ZPublisher.Publish, line 114, in publish * Module The application server.App.startup, line 182, in zpublisher_exception_hook * Module ZPublisher.Publish, line 98, in publish * Module ZPublisher.mapply, line 88, in mapply * Module ZPublisher.Publish, line 39, in call_object * Module App.special_dtml, line 61, in __call__ * Module DocumentTemplate.DT_String, line 474, in __call__ * Module Products.Transience.Transience, line 342, in nudge * Module Products.Transience.Transience, line 467, in _getCurrentBucket * Module Products.TemporaryFolder.LowConflictConnection, line 34, in setstate * Module Products.TemporaryFolder.TemporaryStorage, line 94, in load KeyError: Traceback #2 happens less frequently, though today it seemed like it was trying to catch up (3 of these today). * Module ZPublisher.Publish, line 98, in publish * Module ZPublisher.mapply, line 88, in mapply * Module ZPublisher.Publish, line 39, in call_object * Module OFS.DTMLMethod, line 126, in __call__ * Module DocumentTemplate.DT_String, line 474, in __call__ * Module Products.DotOrg.Pages.KContent, line 149, in __call__ * Module Products.DotOrg.Pages.KContent, line 194, in getEditInfo * Module Products.DotOrg.Pages.KContent, line 506, in hasSessionData * Module Products.Sessions.SessionDataManager, line 101, in hasSessionData * Module Products.Sessions.SessionDataManager, line 175, in _hasSessionDataObject * Module Products.Transience.Transience, line 838, in has_key * Module Products.Transience.Transience, line 807, in get KeyError: 1047409860 -- John Eikenberry [EMAIL PROTECTED] __ A society that will trade a little liberty for a little order will deserve neither and lose both. --B. Franklin ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope ) -- John Eikenberry [EMAIL PROTECTED] __ A society that will trade a little liberty for a little order will deserve neither and lose both. --B. Franklin ___ Zope-Dev maillist - [EMAIL PROTECTED]
Re: [Zope-dev] Session Event Listeners
Beef a wrote: Does anyone have any idea how to register session event listeners in Zope 2? Chris McDonough is tha man when it comes to this. It's probably all documented in the online Zope Book by now, or in the Zope distribution somewhere. In any case, he'll be able to tell you wehre to find it... cheers, Chris ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Session Event Listeners
Does anyone have any idea how to register session event listeners in Zope 2? This is a standard concept in Java/JSP/ASP et al. Basically it allows a method to be registered and executed when a http session is either created, lost or invalidated. Is there such a thing in Zope? Thanks -- Brian _ Express yourself with cool emoticons http://messenger.msn.co.uk ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Session Event Listeners
Beef a writes: Does anyone have any idea how to register session event listeners in Zope 2? [..] Check with the /temp_folder/session_data in your Zope, or create your own Transient Object Container, if there is no such object. Cheers, Clemens ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Session Tracking in 2.5
Chris McDonough wrote There are supposed to be docs for the sessioning stuff, and there *are* API docs in the help system. However, the narrative docs that went along with CST were not translated for the Zope 2.5 stuff. The docs folks here dont seem to think its important, so I may need to do this and put it in the Dev Guide. Well, I know _I_ think Session support is important (hint: I wouldn't have spent the effort on SQLSession otherwise). If the decision was made to take the CST package and change the interface before making it part of the preinstalled set of Products, I'd _strongly_ urge that the documentation for it be updated. Anthony -- Anthony Baxter [EMAIL PROTECTED] It's never too late to have a happy childhood. ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Session Tracking in 2.5
Well, I know _I_ think Session support is important (hint: I wouldn't have spent the effort on SQLSession otherwise). If the decision was made to take the CST package and change the interface before making it part of the preinstalled set of Products, I'd _strongly_ urge that the documentation for it be updated. FTR, and to clarify, the *API* docs for the sessioning capabilities in Zope 2.5 exist in the help system. The narrative docs, however, are scant and consist mainly of a mention in the CVS version of the Zope Book. It is this that I'd like to remedy, once I catch my breath. - C ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
[Zope-dev] Session Tracking in 2.5
I thought it was time to catch up on the CST stuff which has made it into the core, only to find that it's only *based on* Chris' CST stuff. And I can't find a fishbowl project anywhere describing the rationale, apis, whatever... Is there a fishbowl for this? Or was session tracking implemented as a guerilla project? seb ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Session Tracking in 2.5
CST was done via the fishbowl project at http://dev.zope.org/Wikis/DevSite/Projects/CoreSessionTracking/FrontPage. There is an explanation of the difference between the Zope 2.5 stuff and CST on the CurrentStatus page. Note that this situation makes me want to cry as well. There are supposed to be docs for the sessioning stuff, and there *are* API docs in the help system. However, the narrative docs that went along with CST were not translated for the Zope 2.5 stuff. The docs folks here dont seem to think its important, so I may need to do this and put it in the Dev Guide. On 08 Jan 2002 11:39:35 + seb bacon [EMAIL PROTECTED] wrote: I thought it was time to catch up on the CST stuff which has made it into the core, only to find that it's only *based on* Chris' CST stuff. And I can't find a fishbowl project anywhere describing the rationale, apis, whatever... Is there a fishbowl for this? Or was session tracking implemented as a guerilla project? seb ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope ) ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Session Tracking in 2.5
On Tue, 2002-01-08 at 13:03, Chris McDonough wrote: There are supposed to be docs for the sessioning stuff, and there *are* API docs in the help system. However, the narrative docs that went along with CST were not translated for the Zope 2.5 stuff. The docs folks here dont seem to think its important, so I may need to do this and put it in the Dev Guide. It's important because session tracking is such a crucial feature for people migrating from other web platforms - seeing detailed documentation will make them so much more comfortable. it would also be a crying shame not to make use of all the docs you already generated along the way. So, please do try to find the time, and show my plea to those who make room for doing such things :-) seb On 08 Jan 2002 11:39:35 + seb bacon [EMAIL PROTECTED] wrote: I thought it was time to catch up on the CST stuff which has made it into the core, only to find that it's only *based on* Chris' CST stuff. And I can't find a fishbowl project anywhere describing the rationale, apis, whatever... Is there a fishbowl for this? Or was session tracking implemented as a guerilla project? seb ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope ) ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Session Tracking in 2.5
It's important because session tracking is such a crucial feature for people migrating from other web platforms - seeing detailed documentation will make them so much more comfortable. it would also be a crying shame not to make use of all the docs you already generated along the way. I agree. It's silly not to use the existing docs. So, please do try to find the time, and show my plea to those who make room for doing such things :-) They should be reading this list. - C ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Session Tracking in 2.5
Chris McDonough writes: The docs folks here dont seem to think its important, so I may need to do this and put it in the Dev Guide. As always, the developers are industrious and the documentors are lazy. As a result, source is usually far beyond the documentation ;-) Dieter ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
[Zope-dev] Session
How can I use this line ? Do I need any product ?
dtml-call "Session.set('myVariable','somevalue')
Anderson
___
Zope-Dev maillist - [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
http://lists.zope.org/mailman/listinfo/zope-announce
http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Session Objects?
On Tue, 23 May 2000, Adam Pawliuk wrote: Although there is several user characteristics that we would like to store in a session object cached in server RAM, such as the user's unique system ID, greeting name, etc; rather than having to hit the DB for this information on each request. Basically we would like to use something similar to the session objects in ASP (yuck), or Java JSP/servlet model. I've seen several Zope session products although they all seem to hook directly to a DB and don't seem to provide memory persistence which is what we would like to use. Is there any existing products which do this? Boring old ZSQL Methods actually do exactly what you want - check out the caching options under the advanced tab (or the ZSQL docs on zope.org). The database will only be hit the first time you call the method with a given set of parameters, or when maximum time to cache expires. If your GUF hooks make use of ZSQL methods, you should have a look at tuning these too as it can be a big performance improvement. Actually - I tell a slight lie. The DB might be hit once for each thread, as the ZSQL cache is not shared between Zope threads. You need to be aware of this as a cached result may be refreshed in one thread but not yet expired in another and can give ambiguous results and unreproducable bugs if you are not aware of it when you are developing. -- ___ // Zen (alias Stuart Bishop) Work: [EMAIL PROTECTED] // E N Senior Systems Alchemist Play: [EMAIL PROTECTED] //__ Computer Science, RMIT WWW: http://www.cs.rmit.edu.au/~zen ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
