Re: [Zope-dev] When is the configuration read in Zope 2.7?
Things changed around a bit since I did this patch. I'm trying to get a new one working but am having some issues with the 2.7 trunk in svn. I think in general the startup sequence could use some clean up. I'm working on mapping how it all works maybe that would help make it less opaque. -EAD On May 15, 2004, at 12:53 PM, Chris McDonough wrote: I think Fred might have some ideas about how to do the ZOPE_CONFIG feature better if you can coax them out of him. ;-) On Sat, 2004-05-15 at 03:56, Andreas Jung wrote: --On Samstag, 15. Mai 2004 1:59 Uhr +0200 Stefan H. Holek [EMAIL PROTECTED] wrote: Zope tests fail for me, unfortunately Andreas cannot reproduce this. The ZOPE_CONFIG patch was responsible for other failures. I reverted the change and check for a better solution by next week. -aj ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope ) ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: [ZODB-Dev] Re: BTrees strangeness (was Zope 2.X BIG Session problems - blocker - our site dies - need help of experience Zope developer, please)
:-( I grabbed Transience.py and TemporaryStorage.py from the Zope-2_7-branch of CVS this morning and dropped them onto one of our customer's systems. About 20 minutes later I found the following in the error_log: Traceback (most recent call last): File c:\Zope-2.7\lib\python\ZPublisher\Publish.py, line 100, in publish request, bind=1) File c:\Zope-2.7\lib\python\ZPublisher\mapply.py, line 88, in mapply if debug is not None: return debug(object,args,context) File c:\Zope-2.7\lib\python\ZPublisher\Publish.py, line 40, in call_object result=apply(object,args) # Type scr to step into published object. File c:\Zope-2.7\lib\python\OFS\DTMLDocument.py, line 128, in __call__ r=apply(HTML.__call__, (self, (client, bself), REQUEST), kw) File c:\Zope-2.7\lib\python\DocumentTemplate\DT_String.py, line 474, in __call__ try: result = render_blocks(self._v_blocks, md) File c:\Zope-2.7\lib\python\OFS\DTMLDocument.py, line 121, in __call__ r=apply(HTML.__call__, (self, bself, REQUEST), kw) File c:\Zope-2.7\lib\python\DocumentTemplate\DT_String.py, line 474, in __call__ try: result = render_blocks(self._v_blocks, md) File c:\Zope-2.7\lib\python\OFS\DTMLDocument.py, line 121, in __call__ r=apply(HTML.__call__, (self, bself, REQUEST), kw) File c:\Zope-2.7\lib\python\DocumentTemplate\DT_String.py, line 474, in __call__ try: result = render_blocks(self._v_blocks, md) File c:\Zope-2.7\lib\python\DocumentTemplate\DT_Util.py, line 201, in eval return eval(code, d) File string, line 1, in expression File c:\Zope-2.7\lib\python\AccessControl\ZopeGuards.py, line 67, in guarded_getitem v = object[index] File c:\Zope-2.7\lib\python\Products\Transience\TransientObject.py, line 170, in __getitem__ return self._container[k] KeyError: 'accountType' After getting the error, I changed them over to FileStorage. I haven't seen any errors since then. I will be more than happy to spend some time doing whatever I can to help track this down and/or test updates. Steve Chris McDonough wrote: On Sat, 2004-05-15 at 23:55, Steve Jibson wrote: quite yet been vetted (see earlier posts in this thread regarding Publish.py). This should work itself out over the next week or so I suspect. Relatively speaking, this is only a minor concern. It just means I shouldn't try to access session in the standard_error_message until a solution is found. We were doing that in our product and I've already pulled that out. After all, there are NEVER any errors, so why should we need to monkey with standard_error_message ;-) It would be nice, for completeness, if this were resolved for 2.7.1. Yes, it should work itself out over the next week or so, and definitely before 2.7.1. Am I safe just dropping those two files from CVS into an existing 2.7 site or do I need to grab everything else from the Zope-2_7-branch? I think you should be safe with just those two, but were I you I would just get the 2.7 branch out of CVS in its entirety (then you can update more easily if there are fixes). Note that if you have transient object containers in a permanent storage (like FileStorage), the operation is pretty much a one way one if you want to keep the data in the transient object container: you can go to the new code, then go back to the old code if necessarfy, but you will likely lose data in the TOC if you do. Obviously this doesn't effect stuff in a TemporaryStorage (because it goes away when Zope is shut down). - C ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Getting Zope 2.7 trunk working
I'm trying to do work off of the 2.7 trunk in svn. After doing:
svn svn://svn.zope.org/repos/main/Zope/trunk zope
./configure
make
make instance
when I run bin/zopectl debug I get the following, what am I doing wrong?
No handlers could be found for logger Init
Starting debugger (the name app is bound to the top-level Zope object)
2004-05-17 11:33:09 WARNING Init Ambiguous name for method of
OFS.Uninstalled.BrokenClass: 'manage' != 'manage_main'
2004-05-17 11:33:09 WARNING Init Ambiguous name for method of
OFS.Uninstalled.BrokenClass: 'manage' != 'manage_workspace'
2004-05-17 11:33:10 ERROR Zope.ZODBMountPoint Failed to mount database.
exceptions.AttributeError ('Connection' object has no attribute
'_getMountedConnection')
Traceback (most recent call last):
File
/home/edahl/src/zope/lib/python/Products/ZODBMountPoint/Mount.py,
line 93, in _getOrOpenObject
conn = self._getMountedConnection(anyjar)
File
/home/edahl/src/zope/lib/python/Products/ZODBMountPoint/Mount.py,
line 71, in _getMountedConnection
conn = anyjar._getMountedConnection(db_name)
AttributeError: 'Connection' object has no attribute
'_getMountedConnection'
2004-05-17 11:33:10 ERROR Zope Default Object Creation Could not add a
/temp_folder mount point due to an error.
Traceback (most recent call last):
File /home/edahl/src/zope/lib/python/OFS/Application.py, line 358,
in install_tempfolder_and_sdc
manage_addMounts(app, ('/temp_folder',))
File
/home/edahl/src/zope/lib/python/Products/ZODBMountPoint/
MountedObject.py, line 283, in manage_addMounts
mo._test(app)
File
/home/edahl/src/zope/lib/python/Products/ZODBMountPoint/Mount.py,
line 126, in _test
self._getOrOpenObject(parent)
File
/home/edahl/src/zope/lib/python/Products/ZODBMountPoint/Mount.py,
line 93, in _getOrOpenObject
conn = self._getMountedConnection(anyjar)
File
/home/edahl/src/zope/lib/python/Products/ZODBMountPoint/Mount.py,
line 71, in _getMountedConnection
conn = anyjar._getMountedConnection(db_name)
AttributeError: 'Connection' object has no attribute
'_getMountedConnection'
___
Zope-Dev maillist - [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] When is the configuration read in Zope 2.7?
Things changed around a bit since I did this patch. I'm trying to get a new one working but am having some issues with the 2.7 trunk in svn. I think in general the startup sequence could use some clean up. I'm working on mapping how it all works maybe that would help make it less opaque. -EAD On May 15, 2004, at 12:53 PM, Chris McDonough wrote: I think Fred might have some ideas about how to do the ZOPE_CONFIG feature better if you can coax them out of him. ;-) On Sat, 2004-05-15 at 03:56, Andreas Jung wrote: --On Samstag, 15. Mai 2004 1:59 Uhr +0200 Stefan H. Holek [EMAIL PROTECTED] wrote: Zope tests fail for me, unfortunately Andreas cannot reproduce this. The ZOPE_CONFIG patch was responsible for other failures. I reverted the change and check for a better solution by next week. -aj ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope ) ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: Getting Zope 2.7 trunk working
Erik A.Dahl wrote: I'm trying to do work off of the 2.7 trunk in svn. After doing: Just for clarity: the Subversion repository has the head branch of Zope, which will be branched to create the 2.8 release branch. The 2.7 release branch remains in CVS: $ cvs -d :ext:cvs.zope.org:/cvs-repository \ co -r Zope-2_7-branch -d Zope-2.7-branch Zope The 2.7 branch head works fine for me. Tres. -- === Tres Seaver[EMAIL PROTECTED] Zope Corporation Zope Dealers http://www.zope.com ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: [ZODB-Dev] Re: BTrees strangeness (was Zope 2.X BIG Session problems - blocker - our site dies - need help of experience Zope developer, please)
On Mon, 2004-05-17 at 12:09, Steve Jibson wrote:
:-(
I grabbed Transience.py and TemporaryStorage.py from the Zope-2_7-branch
of CVS this morning and dropped them onto one of our customer's systems.
About 20 minutes later I found the following in the error_log:
Traceback (most recent call last):
File c:\Zope-2.7\lib\python\ZPublisher\Publish.py, line 100, in
publish request, bind=1)
File c:\Zope-2.7\lib\python\ZPublisher\mapply.py, line 88, in mapply
if debug is not None: return debug(object,args,context)
File c:\Zope-2.7\lib\python\ZPublisher\Publish.py, line 40, in
call_object
result=apply(object,args) # Type scr to step into published object.
File c:\Zope-2.7\lib\python\OFS\DTMLDocument.py, line 128, in __call__
r=apply(HTML.__call__, (self, (client, bself), REQUEST), kw)
File c:\Zope-2.7\lib\python\DocumentTemplate\DT_String.py, line
474, in __call__
try: result = render_blocks(self._v_blocks, md)
File c:\Zope-2.7\lib\python\OFS\DTMLDocument.py, line 121, in __call__
r=apply(HTML.__call__, (self, bself, REQUEST), kw)
File c:\Zope-2.7\lib\python\DocumentTemplate\DT_String.py, line
474, in __call__
try: result = render_blocks(self._v_blocks, md)
File c:\Zope-2.7\lib\python\OFS\DTMLDocument.py, line 121, in __call__
r=apply(HTML.__call__, (self, bself, REQUEST), kw)
File c:\Zope-2.7\lib\python\DocumentTemplate\DT_String.py, line
474, in __call__
try: result = render_blocks(self._v_blocks, md)
File c:\Zope-2.7\lib\python\DocumentTemplate\DT_Util.py, line 201,
in eval
return eval(code, d)
File string, line 1, in expression
File c:\Zope-2.7\lib\python\AccessControl\ZopeGuards.py, line 67,
in guarded_getitem
v = object[index]
File c:\Zope-2.7\lib\python\Products\Transience\TransientObject.py,
line 170, in __getitem__
return self._container[k]
KeyError: 'accountType'
This looks to me like an application error, not a Transience error.
After getting the error, I changed them over to FileStorage. I haven't
seen any errors since then. I will be more than happy to spend some
time doing whatever I can to help track this down and/or test updates.
It looks like the code assumes 'accountType' is available in the
session; change it to not make that assumption (perhaps via
SESSION.get('accountType', 'default')).
- C
___
Zope-Dev maillist - [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )
Re: [ZODB-Dev] Re: BTrees strangeness (was [Zope-dev] Zope 2.X BIG Session problems - blocker - our site dies - need help of experience Zope developer, please)
Chris McDonough wrote at 2004-5-15 13:04 -0400: ... Dieter, do you think you can read this patch and give a thumbs up or down on it? The patch looks good. On a different subject, the publisher probably shouldn't pass around traceback objects (e.g. when it calls into err_hook) as Tres believes that may be a memory leak waiting to happen. The traceback is vital for error analysis. It may not be necessary that ZPublisher touches the traceback but we will definitely need access to it during error handling. -- Dieter ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Conflict errors on BDBMinimal storage
On Mon, May 17, 2004 at 12:45:16PM -0400, Chris McDonough wrote: On Thu, 2004-05-13 at 05:23, Gerhard Schmidt wrote: Hi, whe are running portal unter zope2.7 with one zeo server and six frontend application servers. I have set up zeo to server two storages filestorage main path $INSTANCE_HOME/var/Data.fs /filestorage minimalstorage temp envdir $INSTANCE_HOME/var/session /minimalstorage temp should provide an shared session. But when is set up the frontends to use temp as /temp_folder, we see many ZODB conflict errors. The performance decreases and PosErrors appear quite often. BDBMininalStorage is no longer maintained and doesn't do any conflict resolution. Why this. I was told some time ago we should use BDBMinimalStorage to fix the conflict problems. Even though you'll need to pack frequently, you might want to try a FileStorage instead. There is no difference between BDB Storage and Filestorage regarding the conflicts. Regardless, using ZEO will presumably always generate more conflict errors than using a local ZODB because the commit time is longer, and thus there's a better chance that two threads will be committing at the same time. The bottom line is that there is no way of getting an shared session running. Why isn´t there an implementation of temporary storage for ZEO I dont care about conflit resolution. Its only session data. I simply need a shared session storage. But as Far as i can see the is no way of doing this. Are we the only site out there running Plone2 (Placeless transaltion Service) at that scale. How do others handle there Session Bye Estartu -- Gerhard Schmidt| Nick : estartu IRC : Estartu | Fischbachweg 3 || PGP Public Key 86856 Hiltenfingen | [EMAIL PROTECTED] | auf Anfrage/ Tel: 08232 77 36 4 ||on request Fax: 08232 77 36 3 || pgpqXUCWUVwl0.pgp Description: PGP signature ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: [ZODB-Dev] Re: BTrees strangeness (was Zope 2.X BIG Session problems - blocker - our site dies - need help of experience Zope developer, please)
On Mon, 17 May 2004 19:00:16 +0200 Dieter Maurer [EMAIL PROTECTED] wrote: Chris McDonough wrote at 2004-5-15 13:04 -0400: ... Dieter, do you think you can read this patch and give a thumbs up or down on it? The patch looks good. On a different subject, the publisher probably shouldn't pass around traceback objects (e.g. when it calls into err_hook) as Tres believes that may be a memory leak waiting to happen. The traceback is vital for error analysis. It may not be necessary that ZPublisher touches the traceback but we will definitely need access to it during error handling. Perhaps the traceback can be passed as a string to avoid leaks? Furthermore why can't the traceback be retrieved later from sys.exc_info()? -Casey ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: [ZODB-Dev] Re: BTrees strangeness (was Zope 2.X BIG Session problems - blocker - our site dies - need help of experience Zope developer, please)
Casey Duncan wrote: On Mon, 17 May 2004 19:00:16 +0200 Dieter Maurer [EMAIL PROTECTED] wrote: Chris McDonough wrote at 2004-5-15 13:04 -0400: ... Dieter, do you think you can read this patch and give a thumbs up or down on it? The patch looks good. On a different subject, the publisher probably shouldn't pass around traceback objects (e.g. when it calls into err_hook) as Tres believes that may be a memory leak waiting to happen. The traceback is vital for error analysis. It may not be necessary that ZPublisher touches the traceback but we will definitely need access to it during error handling. Because the traceback contains stack frames, passing it through another stack frame (via a function call) is inherently tricky: the called function must *not* raise another exception. Perhaps the traceback can be passed as a string to avoid leaks? Furthermore why can't the traceback be retrieved later from sys.exc_info()? +1; I don't want untrusted code handling tracebacks anyway. Tres. -- === Tres Seaver[EMAIL PROTECTED] Zope Corporation Zope Dealers http://www.zope.com ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Conflict errors on BDBMinimal storage
On Mon, 2004-05-17 at 13:06, Gerhard Schmidt wrote: BDBMininalStorage is no longer maintained and doesn't do any conflict resolution. Why this. I was told some time ago we should use BDBMinimalStorage to fix the conflict problems. When was that and who told you so? Even though you'll need to pack frequently, you might want to try a FileStorage instead. There is no difference between BDB Storage and Filestorage regarding the conflicts. OK. Regardless, using ZEO will presumably always generate more conflict errors than using a local ZODB because the commit time is longer, and thus there's a better chance that two threads will be committing at the same time. The bottom line is that there is no way of getting an shared session running. It works doesn't it? You just get conflict errors, right? Conflict errors are normal at a certain level. Why isnt there an implementation of temporary storage for ZEO There is, just use the tempstorage package from Zope. I dont care about conflit resolution. Its only session data. For better or worse, because it is based on ZODB, the implementation makes you need to care. I simply need a shared session storage. But as Far as i can see the is no way of doing this. I don't understand. You seem to have it working. Is performance the issue? Are we the only site out there running Plone2 (Placeless transaltion Service) at that scale. How do others handle there Session AFAIK Zope is the only appserver that even makes an attempt at allowing session data to be shared between servers like this. Most other appservers require you to set up session affinity in your load balancer and have a session database on each appserver. - C ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: [ZODB-Dev] Re: BTrees strangeness (was Zope 2.X BIG Session problems - blocker - our site dies - need help of experience Zope developer, please)
On Mon, 2004-05-17 at 13:06, Steve Jibson wrote: I'm sure 'accountType' should have been in the session. Immediately after a user logs in we populate his session with a bunch of stuff (including accountType). What happens when the session expires and he's still logged in? What I don't want to do is hide problems with session/tempstorage by using get and a default. In this particular case, there is no good default and having the application assume an incorrect accountType will cause me all kids of trouble. I still suspect this is an application error. Also of note: 1 - After having the system run for 45 minutes, I had 8 similar errors. Some were on different web pages and some had different keys that were causing the error. 2 - I have also traced through Z2.log and followed the same path through the web site that produced one of the errors and I did not get an error. 3 - Since changing this server to use FileStorage (1hr 39min ago), there has not been a single error. Is there anything I can do to help here? A small reproducible test case would help if you still believe this error is not in your own application. - C ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [ZODB-Dev] Re: BTrees strangeness (was [Zope-dev] Zope 2.X BIG Session problems - blocker - our site dies - need help of experience Zope developer, please)
On Mon, 2004-05-17 at 13:00, Dieter Maurer wrote: Chris McDonough wrote at 2004-5-15 13:04 -0400: ... Dieter, do you think you can read this patch and give a thumbs up or down on it? The patch looks good. Great, thanks for looking at it. - C ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Conflict errors on BDBMinimal storage
Gerhard Schmidt wrote at 2004-5-13 11:23 +0200: ... Is there any why to get a shared session working. You know that the ZODB is not designed for high frequency concurrent writes? When you want to reduce conflicts you will need to try hard to minimize concurrent writes. I have a (non-open) Transience implementation that writes much less then Zope's standard implementation. However, due to a ZEO weakness, its conflict resolution currently does not work (ZEO no longer supports INSTANCE_HOME). -- Dieter ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] When is the configuration read in Zope 2.7?
Ok, I think I have a fix for this but we have a little semantics issue. I thought that once the Zope config was set it could not be changed. In the new ZOPE_CONFIG patch the environ variable only gets used as a last resort in Zope.app() if no other configuration has been set. To do this I added a isConfigured() function to App.config to check its _config global. I'm assuming that this is the official Zope config, correct? Now there is a test testZopeRunConfigure in lib/python/Zope/Startup/tests/testStarter.py that runs App.config.getConfiguration() and then makes a test config file and sets it. The result is a test where the config is reset within one run. This test fails with the semantics described above. Which is correct? I'm enclosing the patch for reference. -EAD ZOPE_CONFIG.patch Description: Binary data On May 17, 2004, at 12:20 PM, Erik A.Dahl wrote: Things changed around a bit since I did this patch. I'm trying to get a new one working but am having some issues with the 2.7 trunk in svn. I think in general the startup sequence could use some clean up. I'm working on mapping how it all works maybe that would help make it less opaque. -EAD On May 15, 2004, at 12:53 PM, Chris McDonough wrote: I think Fred might have some ideas about how to do the ZOPE_CONFIG feature better if you can coax them out of him. ;-) On Sat, 2004-05-15 at 03:56, Andreas Jung wrote: --On Samstag, 15. Mai 2004 1:59 Uhr +0200 Stefan H. Holek [EMAIL PROTECTED] wrote: Zope tests fail for me, unfortunately Andreas cannot reproduce this. The ZOPE_CONFIG patch was responsible for other failures. I reverted the change and check for a better solution by next week. -aj ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope ) ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope ) ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: [ZODB-Dev] Re: BTrees strangeness (was Zope 2.X BIG Session problems - blocker - our site dies - need help of experience Zope developer, please)
Chris McDonough wrote: On Mon, 2004-05-17 at 13:06, Steve Jibson wrote: I'm sure 'accountType' should have been in the session. Immediately after a user logs in we populate his session with a bunch of stuff (including accountType). What happens when the session expires and he's still logged in? We're using ExUserFolder for authentication. We have it set to log users out after 10 minutes. We have sessions set to expire after 20 minutes. So, in theory, it should never happen. Also, since my last Zope restart (3hr 30min), I've had 57 users let their authentication timeout and then access the site (forcing another login), and still no errors with session. (It's still using FileStorage). 1 - After having the system run for 45 minutes, I had 8 similar errors. Some were on different web pages and some had different keys that were causing the error. 2 - I have also traced through Z2.log and followed the same path through the web site that produced one of the errors and I did not get an error. 3 - Since changing this server to use FileStorage (1hr 39min ago), there has not been a single error. Is there anything I can do to help here? A small reproducible test case would help if you still believe this error is not in your own application. I'm sure you know by now that this is easier said than done. Over the past few weeks, I've probably put close to 30 hours into just trying to reproduce (on demand) these session errors. I'll probably want to look at the test rig you and Michael have been using. ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Conflict errors on BDBMinimal storage
On Mon, May 17, 2004 at 01:38:15PM -0400, Chris McDonough wrote: On Mon, 2004-05-17 at 13:06, Gerhard Schmidt wrote: BDBMininalStorage is no longer maintained and doesn't do any conflict resolution. Why this. I was told some time ago we should use BDBMinimalStorage to fix the conflict problems. When was that and who told you so? Even though you'll need to pack frequently, you might want to try a FileStorage instead. There is no difference between BDB Storage and Filestorage regarding the conflicts. OK. Regardless, using ZEO will presumably always generate more conflict errors than using a local ZODB because the commit time is longer, and thus there's a better chance that two threads will be committing at the same time. The bottom line is that there is no way of getting an shared session running. It works doesn't it? You just get conflict errors, right? Conflict errors are normal at a certain level. No It doesn`t. At the Moment we run with single sessions on each frontend we have rewitten most ob the code using sessions to store ther information in cockies. But that´s not realy a good solution. Why isn´t there an implementation of temporary storage for ZEO There is, just use the tempstorage package from Zope. I didn´t find a way to use Tempstorage in zeo. temporarystorage name temporary storage for sessioning /temporarystorage didn´t work. How do i Set up a temporarystorage in zeo. I dont care about conflit resolution. Its only session data. For better or worse, because it is based on ZODB, the implementation makes you need to care. Than maybe the sessioncode should be redesigned not to use ZODB as storage but store ther information in some different system maybe a RDMB or some other system able to handle the kind of usage. I simply need a shared session storage. But as Far as i can see the is no way of doing this. I don't understand. You seem to have it working. Is performance the issue? The Performance decrase is there but the Problem is far deeper. If an read conflict isn´t resolved after three retries an Exception is raised an the request is aborted. And thats the real problem. Are we the only site out there running Plone2 (Placeless transaltion Service) at that scale. How do others handle there Session AFAIK Zope is the only appserver that even makes an attempt at allowing session data to be shared between servers like this. Most other appservers require you to set up session affinity in your load balancer and have a session database on each appserver. and nealy every system failed to do so. How do you implement this session affinity. Based on source IP. At the moment every large ISP uses multiple Proxyserver so every request comes from a different IP so the affinity doesn´t work for them. The Proxyserver provided by our Computing Center is also split on at least 8 Hosts, so everyone using this proxy would lose session infomation, getting very strange effekts. Bye Estartu -- Gerhard Schmidt| Nick : estartu IRC : Estartu | Fischbachweg 3 || PGP Public Key 86856 Hiltenfingen | [EMAIL PROTECTED] | auf Anfrage/ Tel: 08232 77 36 4 ||on request Fax: 08232 77 36 3 || pgpsBoo3UooYb.pgp Description: PGP signature ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Conflict errors on BDBMinimal storage
On Mon, 2004-05-17 at 14:57, Gerhard Schmidt wrote: Why isnt there an implementation of temporary storage for ZEO There is, just use the tempstorage package from Zope. I didnt find a way to use Tempstorage in zeo. temporarystorage name temporary storage for sessioning /temporarystorage didnt work. How do i Set up a temporarystorage in zeo. Good question. ;-) It involves changing the ZEO configuration machinery to recognize the tempstorage package. The steps to do this I couldn't tell you without actually doing it, however. I dont care about conflit resolution. Its only session data. For better or worse, because it is based on ZODB, the implementation makes you need to care. Than maybe the sessioncode should be redesigned not to use ZODB as storage but store ther information in some different system maybe a RDMB or some other system able to handle the kind of usage. Zope doesn't ship with a useful and maintained relational database. You can implement your own SQL-based TransientObjectContainer using the interfaces defined in Transience/Interfaces.py if you wish, or use Anthony Baxter's SQLSession product. It would be nice if there was an alternate implementation that used a relational database like this. I simply need a shared session storage. But as Far as i can see the is no way of doing this. Of course there is, it's just not built in. ;-) I don't understand. You seem to have it working. Is performance the issue? The Performance decrase is there but the Problem is far deeper. If an read conflict isnt resolved after three retries an Exception is raised an the request is aborted. And thats the real problem. I see. I'm not sure switching to TemporaryStorage will help much. I would first try using a FileStorage and make sure to put the Zope lib/python directory on the ZEO server's Python's PYTHONPATH (to allow for conflict resolution under ZEO, ZEO must be able to import the Products package and subpackages therein). I don't know if this will help very much, but it's worth a shot. Are we the only site out there running Plone2 (Placeless transaltion Service) at that scale. How do others handle there Session AFAIK Zope is the only appserver that even makes an attempt at allowing session data to be shared between servers like this. Most other appservers require you to set up session affinity in your load balancer and have a session database on each appserver. and nealy every system failed to do so. How do you implement this session affinity. Based on source IP. At the moment every large ISP uses multiple Proxyserver so every request comes from a different IP so the affinity doesnt work for them. The Proxyserver provided by our Computing Center is also split on at least 8 Hosts, so everyone using this proxy would lose session infomation, getting very strange effekts. Yes. It's a hard problem. - C ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: [ZODB-Dev] Re: BTrees strangeness (was Zope 2.X BIG Session problems - blocker - our site dies - need help of experience Zope developer, please)
On Mon, 2004-05-17 at 14:58, Steve Jibson wrote: We're using ExUserFolder for authentication. We have it set to log users out after 10 minutes. We have sessions set to expire after 20 minutes. So, in theory, it should never happen. Are you sure that their auth really does time out after exactly 10 minutes? 10 minutes of inactivity? Is it based on a cookie timeout? Does it reset the cookie on every request? This relies on assuming all browsers respect the cookie timeout adequately at that resolution, as well. I don't know how well various browsers do this. Also, since my last Zope restart (3hr 30min), I've had 57 users let their authentication timeout and then access the site (forcing another login), and still no errors with session. (It's still using FileStorage). It might be pedantic, but I still think relying on the auth timeout behavior is wrong. Instead, of populating initial session data at login time, a bit of code called from your main template should probably do something like if not context.session_data_manager.hasSessionData(): ... populate session data based on auth info Or maybe an access rule that does the same thing (might be tricky though, I haven't tested that well). The right thing to do here might be to use an auth product that stores auth info in a session; that would isolate the problem nicely because there wouldn't be competing timeouts. Apparently someone has coded one up as SessionCrumbler somwhere. The fact that it works under FileStorage is curious, but might be a coincidence too. Lots of machinery here. ;-) A small reproducible test case would help if you still believe this error is not in your own application. I'm sure you know by now that this is easier said than done. Sure. I've spent god knows how long doing it. ;-) Over the past few weeks, I've probably put close to 30 hours into just trying to reproduce (on demand) these session errors. I'll probably want to look at the test rig you and Michael have been using. Yes, although that doesn't do any data validity checks, AFAIK. It would be interesting to try to extend it to do so. You might also want to consider running Zope with the Z_TOC_DEBUG envvar set to true. This spews out a bunch of messages to the error log for each session access. Coupled with a log message noting when someone logs in *and their browser id*, this could provide clues. - C ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: [ZODB-Dev] Re: BTrees strangeness (was Zope 2.X BIG Session problems - blocker - our site dies - need help of experience Zope developer, please)
On 18/05/2004, at 5:42 AM, Chris McDonough wrote: On Mon, 2004-05-17 at 13:06, Steve Jibson wrote: Also of note: 1 - After having the system run for 45 minutes, I had 8 similar errors. Some were on different web pages and some had different keys that were causing the error. 2 - I have also traced through Z2.log and followed the same path through the web site that produced one of the errors and I did not get an error. 3 - Since changing this server to use FileStorage (1hr 39min ago), there has not been a single error. Is there anything I can do to help here? A small reproducible test case would help if you still believe this error is not in your own application. Looks like session data can expire prematurely. See attached files for small changes to the test rig that reports cases where context.session_data_manager.hasSessionData() is False. Michael SessionRigExtensions.py Description: Binary data TestSessionRig.py Description: Binary data ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: [ZODB-Dev] Re: BTrees strangeness (was Zope 2.X BIG Session problems - blocker - our site dies - need help of experience Zope developer, please)
On Mon, 2004-05-17 at 17:52, Michael Dunstan wrote: Looks like session data can expire prematurely. See attached files for small changes to the test rig that reports cases where context.session_data_manager.hasSessionData() is False. Michael I think you're right, good eye! The symptoms aren't consistent with what Steve is reporting, however (where the problem appears when using TemporaryStorage but not FileStorage), so I this may be a distinct issue. I am looking in to it now. It feels like an off-by-one error. - C ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: [ZODB-Dev] Re: BTrees strangeness (was Zope 2.X BIG Session problems - blocker - our site dies - need help of experience Zope developer, please)
There indeed is a minor off-by-one error: it manifests itself as sessions timing out at most 20 seconds early. But there is also a deeper issue which involves the fact that a session data object is not properly removed from an older bucket when it moves due to being accessed in a later timeslice; the symptom only appears when a browser id is reused to start a session after it was used to start an older one that had timed out normally. I've got almost no clue why this happens at this point, but I'm working on it. Ugh. This is almost certainly what Steve is experiencing. - C On Mon, 2004-05-17 at 20:26, Chris McDonough wrote: On Mon, 2004-05-17 at 17:52, Michael Dunstan wrote: Looks like session data can expire prematurely. See attached files for small changes to the test rig that reports cases where context.session_data_manager.hasSessionData() is False. Michael I think you're right, good eye! The symptoms aren't consistent with what Steve is reporting, however (where the problem appears when using TemporaryStorage but not FileStorage), so I this may be a distinct issue. I am looking in to it now. It feels like an off-by-one error. - C ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope ) ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
