Re: [Zope-dev] Re: Conflict errors on BDBMinimal storage
Tres Seaver wrote: I recently reworked Anthony Baxter's SQLSessions product for a consulting customer <...snip...> What I would like to do is revisit the effort, using the new PluginRegistry code released as part of the PluggableAuthService release; I am, however, pretty well booked up for the next few weeks. I would be glad to advise anybody who felt called to take on such an effort, however. <...snip...> I actually need this whithin the next couple of weeks - only one caveat: I'd have to do a serious read-up on sessioning, which I don't understand at all and the PluggableAuthService product, which I never heard of until I read about it. We have something similar to SQL-Sessions. However, our security machinery is built on top of Zopes own (in parallell not replacing it). It is loosely modelled around a mix of Zope's way of doing things with a hint of XUF (we couldn't use XUF at the time, because it wasn't stable enough back then). One of the goals wiht the current rewrite of one of our apps is to go back doing things the Zope-way, by using Zope's own security mechanisms. We have working solutions for authentication, but the sessioning part remains - especically since we, like in Steve's setup expire sessions on the server rather than on the browser end. So... any hints on where to begin reading for the completely-lost newbie? /dario -- -- --- Dario Lopez-KÃsten, IT Systems & Services Chalmers University of Tech. ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: Conflict errors on BDBMinimal storage
On Tue, 2004-05-18 at 16:36, Chris McDonough wrote: > In my testing, I'm able to see that naive sessioning applications (ones > that hit the session on every request, whether they really need to or > not) seem to fail down due to the "abort after 3 conflicts in a row" > hardcoded policy under extremely high load (~ 40-60 req/sec or so). > This will happen regardless of what code I put in the sessioning > machinery; attempting to prevent it there is a lost cause: it gets a bit > like trying to prove the Heisenberg Uncertainty Principal or trying to > predict the future. Just as a followup: when I observed this, it was because I had all of my extra debugging code turned on. True to the Uncertainty Principle, the execution of the debugging code caused a raft of extra conflicts. Turning off the debugging code leads to a much rosier scenario. In cases where it was raising conflicts to the end user, it now operates without doing so. I still need to do some really hardcore testing of the thing to see exactly where it buckles, but I can no longer provoke it trivially. > As a result, I am coming to believe that along with the "errors as part > of main" transaction patch for 2.7.1, I should also make the > retry-on-conflict-error policy pluggable for those who really > desperately need to slow their Zope systems to a conflict-induced > crawl. They can bump it up to 100 as far as I care and if it works for > them, great. The more selective people can wait around for a different > (non-ZODB-based) sessioning implementation or change their code to not > pound the snot out of the sessioning machinery unnecessarily. Michael Dunstan also had another good idea. At least one knob exists in the source code to increase/decrease the "resolution" of a part of the sessioning/transience code. Basically this knob allows you to choose a tradeoff between the frequency of a class of writes done by the session code and the "accuracy" of the session "finalization". Setting the knob higher will decrease the aggregate number of database writes that would need to be done over any given period of time in order to keep sessions "current", which might lead to fewer conflicts. On the flip side, it would mean that the session might not be "finalized" on time (sessions are "finalized" when they are garbage collected; sessions are only allowed to be garbage collected every "n" seconds where n is the knob's setting). This would typically be meaningful only to people who use the "Script to call at session deletion time" feature of sessions; that script might be called much later than you think it might be called if you set this value high enough. There's currently no guarantee that it will be called exactly when a session expires even with the default value, but setting the resolution higher will decrease that probability even further. Since that feature isn't used very much (at least I don't think it is), I think this is an ok tradeoff for a lot of people. I'd need to make sure that it actually helps something before doing so, but assuming it did, it would be put into the config file for 2.7.1 as something like "session-resolution-seconds" or somesuch. For those of you with conflict problems now, this knob is the PERIOD variable in Transience.py... you might be able to get away with setting it higher now to see if that helps anything, especially those people trying to use sessions+ZEO. It's preset to 20, which means 20 seconds. Try 120 instead, or something around there. Just don't set it to a number higher than the timeout value for the TOC (which is in minutes, remember to multiply it by 60) or all sorts of insanity will happen. You'll need to restart Zope and recreate your transient object container after doing this, as doing this will effectively invalidate all data previously in the container and using old data left over from running it under the default PERIOD value might make it go a little insane too. > As far as I'm concerned, the new Transience stuff is looking really good > for the common case. I haven't been able to reproduce any of the > corruption issues that happened with the older implementation. This is still true (it's been almost an hour since I posted it, I think that's a new record!) - c ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: Conflict errors on BDBMinimal storage
On 19/05/2004, at 8:36 AM, Chris McDonough wrote: As a result, I am coming to believe that along with the "errors as part of main" transaction patch for 2.7.1, I should also make the retry-on-conflict-error policy pluggable for those who really desperately need to slow their Zope systems to a conflict-induced crawl. They can bump it up to 100 as far as I care and if it works for them, great. Or this could be used to avoid retries altogether. And at the very least it would advertise the fact that zope does retry requests. Michael. ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: [ZODB-Dev] Re: BTrees strangeness (was Zope 2.X BIG Session problems - blocker - our site dies - need help of experience Zope developer, please)
On 18/05/2004, at 10:03 PM, Andrea Patuzzo wrote: Hi, dear developers: We are now randomly getting this kind of error: [...] File "/usr/local/Zope270CVS/lib/python/Products/Transience/Transience.py", line 341, in __setitem__ current_bucket = self._data[current_ts] KeyError: 1084872640 This one was with the latest publish.py / startup.py patch and the new Transience.py implementation, on a yesterday's Zope-2_7-branch checkout. New Zope instance with FileStorage does not solve the problem. Same happens with the patches applied to the official Zope-2.7.0 release. Note: once the error happens, zope has to be restarted to function again. Before applying patches we were getting "get" errors (instead of "__setitem__") and we just had to close browser to get back to work. Here we have to restart Zope (with TemporaryStorage) or delete Sessions.fs and restart (with FileStorage). My guess at the moment is that somewhere a ConflictError is being swallowed. There are a few things that can be responsible for this: - Use of hasattr on Persistent objects. - try/except blocks that don't raise ConflictErrors. From your trace that looks like a Plone site. A very quick glance through Plone and its friends indicates that there may be some chance of either of the above causing some problems. Although Plone 2.0.1 includes the following fix: - Fixed bare 'except:' statements in Plone so we don't swallow ZODB ConflictErrors. Mostly by adding 'except ConflictError: raise'. [stefan] Someone please shoot me down and claim that Plone and friends all do the right thing. ;-) Michael ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Threads dying on zope 2.7.0 on one Zeo setup, ZServer exception
Heimo Laukkanen wrote at 2004-5-18 14:10 +0300:
> ...
>At the moment with testing setup under load test ZServer dies - leaving
>all but one thread at almost 0% cpu and that one thread working in high
>loads of 90%. Testing similar zeo-setup with development instances do
>not produce this.
Provided, you work under Linux, you can try to attach
to the process with GDB and analyse the thread behaviour.
I use the functions "ps" (Print String) and "pfr" (Print FRame)
during such analysis. They are defined by:
def ps
x/s ({PyStringObject}$arg0)->ob_sval
end
def pfr
ps f->f_code->co_filename
ps f->f_code->co_name
p f->f_lineno
end
You can apply "ps" to an string object.
You can call "pfr" in an "eval_frame" frame context.
--
Dieter
___
Zope-Dev maillist - [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Conflict errors on BDBMinimal storage
Gerhard Schmidt wrote at 2004-5-17 21:01 +0200: > ... >> However, due to a ZEO weakness, its conflict resolution >> currently does not work (ZEO no longer supports "INSTANCE_HOME"). > >Is that a bug or intetional. I expect it to be a bug. I plan to fix it soon -- but currently, I have lots of things to do. May take some months... -- Dieter ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: Conflict errors on BDBMinimal storage
robert rottermann wrote: On Tue, 2004-05-18 at 14:15, Tres Seaver wrote: I recently reworked Anthony Baxter's SQLSessions product for a consulting customer, essentially modernizing the code and making it compliant with the new Sessions interface (I dropped the bits which do the work currently done by BrowserIDManager). It took me about six hours or so, including writing unit tests. Unfortunately, I don't have access to that code, due to the nature of the consulting arrangement. What I would like to do is revisit the effort, using the new PluginRegistry code released as part of the PluggableAuthService release; I am, however, pretty well booked up for the next few weeks. I would be glad to advise anybody who felt called to take on such an effort, however. I do not know whether I understand this email correctly. So mabe I am off topic. We would like to use thre PluggableAuthService to replace a userfolder that authenticates against a an email server (calling a web service to do so) and then caches authentification and other user info in a session. So we would like to dig into what both of you seem not to have time to do. How can we get at PluggableAuthService? I did not find it anywhere. The PluggableAuthService product is available in CVS: http://cvs.zope.org/Products/PluggableAuthService It depends on the PluginRegistry product: http://cvs.zope.org/Products/PluginRegistry Both are "releasable", but not yet released (I just reminded Zac to cut the release tarballs). PAS has a SessionAuthHelper plugin which should serve as a model for the part of your problem which revolves around the session; you would need to write an "authenticator" plugin which validated credentials against your web service. And we would of course gladly accept Tres offer for advise. I was referring to using the PluginRegistry, which is a lower-level utility, to (re)write the SQLSessions product, which needs to be "pluggable" in order to accomodate different backends. The PIR is a Zope 2 analog to the Zope 3 "local utility" service: it stores a perstent map of the plugins registered / activated for a set of application-defined plugin interfaces. Tres. -- === Tres Seaver[EMAIL PROTECTED] Zope Corporation "Zope Dealers" http://www.zope.com ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: Conflict errors on BDBMinimal storage
On Tue, 2004-05-18 at 14:58, Chris McDonough wrote: > On Tue, 2004-05-18 at 14:15, Tres Seaver wrote: > > I recently reworked Anthony Baxter's SQLSessions product for a > > consulting customer, essentially modernizing the code and making it > > compliant with the new Sessions interface (I dropped the bits which do > > the work currently done by BrowserIDManager). It took me about six > > hours or so, including writing unit tests. Unfortunately, I don't have > > access to that code, due to the nature of the consulting arrangement. > > > > What I would like to do is revisit the effort, using the new > > PluginRegistry code released as part of the PluggableAuthService > > release; I am, however, pretty well booked up for the next few weeks. > > I would be glad to advise anybody who felt called to take on such an > > effort, however. > > I would also like to see this happen. I don't have the time to help > implement this right now, but I may be able to work it in to an ongoing > engagement at some point in the next few months. In my testing, I'm able to see that naive sessioning applications (ones that hit the session on every request, whether they really need to or not) seem to fail down due to the "abort after 3 conflicts in a row" hardcoded policy under extremely high load (~ 40-60 req/sec or so). This will happen regardless of what code I put in the sessioning machinery; attempting to prevent it there is a lost cause: it gets a bit like trying to prove the Heisenberg Uncertainty Principal or trying to predict the future. As a result, I am coming to believe that along with the "errors as part of main" transaction patch for 2.7.1, I should also make the retry-on-conflict-error policy pluggable for those who really desperately need to slow their Zope systems to a conflict-induced crawl. They can bump it up to 100 as far as I care and if it works for them, great. The more selective people can wait around for a different (non-ZODB-based) sessioning implementation or change their code to not pound the snot out of the sessioning machinery unnecessarily. As far as I'm concerned, the new Transience stuff is looking really good for the common case. I haven't been able to reproduce any of the corruption issues that happened with the older implementation. - C ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: Conflict errors on BDBMinimal storage
Chris McDonough wrote: On Tue, 2004-05-18 at 14:15, Tres Seaver wrote: I recently reworked Anthony Baxter's SQLSessions product for a consulting customer, essentially modernizing the code and making it compliant with the new Sessions interface (I dropped the bits which do the work currently done by BrowserIDManager). It took me about six hours or so, including writing unit tests. Unfortunately, I don't have access to that code, due to the nature of the consulting arrangement. What I would like to do is revisit the effort, using the new PluginRegistry code released as part of the PluggableAuthService release; I am, however, pretty well booked up for the next few weeks. I would be glad to advise anybody who felt called to take on such an effort, however. I would also like to see this happen. I don't have the time to help implement this right now, but I may be able to work it in to an ongoing engagement at some point in the next few months. - C ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope ) I do not know whether I understand this email correctly. So mabe I am off topic. We would like to use thre PluggableAuthService to replace a userfolder that authenticates against a an email server (calling a web service to do so) and then caches authentification and other user info in a session. So we would like to dig into what both of you seem not to have time to do. How can we get at PluggableAuthService? I did not find it anywhere. And we would of course gladly accept Tres offer for advise. Robert ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: Conflict errors on BDBMinimal storage
On Tue, 2004-05-18 at 14:15, Tres Seaver wrote: > I recently reworked Anthony Baxter's SQLSessions product for a > consulting customer, essentially modernizing the code and making it > compliant with the new Sessions interface (I dropped the bits which do > the work currently done by BrowserIDManager). It took me about six > hours or so, including writing unit tests. Unfortunately, I don't have > access to that code, due to the nature of the consulting arrangement. > > What I would like to do is revisit the effort, using the new > PluginRegistry code released as part of the PluggableAuthService > release; I am, however, pretty well booked up for the next few weeks. > I would be glad to advise anybody who felt called to take on such an > effort, however. I would also like to see this happen. I don't have the time to help implement this right now, but I may be able to work it in to an ongoing engagement at some point in the next few months. - C ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: [ZODB-Dev] Re: BTrees strangeness (was Zope 2.X BIG Session problems - blocker - our site dies - need help of experience Zope developer, please)
On Tue, 2004-05-18 at 13:11, Steve Jibson wrote: > Well, after much log reading, I have found that the KeyError we got last > night was OUR fault. Thanks for reporting this, it's reasurring. I was fixing to go insane. ;-) > I will fix the problem with our app, then I want to change back to > TemporaryStorage and watch the system some more. I'll keep you posted. Great, thanks. > Here's the brief explanation of our problem (you can skip it if you like): > > A user logged in and did some stuff then left his browser for almost an > hour. When he returned and tried to do more stuff, he was no longer in > the ExUserFolder's credential cache and his session had expired. He was > forced to log in again. Upon supplying his ID and password, he was sent > to the "loginSuccess" page. This is the one that calls our method to > set up his user session. The Z2.log shows a 302 result code on this > page. Ooh, that's bad. I guess that means that your users might not be logging in as whom they think they're logging in as... CookieCrumbler handles this by injecting Pragma: No-Cache and the HTTP/1.1 equivalent into the header (currently on *every* request, not so great actually). > His browser had the "loginSuccess" page in cache, so it did not > request it again and his session was never re-created. > > Score one for Chris's suggestion on how we should be setting up the > user's session. For now, however, I think I'll just add the > please-don't-cache-me header stuff to the RESPONSE. Right... although caching+auth is kind of tricky. I wouldn't be surprised if you found other places where caching is defeating some other intentions you had. If the site is meant to be accessed only by authenticated users, I'd be tempted to arrange for the no-cache headers to be injected by the main template itself. I don't understand your setup 100%, although it sounds interesting. By your description of it sounds like you're implementing a bit of your own security machinery (a user can either be an "admin" or a normal user, depending on some setting he chooses at login time, even though it's physically only one account). But maybe an alternate way to implement the machinery to do this is to do the following: - Associate "member data" with each user. This is built in to CMF, or you can roll your own if you're not using CMF (maybe by manually using a Transient Object Container in a FileStorage with the timeout set to 0... it basically then just becomes a big dictionary). Store user "preferences" there. Give each user a default set of preferences for first-time login. - Allow the user to log in with just a username and password (don't require him to enter his profile data on each login). - In the session "add notify" script, copy his prefs into his session. Your existing code will then work normally. - In the session "delete notify" script, copy the prefs data in the session back into the member data. Just an idea... - C ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: Conflict errors on BDBMinimal storage
Chris McDonough wrote: Hi Dr. Wagner, thanks for the hint with minimal storage. however the same problem occurs with filestorage and is even worse. we know very well, that we cannot avoid the appearance of conflict errors in particular with zeo, and we understand what causes them. what wonders us is, that we find some contributions about this errors in the mailing lists, but no one seems to have a real problem with it. but we have! we are not worried about some errors appearing in a log, but we are using plone2 witch currently 30 concurrent authenticated users, so that this error occurs permanently, thus decreasing system performance due to several retries and moreover leading to very frequent nasty 'POS.Exception Errors' in the browser, whenever four successive tries fail (almost every 100. request). this is not acceptable, and i wonder wheter there are any plone sites with more than a handful of authenticated users that experience the same problem. we found that one (there are more) product responsible for it is the placeless translation service, which heavily writes to the session. Are you using ZEO for your session storage or are you using a local storage? (You hint at it above but you don't actually say one way or the other.) so we desperately need a solution for this problem, maybe uncouple the session of the locking mechanism or whatever, and this should be a problem that should be addressed by the core team. because this is a strong reason (and so far the only one) to avoid using zope/plone in a large user environment. I also need reliable sessioning for my own customers, so I am working towards fixing the issues I know about. But the issues I know about may not be the same issues as those that you want fixed. You can ensure your issues are addressed by devoting resources towards the problem, be that time+expertise or money or both. I recently reworked Anthony Baxter's SQLSessions product for a consulting customer, essentially modernizing the code and making it compliant with the new Sessions interface (I dropped the bits which do the work currently done by BrowserIDManager). It took me about six hours or so, including writing unit tests. Unfortunately, I don't have access to that code, due to the nature of the consulting arrangement. What I would like to do is revisit the effort, using the new PluginRegistry code released as part of the PluggableAuthService release; I am, however, pretty well booked up for the next few weeks. I would be glad to advise anybody who felt called to take on such an effort, however. Tres. -- === Tres Seaver[EMAIL PROTECTED] Zope Corporation "Zope Dealers" http://www.zope.com ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: [ZODB-Dev] Re: BTrees strangeness (was Zope 2.X BIG Session problems - blocker - our site dies - need help of experience Zope developer, please)
Well, after much log reading, I have found that the KeyError we got last night was OUR fault. I will fix the problem with our app, then I want to change back to TemporaryStorage and watch the system some more. I'll keep you posted. Here's the brief explanation of our problem (you can skip it if you like): A user logged in and did some stuff then left his browser for almost an hour. When he returned and tried to do more stuff, he was no longer in the ExUserFolder's credential cache and his session had expired. He was forced to log in again. Upon supplying his ID and password, he was sent to the "loginSuccess" page. This is the one that calls our method to set up his user session. The Z2.log shows a 302 result code on this page. His browser had the "loginSuccess" page in cache, so it did not request it again and his session was never re-created. Score one for Chris's suggestion on how we should be setting up the user's session. For now, however, I think I'll just add the please-don't-cache-me header stuff to the RESPONSE. Steve Jibson wrote: Good morning. I just got in and checked on my customer's system. In the past 22 1/2 hours they've had 15000 page hits and last night at about 9:30, ONE person got a KeyError. Actually, this same person got twenty KeyErrors over a period of about 45 seconds. I'm downloading their log files now and plan to spend some time this morning going through them. Anyway, it appears that I was wrong when I said that the problem doesn't show up when I use FileStorage (although it does seem to happen less frequently -- but who can be sure of anything at this point?). In answer to your questions earlier, Chris, we set up the user session at login time because we make the user answer some questions at login time that determine which portions of the interface to present to him/her. For example, using the same login id and password, a user may choose to login as an administrator or as a normal user. We store this choice and other info based on this choice in the session. Also, we don't rely on the browser to time out the authentication cookie. Once a user authenticates with ExUserFolder, ExUserFolder keeps their credentials in a cache until they have been inactive for 10 minutes (the timer resets with each cache hit). If their credentials are not in the cache, rather than looking them up again, the user is logged out and must re-authenticate. It seems like a reasonable way to handle logins and sessions. In addition to going through log files, I will spend some more time today making sure we're not doing something stupid in our app. Thanks again (to Chris, Michael, Alex and everyone else who has lost sleep over this session stuff). I'll keep you posted on any new information I find. Steve ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: [ZODB-Dev] Re: BTrees strangeness (was Zope 2.X BIG Session problems - blocker - our site dies - need help of experience Zope developer, please)
Andrea Patuzzo wrote: We are now randomly getting this kind of error: Note: once the error happens, zope has to be restarted to function again. Before applying patches we were getting "get" errors (instead of "__setitem__") and we just had to close browser to get back to work. That's right. Under Plone 2.0 and Zope 2.7 I had the same kind of problem. But when I restarted the browser it would be solved. That could point in the direction of a session id being somehow involved. -- hilsen/regards Max M, Denmark http://www.mxm.dk/ IT's Mad Science ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: [ZODB-Dev] Re: BTrees strangeness (was Zope 2.X BIG Session problems - blocker - our site dies - need help of experience Zope developer, please)
Good morning. I just got in and checked on my customer's system. In the past 22 1/2 hours they've had 15000 page hits and last night at about 9:30, ONE person got a KeyError. Actually, this same person got twenty KeyErrors over a period of about 45 seconds. I'm downloading their log files now and plan to spend some time this morning going through them. Anyway, it appears that I was wrong when I said that the problem doesn't show up when I use FileStorage (although it does seem to happen less frequently -- but who can be sure of anything at this point?). In answer to your questions earlier, Chris, we set up the user session at login time because we make the user answer some questions at login time that determine which portions of the interface to present to him/her. For example, using the same login id and password, a user may choose to login as an administrator or as a normal user. We store this choice and other info based on this choice in the session. Also, we don't rely on the browser to time out the authentication cookie. Once a user authenticates with ExUserFolder, ExUserFolder keeps their credentials in a cache until they have been inactive for 10 minutes (the timer resets with each cache hit). If their credentials are not in the cache, rather than looking them up again, the user is logged out and must re-authenticate. It seems like a reasonable way to handle logins and sessions. In addition to going through log files, I will spend some more time today making sure we're not doing something stupid in our app. Thanks again (to Chris, Michael, Alex and everyone else who has lost sleep over this session stuff). I'll keep you posted on any new information I find. Steve Chris McDonough wrote: On Mon, 2004-05-17 at 23:08, Chris McDonough wrote: There indeed is a minor off-by-one error: it manifests itself as sessions timing out at most 20 seconds early. But there is also a deeper issue which involves the fact that a session data object is not properly removed from an older bucket when it "moves" due to being accessed in a later timeslice; the symptom only appears when a browser id is "reused" to start a session after it was used to start an older one that had timed out normally. I've got almost no clue why this happens at this point, but I'm working on it. Ugh. This is almost certainly what Steve is experiencing. I take that back. Actually, I think I was just reading the test results and debug output wrong. It appears to be operating normally except for the off-by-one problem (which is minor). I need to jack up the tests to do some comparisons of data values; currently I'm just testing to ensure that *something* is in the session.. I need to test if the "right" thing is in the session over time. - C ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope ) ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: [ZODB-Dev] Re: BTrees strangeness (was Zope 2.X BIG Session problems - blocker - our site dies - need help of experience Zope developer, please)
Hi, dear developers:
We are now randomly getting this kind of error:
2004-05-18T11:30:41 ERROR(200) SiteError
http://XXX/prefs_error_log_form
Traceback (most recent call last):
File "/usr/local/Zope270CVS/lib/python/ZPublisher/Publish.py", line 100,
in publish
request, bind=1)
File "/usr/local/Zope270CVS/lib/python/ZPublisher/mapply.py", line 88, in
mapply
if debug is not None: return debug(object,args,context)
File "/usr/local/Zope270CVS/lib/python/ZPublisher/Publish.py", line 40, in
call_object
result=apply(object,args) # Type s to step into published object.
File "/usr/local/Zope270CVS/lib/python/Shared/DC/Scripts/Bindings.py",
line 306, in __call__
return self._bindAndExec(args, kw, None)
File "/usr/local/Zope270CVS/lib/python/Shared/DC/Scripts/Bindings.py",
line 343, in _bindAndExec
return self._exec(bound_data, args, kw)
File "/zope_instances/patuzzo.ch/Products/CMFCore/FSPageTemplate.py", line
191, in _exec
result = self.pt_render(extra_context=bound_names)
File "/zope_instances/patuzzo.ch/Products/CMFCore/FSPageTemplate.py", line
124, in pt_render
result = FSPageTemplate.inheritedAttribute('pt_render')(
File
"/usr/local/Zope270CVS/lib/python/Products/PageTemplates/PageTemplate.py",
line 96, in pt_render
tal=not source, strictinsert=0)()
File "/usr/local/Zope270CVS/lib/python/TAL/TALInterpreter.py", line 189,
in __call__
self.interpret(self.program)
File "/usr/local/Zope270CVS/lib/python/TAL/TALInterpreter.py", line 233,
in interpret
handlers[opcode](self, args)
File "/usr/local/Zope270CVS/lib/python/TAL/TALInterpreter.py", line 663,
in do_useMacro
self.interpret(macro)
File "/usr/local/Zope270CVS/lib/python/TAL/TALInterpreter.py", line 233,
in interpret
handlers[opcode](self, args)
File "/usr/local/Zope270CVS/lib/python/TAL/TALInterpreter.py", line 408,
in do_optTag_tal
self.do_optTag(stuff)
File "/usr/local/Zope270CVS/lib/python/TAL/TALInterpreter.py", line 393,
in do_optTag
return self.no_tag(start, program)
File "/usr/local/Zope270CVS/lib/python/TAL/TALInterpreter.py", line 388,
in no_tag
self.interpret(program)
File "/usr/local/Zope270CVS/lib/python/TAL/TALInterpreter.py", line 233,
in interpret
handlers[opcode](self, args)
File "/usr/local/Zope270CVS/lib/python/TAL/TALInterpreter.py", line 663,
in do_useMacro
self.interpret(macro)
File "/usr/local/Zope270CVS/lib/python/TAL/TALInterpreter.py", line 233,
in interpret
handlers[opcode](self, args)
File "/usr/local/Zope270CVS/lib/python/TAL/TALInterpreter.py", line 408,
in do_optTag_tal
self.do_optTag(stuff)
File "/usr/local/Zope270CVS/lib/python/TAL/TALInterpreter.py", line 393,
in do_optTag
return self.no_tag(start, program)
File "/usr/local/Zope270CVS/lib/python/TAL/TALInterpreter.py", line 388,
in no_tag
self.interpret(program)
File "/usr/local/Zope270CVS/lib/python/TAL/TALInterpreter.py", line 233,
in interpret
handlers[opcode](self, args)
File "/usr/local/Zope270CVS/lib/python/TAL/TALInterpreter.py", line 663,
in do_useMacro
self.interpret(macro)
File "/usr/local/Zope270CVS/lib/python/TAL/TALInterpreter.py", line 233,
in interpret
handlers[opcode](self, args)
File "/usr/local/Zope270CVS/lib/python/TAL/TALInterpreter.py", line 629,
in do_condition
self.interpret(block)
File "/usr/local/Zope270CVS/lib/python/TAL/TALInterpreter.py", line 233,
in interpret
handlers[opcode](self, args)
File "/usr/local/Zope270CVS/lib/python/TAL/TALInterpreter.py", line 406,
in do_optTag_tal
self.no_tag(stuff[-2], stuff[-1])
File "/usr/local/Zope270CVS/lib/python/TAL/TALInterpreter.py", line 388,
in no_tag
self.interpret(program)
File "/usr/local/Zope270CVS/lib/python/TAL/TALInterpreter.py", line 233,
in interpret
handlers[opcode](self, args)
File "/usr/local/Zope270CVS/lib/python/TAL/TALInterpreter.py", line 605,
in do_loop_tal
self.interpret(block)
File "/usr/local/Zope270CVS/lib/python/TAL/TALInterpreter.py", line 233,
in interpret
handlers[opcode](self, args)
File "/usr/local/Zope270CVS/lib/python/TAL/TALInterpreter.py", line 290,
in do_startTag
ok, name, s = attrAction(self, item)
File "/usr/local/Zope270CVS/lib/python/TAL/TALInterpreter.py", line 369,
in attrAction_tal
translated = self.translate(msgid or value, value, {})
File "/usr/local/Zope270CVS/lib/python/TAL/TALInterpreter.py", line 615,
in translate
msgid, i18ndict, default=default)
File "/usr/local/Zope270CVS/lib/python/Products/PageTemplates/TALES.py",
line 264, in translate
target_language=target_language)
File
"/zope_instances/patuzzo.ch/Products/PlacelessTranslationService/PlacelessTr
anslationService.py", line 109, in translate
return service.translate(domain, msgid, mapping, context,
target_language, default)
File
"/zope_instances/patuzzo.ch/Products/PlacelessTranslationService/PlacelessTr
anslationService.py", line 423, in translate
catalogs = self.getCatal
[Zope-dev] Threads dying on zope 2.7.0 on one Zeo setup, ZServer exception
Hi ya, I noticed some weird bahivour that was bringing a setup down on our servers on testing installation. In our setup we create development sandboxes with makefile and in similar fashion also testing/production servers. Our testing/production setup has a little bit different config files where from standard config all the commented out lines have been removed. Also compared to development instances the instance home is in other place. Other than that setups are same. At the moment with testing setup under load test ZServer dies - leaving all but one thread at almost 0% cpu and that one thread working in high loads of 90%. Testing similar zeo-setup with development instances do not produce this. We are a bit scared on this, since we do not have clue on what causes following behaviour - and eventhough we see that we can get it to work, we would like to have an idea on what might be behind this - so that it does not bite our butt again. Below is log snip and traceback of the exceptions when other threads go down. In log-file we get: 2004-05-17T16:47:17 ERROR(200) ZServer uncaptured python exception, closing channel 127.0.0.1:47855 at 0x42fdb36c channel#: 589 requests:> (socket.error:(104, 'Connection reset by peer') [/opt/portaali/ContentManagement-1.0/python-2.3.3/lib/python2.3/asynchat.py|initiate_send|218] [/opt/portaali/ContentManagement-1.0/zope-2.7.0/lib/python/ZServer/medusa/http_server.py|send|417] [/opt/portaali/ContentManagement-1.0/python-2.3.3/lib/python2.3/asyncore.py|send|337]) 2004-05-17T16:48:49 ERROR(200) ZServer uncaptured python exception, closing channel 127.0.0.1:47924 at 0x430538ac channel#: 651 requests:> (socket.error:(104, 'Connection reset by peer') [/opt/portaali/ContentManagement-1.0/python-2.3.3/lib/python2.3/asynchat.py|initiate_send|218] [/opt/portaali/ContentManagement-1.0/zope-2.7.0/lib/python/ZServer/medusa/http_server.py|send|417] [/opt/portaali/ContentManagement-1.0/python-2.3.3/lib/python2.3/asyncore.py|send|337]) 2004-05-17T16:50:05 ERROR(200) ZServer uncaptured python exception, closing channel 127.0.0.1:47961 at 0x42f3ba4c channel#: 683 requests:> (socket.error:(104, 'Connection reset by peer') [/opt/portaali/ContentManagement-1.0/python-2.3.3/lib/python2.3/asynchat.py|initiate_send|218] [/opt/portaali/ContentManagement-1.0/zope-2.7.0/lib/python/ZServer/medusa/http_server.py|send|417] [/opt/portaali/ContentManagement-1.0/python-2.3.3/lib/python2.3/asyncore.py|send|337]) In tracback we will get: 2004-05-18T11:44:14 INFO(0) Zope Ready to handle requests Unhandled exception in thread started by ZServer.PubCore.ZServerPublisher.ZServerPublisher at 0x405435cc> Traceback (most recent call last): File "/opt/portaali/ContentManagement-1.0/zope-2.7.0/lib/python/ZServer/PubCore/ZServerPublisher.py", line 23, in __init__ response=response) File "/opt/portaali/ContentManagement-1.0/zope-2.7.0/lib/python/ZPublisher/Publish.py", line 372, in publish_module environ, debug, request, response) File "/opt/portaali/ContentManagement-1.0/zope-2.7.0/lib/python/ZPublisher/Publish.py", line 173, in publish_module_standard request.response.exception() AttributeError: 'NoneType' object has no attribute 'exception' Unhandled exception in thread started by ZServer.PubCore.ZServerPublisher.ZServerPublisher at 0x405435cc> Traceback (most recent call last): File "/opt/portaali/ContentManagement-1.0/zope-2.7.0/lib/python/ZServer/PubCore/ZServerPublisher.py", line 23, in __init__ response=response) File "/opt/portaali/ContentManagement-1.0/zope-2.7.0/lib/python/ZPublisher/Publish.py", line 372, in publish_module environ, debug, request, response) File "/opt/portaali/ContentManagement-1.0/zope-2.7.0/lib/python/ZPublisher/Publish.py", line 173, in publish_module_standard request.response.exception() AttributeError: 'NoneType' object has no attribute 'exception' Unhandled exception in thread started by ZServer.PubCore.ZServerPublisher.ZServerPublisher at 0x405435cc> Traceback (most recent call last): File "/opt/portaali/ContentManagement-1.0/zope-2.7.0/lib/python/ZServer/PubCore/ZServerPublisher.py", line 23, in __init__ response=response) File "/opt/portaali/ContentManagement-1.0/zope-2.7.0/lib/python/ZPublisher/Publish.py", line 372, in publish_module environ, debug, request, response) File "/opt/portaali/ContentManagement-1.0/zope-2.7.0/lib/python/ZPublisher/Publish.py", line 173, in publish_module_standard request.response.exception() AttributeError: 'NoneType' object has no attribute 'exception' Unhandled exception in thread started by ZServer.PubCore.ZServerPublisher.ZServerPublisher at 0x405435cc> Traceback (most recent call last): File "/opt/portaali/ContentManagement-1.0/zope-2.7.0/lib/python/ZServer/PubCore/ZServerPublisher.py", line 23, in __init__ response=response) File "/opt/portaali/ContentManagement-1.0/zope-2.7.0/lib/python/ZPublisher/Publish.py", line 372, in publi
Re: [Zope-dev] Re: [ZODB-Dev] Re: BTrees strangeness (was Zope 2.X BIG Session problems - blocker - our site dies - need help of experience Zope developer, please)
On Mon, 2004-05-17 at 23:08, Chris McDonough wrote: > There indeed is a minor off-by-one error: it manifests itself as > sessions timing out at most 20 seconds early. > > But there is also a deeper issue which involves the fact that a session > data object is not properly removed from an older bucket when it "moves" > due to being accessed in a later timeslice; the symptom only appears > when a browser id is "reused" to start a session after it was used to > start an older one that had timed out normally. I've got almost no clue > why this happens at this point, but I'm working on it. Ugh. This is > almost certainly what Steve is experiencing. I take that back. Actually, I think I was just reading the test results and debug output wrong. It appears to be operating normally except for the off-by-one problem (which is minor). I need to jack up the tests to do some comparisons of data values; currently I'm just testing to ensure that *something* is in the session.. I need to test if the "right" thing is in the session over time. - C ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Conflict errors on BDBMinimal storage
Hi Dr. Wagner, > thanks for the hint with minimal storage. however the same problem > occurs with filestorage and is even worse. > > we know very well, that we cannot avoid the appearance of conflict > errors in particular with zeo, and we understand what causes them. > > what wonders us is, that we find some contributions about this errors in > the mailing lists, but no one seems to have a real problem with it. but > we have! we are not worried about some errors appearing in a log, but we > are using plone2 witch currently 30 concurrent authenticated users, so > that this error occurs permanently, thus decreasing system performance > due to several retries and moreover leading to very frequent nasty > 'POS.Exception Errors' in the browser, whenever four successive tries > fail (almost every 100. request). > > this is not acceptable, and i wonder wheter there are any plone sites > with more than a handful of authenticated users that experience the same > problem. we found that one (there are more) product responsible for it > is the placeless translation service, which heavily writes to the session. Are you using ZEO for your session storage or are you using a local storage? (You hint at it above but you don't actually say one way or the other.) > so we desperately need a solution for this problem, maybe uncouple the > session of the locking mechanism or whatever, and this should be a > problem that should be addressed by the core team. because this is a > strong reason (and so far the only one) to avoid using zope/plone in a > large user environment. I also need reliable sessioning for my own customers, so I am working towards fixing the issues I know about. But the issues I know about may not be the same issues as those that you want fixed. You can ensure your issues are addressed by devoting resources towards the problem, be that time+expertise or money or both. - C ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
