Re: [ZODB-Dev] Re: BTrees strangeness (was [Zope-dev] Zope 2.X BIG Session problems - blocker - our site dies - need help of experience Zope developer, please)

2004-05-17 Thread Dieter Maurer
Chris McDonough wrote at 2004-5-15 13:04 -0400: ... Dieter, do you think you can read this patch and give a thumbs up or down on it? The patch looks good. On a different subject, the publisher probably shouldn't pass around traceback objects (e.g. when it calls into err_hook) as Tres believes

Re: [ZODB-Dev] Re: BTrees strangeness (was [Zope-dev] Zope 2.X BIG Session problems - blocker - our site dies - need help of experience Zope developer, please)

2004-05-17 Thread Chris McDonough
On Mon, 2004-05-17 at 13:00, Dieter Maurer wrote: Chris McDonough wrote at 2004-5-15 13:04 -0400: ... Dieter, do you think you can read this patch and give a thumbs up or down on it? The patch looks good. Great, thanks for looking at it. - C

Re: [ZODB-Dev] Re: BTrees strangeness (was [Zope-dev] Zope 2.X BIG Session problems - blocker - our site dies - need help of experience Zope developer, please)

2004-05-15 Thread Chris McDonough
On Fri, 2004-05-14 at 22:49, Michael Dunstan wrote: hmm... I suspect there are few holes in this... - Need to make sure there is no get_transaction().begin() in zpublisher_exception_hook() of lib/python/Zope/App/startup.py That begin() would mean that the error handling is not in

Re: [ZODB-Dev] Re: BTrees strangeness (was [Zope-dev] Zope 2.X BIG Session problems - blocker - our site dies - need help of experience Zope developer, please)

2004-05-14 Thread Michael Dunstan
On 14/05/2004, at 10:06 AM, Chris McDonough wrote: I am tempted to check the following into the 2.7 branch and HEAD: - error occurs in same transaction as main request patch to Publish.py. See http://www.plope.com/Members/chrism/ publishpy_errorinmaintrainsaction.patch/file_view for the

Re: [ZODB-Dev] Re: BTrees strangeness (was [Zope-dev] Zope 2.X BIG Session problems - blocker - our site dies - need help of experience Zope developer, please)

2004-05-13 Thread Chris McDonough
On Wed, 2004-05-12 at 22:54, Michael Dunstan wrote: On 11/05/2004, at 4:54 PM, Chris McDonough wrote: On Mon, 2004-04-26 at 18:34, Michael Dunstan wrote: I removed all the above hacks leaving just the one that comments out the construction of the to_notify list. Reran the test rig.

Re: [ZODB-Dev] Re: BTrees strangeness (was [Zope-dev] Zope 2.X BIG Session problems - blocker - our site dies - need help of experience Zope developer, please)

2004-05-13 Thread Chris McDonough
Largely due to Michael I believe I have isolated and fixed every reported sessioning error except this (still-difficult-to-reproduce-but-definitely-still-existing) KeyError bug in temporary storage. I can let the test rig run for several hours; it happens maybe once every hour or two, so I've not

Re: [ZODB-Dev] Re: BTrees strangeness (was [Zope-dev] Zope 2.X BIG Session problems - blocker - our site dies - need help of experience Zope developer, please)

2004-05-12 Thread Michael Dunstan
On 11/05/2004, at 4:54 PM, Chris McDonough wrote: On Mon, 2004-04-26 at 18:34, Michael Dunstan wrote: I removed all the above hacks leaving just the one that comments out the construction of the to_notify list. Reran the test rig. And have yet to see any KeyErrors. Have you seen any since the

Re: [ZODB-Dev] Re: BTrees strangeness (was [Zope-dev] Zope 2.X BIG Session problems - blocker - our site dies - need help of experience Zope developer, please)

2004-05-11 Thread Dieter Maurer
Hi Chris, Chris McDonough wrote at 2004-5-11 00:54 -0400: ... So what do folks think of this error-case transaction isolation patch? Michael's original patch is preserved here (although it didn't survive cut and paste from my mail client in a pristine way, you'll get the idea):

Re: [ZODB-Dev] Re: BTrees strangeness (was [Zope-dev] Zope 2.X BIG Session problems - blocker - our site dies - need help of experience Zope developer, please)

2004-05-11 Thread Chris McDonough
On Tue, 2004-05-11 at 13:04, Dieter Maurer wrote: Hi Chris, Chris McDonough wrote at 2004-5-11 00:54 -0400: ... So what do folks think of this error-case transaction isolation patch? Michael's original patch is preserved here (although it didn't survive cut and paste from my mail client

Re: [ZODB-Dev] Re: BTrees strangeness (was [Zope-dev] Zope 2.X BIG Session problems - blocker - our site dies - need help of experience Zope developer, please)

2004-05-10 Thread Chris McDonough
Whew. I finally got a chance to review at least a bit of Michael's excellent work... On Mon, 2004-04-26 at 18:34, Michael Dunstan wrote: Okay - I've gone ahead and implemented a flavour of transaction isolation for errors. (See below for patch.) That seems to have done the trick just

Re: [ZODB-Dev] Re: BTrees strangeness (was [Zope-dev] Zope 2.X BIG Session problems - blocker - our site dies - need help of experience Zope developer, please)

2004-04-26 Thread Michael Dunstan
On 11/03/2004, at 7:25 PM, Chris McDonough wrote: On Wed, 2004-03-10 at 23:14, michael wrote: Thanks for your excellent isolation here. I don't get a KeyError thrown until *after* at least one rendering of standard_error_message that includes an access to a session variable. And I'm only

Re: [ZODB-Dev] Re: BTrees strangeness (was [Zope-dev] Zope 2.X BIG Session problems - blocker - our site dies - need help of experience Zope developer, please)

2004-03-11 Thread Chris McDonough
On Thu, 2004-03-11 at 06:52, Chris Withers wrote: Well, what if you want to make a note in some object (say the error_log), that something bad happened? The error_log doesn't use persistent objects to keep its messages. What if you want to make a change in the error handler? If anyone had

Re: [ZODB-Dev] Re: BTrees strangeness (was [Zope-dev] Zope 2.X BIG Session problems - blocker - our site dies - need help of experience Zope developer, please)

2004-03-11 Thread Toby Dickenson
On Thursday 11 March 2004 12:19, Chris McDonough wrote: What if you want to make a change in the error handler? If anyone had been depending on this behavior in their current error handlers, they would have seen weirdnesses in their applications like those being reported now. I have

Re: [ZODB-Dev] Re: BTrees strangeness (was [Zope-dev] Zope 2.X BIG Session problems - blocker - our site dies - need help of experience Zope developer, please)

2004-03-11 Thread alex
Hi Chris, On Tue, 9 Mar 2004, Chris McDonough wrote: But it seems I forget to install new Transience module to the new Zope instance. Should I? Yes, please! Sorry, it does not seems to work on my first attempt: Traceback (innermost last): * Module ZPublisher.Publish, line 163, in

Re: [ZODB-Dev] Re: BTrees strangeness (was [Zope-dev] Zope 2.X BIG Session problems - blocker - our site dies - need help of experience Zope developer, please)

2004-03-11 Thread Chris McDonough
On Thu, 2004-03-11 at 08:32, [EMAIL PROTECTED] wrote: Hi Chris, On Tue, 9 Mar 2004, Chris McDonough wrote: But it seems I forget to install new Transience module to the new Zope instance. Should I? Yes, please! Sorry, it does not seems to work on my first attempt: Right. Could

Re: [ZODB-Dev] Re: BTrees strangeness (was [Zope-dev] Zope 2.X BIG Session problems - blocker - our site dies - need help of experience Zope developer, please)

2004-03-11 Thread Chris McDonough
On Thu, 2004-03-11 at 07:29, Toby Dickenson wrote: I have applications that send emails in error handlers. MailHost isnt transactional today, but Ive always wanted it to be. I suppose that's true. If someone else doesn't do it first, during the bug day we'll try to support this use case,

Re: [ZODB-Dev] Re: BTrees strangeness (was [Zope-dev] Zope 2.X BIG Session problems - blocker - our site dies - need help of experience Zope developer, please)

2004-03-10 Thread Dieter Maurer
michael wrote at 2004-3-10 15:22 +1300: ... I have been trying on and off to recreate this error via brute force loading of the simplest possible site that uses sessions. I failed to see this particular KeyError *until* I tried reading a session variable from standard_error_message. Now I can

Re: [ZODB-Dev] Re: BTrees strangeness (was [Zope-dev] Zope 2.X BIG Session problems - blocker - our site dies - need help of experience Zope developer, please)

2004-03-10 Thread Chris McDonough
On Wed, 2004-03-10 at 15:20, Dieter Maurer wrote: michael wrote at 2004-3-10 15:22 +1300: ... I have been trying on and off to recreate this error via brute force loading of the simplest possible site that uses sessions. I failed to see this particular KeyError *until* I tried reading a

Re: [ZODB-Dev] Re: BTrees strangeness (was [Zope-dev] Zope 2.X BIG Session problems - blocker - our site dies - need help of experience Zope developer, please)

2004-03-10 Thread Chris McDonough
Nevermind. http://zope.org/Collectors/Zope/789 and http://zope.org/Collectors/Zope/786 The bug neglector is really living up to its name lately (not pointing fingers, mea culpa). - C On Wed, 2004-03-10 at 15:20, Dieter Maurer wrote: michael wrote at 2004-3-10 15:22 +1300: ... I have been

Re: [ZODB-Dev] Re: BTrees strangeness (was [Zope-dev] Zope 2.X BIG Session problems - blocker - our site dies - need help of experience Zope developer, please)

2004-03-10 Thread Chris McDonough
On Wed, 2004-03-10 at 23:14, michael wrote: Thanks for your excellent isolation here. I don't get a KeyError thrown until *after* at least one rendering of standard_error_message that includes an access to a session variable. And I'm only getting KeyError's for sessions that have had a

Re: [ZODB-Dev] Re: BTrees strangeness (was [Zope-dev] Zope 2.X BIG Session problems - blocker - our site dies - need help of experience Zope developer, please)

2004-03-10 Thread michael
On 11/03/2004, at 7:25 PM, Chris McDonough wrote: On Wed, 2004-03-10 at 23:14, michael wrote: Thanks for your excellent isolation here. I don't get a KeyError thrown until *after* at least one rendering of standard_error_message that includes an access to a session variable. And I'm only

Re: [ZODB-Dev] Re: BTrees strangeness (was [Zope-dev] Zope 2.X BIG Session problems - blocker - our site dies - need help of experience Zope developer, please)

2004-03-09 Thread Alex V. Koval
Hi Chris, On Fri, 05 Mar 2004 13:08:01 -0500, Chris McDonough [EMAIL PROTECTED] wrote: I am monitoring site now, and will tell you about the results. OK, many thanks! Running For: 3 days 4 hours 28 min 29 sec. I have enabled 400 error_log ex exceptions to keep, and during 3 days I got 2

Re: [ZODB-Dev] Re: BTrees strangeness (was [Zope-dev] Zope 2.X BIG Session problems - blocker - our site dies - need help of experience Zope developer, please)

2004-03-09 Thread michael
On 10/03/2004, at 6:29 AM, Chris McDonough wrote: On Tue, 2004-03-09 at 06:36, Alex V. Koval wrote: I have enabled 400 error_log ex exceptions to keep, and during 3 days I got 2 errors on the site: TimeUsername (User Id) Exception 16:35:01Anonymous User (None)

Re: BTrees strangeness (was [Zope-dev] Zope 2.X BIG Session problems - blocker - our site dies - need help of experience Zope developer, please)

2004-03-05 Thread alex
Hi Chris, On Wed, 3 Mar 2004, Chris McDonough wrote: (boldly crossposting this to zodb-dev, please respond on one list or the other but not both) That error *appears* to be caused by reaching a state that is impossible to reach. The code in question is: for key in

Re: [ZODB-Dev] Re: BTrees strangeness (was [Zope-dev] Zope 2.X BIG Session problems - blocker - our site dies - need help of experience Zope developer, please)

2004-03-05 Thread alex
Hi Chris, On Wed, 3 Mar 2004, Chris McDonough wrote: With this text: zodb_db temporary # Temporary storage database (for sessions) filestorage path $INSTANCE/var/Sessions.fs /filestorage mount-point /temp_folder container-class

Re: [ZODB-Dev] Re: BTrees strangeness (was [Zope-dev] Zope 2.X BIG Session problems - blocker - our site dies - need help of experience Zope developer, please)

2004-03-05 Thread Chris McDonough
On Fri, 2004-03-05 at 09:15, [EMAIL PROTECTED] wrote: Installed. I had to install new zope instance first, then import old content via zexp recreate manually all admins. Because when you change the running zope config in Zope 2.7.0 it has no impact on temp_folder - Sessions.fs is not

Re: [ZODB-Dev] Re: BTrees strangeness (was [Zope-dev] Zope 2.X BIG Session problems - blocker - our site dies - need help of experience Zope developer, please)

2004-03-04 Thread John Belmonte
Chris McDonough wrote: On Wed, 2004-03-03 at 22:20, Casey Duncan wrote: for key in list(self._data.keys(None, max_ts)): assert(key = max_ts) STRICT and _assert(self._data.has_key(key)) for v in self._data[key].values(): to_notify.append(v)

Re: [ZODB-Dev] Re: BTrees strangeness (was [Zope-dev] Zope 2.X BIG Session problems - blocker - our site dies - need help of experience Zope developer, please)

2004-03-04 Thread Chris McDonough
On Wed, 2004-03-03 at 22:53, John Belmonte wrote: If I'm following this thread correctly, isn't the code failing because the BTree is corrupted (that is, BTrees.check.check chokes)? If that's the case then you're certainly right to avoid masking the problem. We don't know that it's

Re: [Zope-dev] Zope 2.X BIG Session problems - blocker - our site dies - need help of experience Zope developer, please

2004-03-03 Thread alex
Chris, No, just a few minutes ago I got this again: Time2004/03/03 07:45:04.662 GMT User Name (User Id) Anonymous User (None) Request URL http://www.chalkface.com/catalog/html/custom/index_html Exception Type KeyError Exception Value 1078236460 Traceback (innermost last):

Re: BTrees strangeness (was [Zope-dev] Zope 2.X BIG Session problems - blocker - our site dies - need help of experience Zope developer, please)

2004-03-03 Thread Chris McDonough
(boldly crossposting this to zodb-dev, please respond on one list or the other but not both) That error *appears* to be caused by reaching a state that is impossible to reach. The code in question is: for key in list(self._data.keys(None, max_ts)): assert(key = max_ts)

Re: [Zope-dev] Zope 2.X BIG Session problems - blocker - our site dies - need help of experience Zope developer, please

2004-03-03 Thread Chris Withers
[EMAIL PROTECTED] wrote: * Module Products.Transience.Transience, line 419, in _gc KeyError: 1078236460 (on the other Chris's behalf) wagh! Chris ;-) -- Simplistix - Content Management, Zope Python Consulting - http://www.simplistix.co.uk

Re: [ZODB-Dev] Re: BTrees strangeness (was [Zope-dev] Zope 2.X BIG Session problems - blocker - our site dies - need help of experience Zope developer, please)

2004-03-03 Thread Jeremy Hylton
On Wed, 2004-03-03 at 04:55, Chris McDonough wrote: (boldly crossposting this to zodb-dev, please respond on one list or the other but not both) That error *appears* to be caused by reaching a state that is impossible to reach. The code in question is: for key in

Re: BTrees strangeness (was [Zope-dev] Zope 2.X BIG Session problems - blocker - our site dies - need help of experience Zope developer, please)

2004-03-03 Thread Dieter Maurer
Chris McDonough wrote at 2004-3-3 04:55 -0500: (boldly crossposting this to zodb-dev, please respond on one list or the other but not both) That error *appears* to be caused by reaching a state that is impossible to reach. The code in question is: for key in list(self._data.keys(None,

Re: [ZODB-Dev] Re: BTrees strangeness (was [Zope-dev] Zope 2.X BIG Session problems - blocker - our site dies - need help of experience Zope developer, please)

2004-03-03 Thread Chris McDonough
On Wed, 2004-03-03 at 22:20, Casey Duncan wrote: for key in list(self._data.keys(None, max_ts)): assert(key = max_ts) STRICT and _assert(self._data.has_key(key)) for v in self._data[key].values(): to_notify.append(v)

Re: [Zope-dev] Zope 2.X BIG Session problems - blocker - our site dies - need help of experience Zope developer, please

2004-03-02 Thread Chris McDonough
Great, I'm going to consider that a resounding endorsement and check it in soon; please do let me know if you see anything odd come up. If anyone else has been having issues with the old Transience module, and would like to provide feedback on the newer implementation, please get this file:

Re: [Zope-dev] Zope 2.X BIG Session problems - blocker - our site dies - need help of experience Zope developer, please

2004-03-01 Thread Anthony Baxter
Chris McDonough wrote: That is very critital and 'blocker' bug for Zope. I do not understand how people could use Zope on a sites with high load.. They don't use sessions under high load. Or they don't use the standard session machinery. We use my SQLSession stuff, and it's fine under load. --

Re: [Zope-dev] Zope 2.X BIG Session problems - blocker - our site dies - need help of experience Zope developer, please

2004-03-01 Thread Chris McDonough
I installed new Transience.py. During my little test it works fine. But real test will be on Monday when students start logging in as complete classes, sometimes there are hundreds of them logging on simultaneously, so we will see. Any news? ;-)

Re: [Zope-dev] Zope 2.X BIG Session problems - blocker - our site dies - need help of experience Zope developer, please

2004-03-01 Thread Steve Jibson
Chris, I'm not sure if you'd even planned to, but have you looked over this replacement Transience.py? Do you have an opinion on the re-implementation? I plan on looking over it myself, but even if I like it, I'd sleep better while it runs on my customer's servers if I knew it had your

Re: [Zope-dev] Zope 2.X BIG Session problems - blocker - our site dies - need help of experience Zope developer, please

2004-02-29 Thread Chris McDonough
Well, I was also shamed into creating a new implementation of the sessioning stuff that might work better under higher load. The people that are now having problems should try to replace the file in their Zope software home named lib/python/Products/Transience/Transience.py with this one:

Re: [Zope-dev] Zope 2.X BIG Session problems - blocker - our site dies - need help of experience Zope developer, please

2004-02-29 Thread alex
Hi Chris, Well, I was also shamed into creating a new implementation of the sessioning stuff that might work better under higher load. Thank you very much for a fast fix/replacement of Transience :) The people that are now having problems should try to replace the file in their Zope

[Zope-dev] Zope 2.X BIG Session problems - blocker - our site dies - need help of experience Zope developer, please

2004-02-28 Thread alex
Hi On Tue, 24 Feb 2004, Chris McDonough wrote: Yes, it's still broke. No, there is no fix. Sorry. Thanks Cris. Thats very bad. It is very bad from end-user perspective that such critical bug has been found more then year ago, and nobody at Zope has fixed their code during a year time. I

Re: [Zope-dev] Zope 2.X BIG Session problems - blocker - our site dies - need help of experience Zope developer, please

2004-02-28 Thread alex
Hi As far I understand the error happends in this code Question to Python developers: As far I understand, index does contain b, and data does not contain 'b'. Simple question: why does the line: v = self._data[b].get(k, notfound) throw KeyError at any case? get does have next argument,

Re: [Zope-dev] Zope 2.X BIG Session problems - blocker - our site dies - need help of experience Zope developer, please

2004-02-28 Thread Chris McDonough
On Sat, 2004-02-28 at 04:07, [EMAIL PROTECTED] wrote: Hi On Tue, 24 Feb 2004, Chris McDonough wrote: Yes, it's still broke. No, there is no fix. Sorry. Thanks Cris. Thats very bad. Yes it is. It is very bad from end-user perspective that such critical bug has been found more then

Re: [Zope-dev] Zope 2.X BIG Session problems - blocker - our site dies - need help of experience Zope developer, please

2004-02-28 Thread Chris McDonough
On Sat, 2004-02-28 at 04:51, [EMAIL PROTECTED] wrote: Hi As far I understand the error happends in this code Question to Python developers: As far I understand, index does contain b, and data does not contain 'b'. Simple question: why does the line: v = self._data[b].get(k,

Re: [Zope-dev] Zope 2.X BIG Session problems - blocker - our site dies - need help of experience Zope developer, please

2004-02-28 Thread Dieter Maurer
[EMAIL PROTECTED] wrote at 2004-2-28 11:07 +0200: ... Yes, it's still broke. No, there is no fix. Sorry. Thanks Cris. Thats very bad. It is very bad from end-user perspective that such critical bug has been found more then year ago, and nobody at Zope has fixed their code during a year time.

Re: [Zope-dev] Zope 2.X BIG Session problems - blocker - our site dies - need help of experience Zope developer, please

2004-02-28 Thread Chris McDonough
On Sat, 2004-02-28 at 16:32, Dieter Maurer wrote: When we started using Zope sessions, we experienced a set of session problems. I started fixing them, but it turned out that the implementation was very ambitious, too ambitious to get it safe. You're very kind to put it that way. ;-) I,

Re: [Zope-dev] Zope 2.X BIG Session problems - blocker - our site dies - need help of experience Zope developer, please

2004-02-28 Thread Santi Camps
When we started using Zope sessions, we experienced a set of session problems. I started fixing them, but it turned out that the implementation was very ambitious, too ambitious to get it safe. I, therefore, switched plans and implemented an alternative Transience module -- much simpler and