RE: [Zope-dev] Re: Preliminary findings: Zope 2.7 leakage caused by errors

2004-05-21 Thread Jean-Francois . Doyon
Juan David,

Indeed, thanks for the input, I've ruled out new_publish as the cause as
well ...

Right now, I'm at the stage where I severely suspect it's got something to
do
with i18n support or translation somehow, but don't know where exactly yet.

If I remove the message_catalog, the problem goes away.

If I remove the TranslationService instance, the problem goes away. Do you
use this? Does anybody know of alternatives to the
TranslationService+Localizer
combo to enable i18n in ZPT's on Zope 2?

If I keep the above two, but delete my custom Localizer instances, the
problem persists.

I was trying to find an alternative message_catalog to use, but couldn't
find one (CMFLocalizer is based on yours).

So I'm going to try and narrow the field some more.  I *quickly* looked
the message_catalog and TranslationService code, and nothing jumped out.
I have to also look at the ZPT engine itself, maybe it's i18n
support is the cause of the problem.

I'll run further tests ...

Thanks,
J.F.

-Original Message-
From: J. David Ibáñez [mailto:[EMAIL PROTECTED]
Sent: May 21, 2004 8:17 AM
To: [EMAIL PROTECTED]
Cc: [EMAIL PROTECTED]; [EMAIL PROTECTED]
Subject: Re: [Zope-dev] Re: Preliminary findings: Zope 2.7 leakage
caused by errors



The problem I have with one of my sites seems to have a different cause, 
as it
does not show up when errors occur, but when requests are succesful.

So far I haven't been able to reproduce the problem.

You can try to modify new_publish to catch exceptions:

def new_publish(request, module_name, after_list, debug=0):
id = get_ident()
Publish._requests[id] = request
try:
x = Publish.old_publish(request, module_name, after_list, debug)
except:
del Publish._requests[id]
raise
else:
try:
del Publish._requests[id]
except KeyError:
...


Though it is unlikely the leak to be there.




[EMAIL PROTECTED] wrote:

Indeed, I was hoping the ref in the dictionary there was getting in the way
when old_publish was raising an exception, but I tried using a
weakref.proxy()
instead and that didn't help.

I managed to test without Localizer (It dawned on me that since I'm looking
at errors, it really doesn't matter if I don't have Localizer installed
since
not having it will generate errors, which is what I want!) ... And the
problem
diappeared ...

So it's definitely Localizer, but it's not the Globals() request patch ...
Or
at least not the act of patching it itself, that causes the problem.

*sigh*
J.F.

-Original Message-
From: Dieter Maurer [mailto:[EMAIL PROTECTED]
Sent: May 20, 2004 2:58 PM
To: [EMAIL PROTECTED]
Cc: [EMAIL PROTECTED]; [EMAIL PROTECTED]; [EMAIL PROTECTED];
[EMAIL PROTECTED]; [EMAIL PROTECTED]
Subject: RE: [Zope-dev] Re: Preliminary findings: Zope 2.7 leakage
caused by errors


[EMAIL PROTECTED] wrote at 2004-5-20 09:58 -0400:
  

...
def new_publish(request, module_name, after_list, debug=0): 
   id = get_ident() 
   print Localizer got thread id:  + str(id)
   Publish._requests[id] = request 
   print Request dict is now:  + str(Publish._requests)   
   x = Publish.old_publish(request, module_name, after_list, debug) 
   try:
   del Publish._requests[id]
   except KeyError:
...



This code cannot leak requests in large numbers (though it may
leak a few requests for some time).

Reason: Zope does not normally create new threads. This implies
that thread_ids are reused and thereby old requests flushed from
the dict.

  



-- 
J. David Ibáñez
Founder and CTO of Itaapy http://www.itaapy.com
9 rue Darwin, 75018 Paris
Tel +33 (0)1 42 23 67 45 / Fax +33 (0)1 53 28 27 88 

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists -
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )


Re: [Zope-dev] Re: Preliminary findings: Zope 2.7 leakage caused by errors

2004-05-21 Thread J. David Ibáñez
The problem I have with one of my sites seems to have a different cause, 
as it
does not show up when errors occur, but when requests are succesful.

So far I haven't been able to reproduce the problem.
You can try to modify new_publish to catch exceptions:
def new_publish(request, module_name, after_list, debug=0):
   id = get_ident()
   Publish._requests[id] = request
   try:
   x = Publish.old_publish(request, module_name, after_list, debug)
   except:
   del Publish._requests[id]
   raise
   else:
   try:
   del Publish._requests[id]
   except KeyError:
...
Though it is unlikely the leak to be there.

[EMAIL PROTECTED] wrote:
Indeed, I was hoping the ref in the dictionary there was getting in the way
when old_publish was raising an exception, but I tried using a
weakref.proxy()
instead and that didn't help.
I managed to test without Localizer (It dawned on me that since I'm looking
at errors, it really doesn't matter if I don't have Localizer installed
since
not having it will generate errors, which is what I want!) ... And the
problem
diappeared ...
So it's definitely Localizer, but it's not the Globals() request patch ...
Or
at least not the act of patching it itself, that causes the problem.
*sigh*
J.F.
-Original Message-
From: Dieter Maurer [mailto:[EMAIL PROTECTED]
Sent: May 20, 2004 2:58 PM
To: [EMAIL PROTECTED]
Cc: [EMAIL PROTECTED]; [EMAIL PROTECTED]; [EMAIL PROTECTED];
[EMAIL PROTECTED]; [EMAIL PROTECTED]
Subject: RE: [Zope-dev] Re: Preliminary findings: Zope 2.7 leakage
caused by errors
[EMAIL PROTECTED] wrote at 2004-5-20 09:58 -0400:
 

...
def new_publish(request, module_name, after_list, debug=0): 
  id = get_ident() 
  print Localizer got thread id:  + str(id)
  Publish._requests[id] = request 
  print Request dict is now:  + str(Publish._requests)   
  x = Publish.old_publish(request, module_name, after_list, debug) 
  try:
  del Publish._requests[id]
  except KeyError:
...
   

This code cannot leak requests in large numbers (though it may
leak a few requests for some time).
Reason: Zope does not normally create new threads. This implies
that thread_ids are reused and thereby old requests flushed from
the dict.
 


--
J. David Ibáñez
Founder and CTO of Itaapy http://www.itaapy.com
9 rue Darwin, 75018 Paris
Tel +33 (0)1 42 23 67 45 / Fax +33 (0)1 53 28 27 88 

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )


Re: [Zope-dev] Re: Preliminary findings: Zope 2.7 leakage caused by errors

2004-05-21 Thread J. David Ibanez
No, I don't use TranslationService. A quick glance at the code shows
that TS uses several caches (dictionaries) which are stored in the request
object: _localizer_placeful_mc_cache, _translation_service_cache
and _ts_domain_cache.
The values of these dictionaries are acquisition wrappers (objects returned
by unrestrictedTraverse or explicitly built). Maybe the leak is there.
[EMAIL PROTECTED] wrote:
Juan David,
Indeed, thanks for the input, I've ruled out new_publish as the cause as
well ...
Right now, I'm at the stage where I severely suspect it's got something to
do
with i18n support or translation somehow, but don't know where exactly yet.
If I remove the message_catalog, the problem goes away.
If I remove the TranslationService instance, the problem goes away. Do you
use this? Does anybody know of alternatives to the
TranslationService+Localizer
combo to enable i18n in ZPT's on Zope 2?
If I keep the above two, but delete my custom Localizer instances, the
problem persists.
I was trying to find an alternative message_catalog to use, but couldn't
find one (CMFLocalizer is based on yours).
So I'm going to try and narrow the field some more.  I *quickly* looked
the message_catalog and TranslationService code, and nothing jumped out.
I have to also look at the ZPT engine itself, maybe it's i18n
support is the cause of the problem.
I'll run further tests ...
Thanks,
J.F.
-Original Message-
From: J. David Ibáñez [mailto:[EMAIL PROTECTED]
Sent: May 21, 2004 8:17 AM
To: [EMAIL PROTECTED]
Cc: [EMAIL PROTECTED]; [EMAIL PROTECTED]
Subject: Re: [Zope-dev] Re: Preliminary findings: Zope 2.7 leakage
caused by errors

The problem I have with one of my sites seems to have a different cause, 
as it
does not show up when errors occur, but when requests are succesful.

So far I haven't been able to reproduce the problem.
You can try to modify new_publish to catch exceptions:
def new_publish(request, module_name, after_list, debug=0):
   id = get_ident()
   Publish._requests[id] = request
   try:
   x = Publish.old_publish(request, module_name, after_list, debug)
   except:
   del Publish._requests[id]
   raise
   else:
   try:
   del Publish._requests[id]
   except KeyError:
...
Though it is unlikely the leak to be there.
 

--
J. David Ibáñez
Founder and CTO of Itaapy http://www.itaapy.com
9 rue Darwin, 75018 Paris
Tel +33 (0)1 42 23 67 45 / Fax +33 (0)1 53 28 27 88 

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )


RE: [Zope-dev] Re: Preliminary findings: Zope 2.7 leakage caused by errors

2004-05-21 Thread Dieter Maurer
[EMAIL PROTECTED] wrote at 2004-5-21 09:34 -0400:
 ...
If I remove the TranslationService instance, the problem goes away. Do you
use this? Does anybody know of alternatives to the
TranslationService+Localizer
combo to enable i18n in ZPT's on Zope 2?

I am no internationalization expert. But many posts suggest
to use PTS (Placeless Translation Service) nowadays...

-- 
Dieter

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )


Re: [Zope-dev] Re: Preliminary findings: Zope 2.7 leakage caused by errors

2004-05-21 Thread Dieter Maurer
J. David Ibanez wrote at 2004-5-21 16:50 +0200:

No, I don't use TranslationService. A quick glance at the code shows
that TS uses several caches (dictionaries) which are stored in the request
object: _localizer_placeful_mc_cache, _translation_service_cache
and _ts_domain_cache.

The values of these dictionaries are acquisition wrappers (objects returned
by unrestrictedTraverse or explicitly built). Maybe the leak is there.

When these dictionaries are held in other (or form or Lazy),
then there is no problem -- as other is explicitly clear
when the request is closed.

However, when they are (direct) attributes of the request
object, then they will form cycles which almost surely
cause leakage.

-- 
Dieter

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )


RE: [Zope-dev] Re: Preliminary findings: Zope 2.7 leakage caused by errors

2004-05-20 Thread Jean-Francois . Doyon
Juan David,

I'm using 1.0.1 ... I'm just abotu to try and test with 1.1.0a3 ...

BTW, does that mean it's alpha software or something? is 1.1.0 stable?

If the problem comes from Localizer I suspect this patch might be the cause:

def new_publish(request, module_name, after_list, debug=0): 
id = get_ident() 
print Localizer got thread id:  + str(id)
Publish._requests[id] = request 
print Request dict is now:  + str(Publish._requests)   
x = Publish.old_publish(request, module_name, after_list, debug) 
try:
del Publish._requests[id]
except KeyError:
# XXX 
# Some people has reported that sometimes a KeyError exception is
# raised in the previous line, I haven't been able to reproduce it.
# This try/except clause seems to work. I'd prefer to understand
# what is happening.
LOG('Localizer', PROBLEM,
The thread number %s don't has a request object associated. %
id)
except:
print Localizer got an exception 
 
return x 

What happens if an exception occurs here:

x = Publish.old_publish(request, module_name, after_list, debug) 

??

If some clean up code is supposed to occur it might be held back by the fact
that
there's a reference to the request kept in the global dictionary ...
At least that's my theory :)
But then I'm mostly uneducated about the intricacies of exception handling
at this
level, though I've been forced to learn quickly :)

I'd be curious to know at least if you can reproduce the leak by generating
loads
of errors on a Localizer enabled site?

One option would also be to simply wrap it in a try:catch and see what
happens there
... But I noticed 1.1.0a3 uses a different model for wrapping the request,
so that
just might do the trick.

Thanks for a very cool and useful product BTW.  This is combination with the
TranslationService is saving me loads of time.

Cheers,
J.F.

-Original Message-
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]
Behalf Of J. David Ibáñez
Sent: May 20, 2004 6:47 AM
To: Tres Seaver
Cc: [EMAIL PROTECTED]; [EMAIL PROTECTED];
[EMAIL PROTECTED]; [EMAIL PROTECTED]
Subject: [Zope-dev] Re: Preliminary findings: Zope 2.7 leakage caused by
errors



Localizer (1.0.1) dynamically patches Zope, it stores a dictionary in 
the module
ZPublisher.Publish. The keys are integers, they are the values 
returned by Python's
thread.get_ident. The values are ZPublisher.HTTPRequest.HTTPRequest
instances, they are not acquisition wrappers.

Jean-François, which Localizer version do you use? Just to be sure we 
look at
the same code.


Tres Seaver wrote:

 [EMAIL PROTECTED] wrote:

 Gents,

 In following to the discussions regarding the leakage zope.org and my 
 site
 suffer from, I've done some testing, and here's what I've found.

 First of all, the leak does seem to occur when errors occur.  Unlike 
 what
 Chris suggested, I still suffer from this leak even when I completely
 remove standard_error_message.

 How I tested:

 I use JMeter, and a 99% identical box, dedicated for testing:

 - RH 7.3
 - Python 2.3.3 compiled with ./configure and no options
 - Zope 2.7 + a variety of products, both custom and well known (CMF,
 Localizer, etc ...)

 I look at my error log on my main site to see the types of errors 
 coming up
 and note the URL/parameters.
 Such errors include KeyErrors, a RecursionDepth error, a
 Method-something-or-other error, and so on.

 I setup JMeter with multiple concurrent threads and run it.  Within 
 minutes
 I can see refcounts growing abnormally on the test server.

 I then shut everything down and minimize the caches.  The following 
 are left
 around that shouldn't be:

 Class  May 19, 2004 1:52 pm  May 19, 2004 2:26 pm  Delta  
 Acquisition.ImplicitAcquirerWrapper  1025  1309  +284  
 ZServer.HTTPResponse.ZServerHTTPResponse  150  247  +97  
 ZPublisher.BaseRequest.RequestContainer  146  240  +94  
 ZPublisher.HTTPRequest.HTTPRequest  172  262  +90

 Right now I've focused on finding out why the Requests are still around.

 Using the gc module, I've found that all leaked requests are being 
 held by a
 dictionary.  If I look at said dictionaries, this is what they might 
 look
 like:

 {'REQUEST': HTTPRequest,

URL=/VirtualHostBase/http/atlastest2.ccrs.nrcan.gc.ca:80/VirtualHostRoot/_vh


 _site/francais/mapsatoz},
  {'REQUEST': HTTPRequest,

URL=/VirtualHostBase/http/atlastest2.ccrs.nrcan.gc.ca:80/VirtualHostRoot/_vh


 _site/francais/mapsatoz},
  {'REQUEST': HTTPRequest,

URL=/VirtualHostBase/http/atlastest2.ccrs.nrcan.gc.ca:80/VirtualHostRoot/_vh


 _site/english/search/political},
  {'REQUEST': HTTPRequest,

URL=/VirtualHostBase/http/atlastest2.ccrs.nrcan.gc.ca:80/VirtualHostRoot/_vh


 _site/english/search/political},
  {'REQUEST': HTTPRequest,

URL=/VirtualHostBase/http/atlastest2.ccrs.nrcan.gc.ca:80/VirtualHostRoot/_vh


 _site/english/search/political},
  {'REQUEST': HTTPRequest,

URL=/VirtualHostBase/http/atlastest2.ccrs.nrcan.gc.ca:80

Re: [Zope-dev] Re: Preliminary findings: Zope 2.7 leakage caused by errors

2004-05-20 Thread J. David Ibáñez
The a in 1.1.0a3 means alpha. If the problem is in Localizer
1.0.1 I think it will be in 1.1.0a3 too.
Yes, I have a site with a similar behaviour, unfortunately today
the network connection is too slooow, so I can't test that site. I'm
going to do some local tests, though.
Yes, the problem might be the fact that new_publish does not
catch exceptions to remove the request from the dictionary.

[EMAIL PROTECTED] wrote:
Juan David,
I'm using 1.0.1 ... I'm just abotu to try and test with 1.1.0a3 ...
BTW, does that mean it's alpha software or something? is 1.1.0 stable?
If the problem comes from Localizer I suspect this patch might be the cause:
def new_publish(request, module_name, after_list, debug=0): 
   id = get_ident() 
   print Localizer got thread id:  + str(id)
   Publish._requests[id] = request 
   print Request dict is now:  + str(Publish._requests)   
   x = Publish.old_publish(request, module_name, after_list, debug) 
   try:
   del Publish._requests[id]
   except KeyError:
   # XXX 
   # Some people has reported that sometimes a KeyError exception is
   # raised in the previous line, I haven't been able to reproduce it.
   # This try/except clause seems to work. I'd prefer to understand
   # what is happening.
   LOG('Localizer', PROBLEM,
   The thread number %s don't has a request object associated. %
id)
   except:
   print Localizer got an exception 

   return x 

What happens if an exception occurs here:
   x = Publish.old_publish(request, module_name, after_list, debug) 

??
If some clean up code is supposed to occur it might be held back by the fact
that
there's a reference to the request kept in the global dictionary ...
At least that's my theory :)
But then I'm mostly uneducated about the intricacies of exception handling
at this
level, though I've been forced to learn quickly :)
I'd be curious to know at least if you can reproduce the leak by generating
loads
of errors on a Localizer enabled site?
One option would also be to simply wrap it in a try:catch and see what
happens there
... But I noticed 1.1.0a3 uses a different model for wrapping the request,
so that
just might do the trick.
Thanks for a very cool and useful product BTW.  This is combination with the
TranslationService is saving me loads of time.
Cheers,
J.F.
-Original Message-
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]
Behalf Of J. David Ibáñez
Sent: May 20, 2004 6:47 AM
To: Tres Seaver
Cc: [EMAIL PROTECTED]; [EMAIL PROTECTED];
[EMAIL PROTECTED]; [EMAIL PROTECTED]
Subject: [Zope-dev] Re: Preliminary findings: Zope 2.7 leakage caused by
errors

Localizer (1.0.1) dynamically patches Zope, it stores a dictionary in 
the module
ZPublisher.Publish. The keys are integers, they are the values 
returned by Python's
thread.get_ident. The values are ZPublisher.HTTPRequest.HTTPRequest
instances, they are not acquisition wrappers.

Jean-François, which Localizer version do you use? Just to be sure we 
look at
the same code.

 


--
J. David Ibáñez
Founder and CTO of Itaapy http://www.itaapy.com
9 rue Darwin, 75018 Paris
Tel +33 (0)1 42 23 67 45 / Fax +33 (0)1 53 28 27 88 

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )


RE: [Zope-dev] Re: Preliminary findings: Zope 2.7 leakage caused by errors

2004-05-20 Thread Jean-Francois . Doyon
Juan David,

OK, 1.1.0a3 indeed suffers from the same problem, I'm going to take a closer
look the the publishing mechanism to see if I can find anything relevant.

Thanks,
J.F.

-Original Message-
From: J. David Ibáñez [mailto:[EMAIL PROTECTED]
Sent: May 20, 2004 10:17 AM
To: [EMAIL PROTECTED]
Cc: [EMAIL PROTECTED]; [EMAIL PROTECTED]; [EMAIL PROTECTED];
[EMAIL PROTECTED]
Subject: Re: [Zope-dev] Re: Preliminary findings: Zope 2.7 leakage
caused by errors



The a in 1.1.0a3 means alpha. If the problem is in Localizer
1.0.1 I think it will be in 1.1.0a3 too.

Yes, I have a site with a similar behaviour, unfortunately today
the network connection is too slooow, so I can't test that site. I'm
going to do some local tests, though.

Yes, the problem might be the fact that new_publish does not
catch exceptions to remove the request from the dictionary.



[EMAIL PROTECTED] wrote:

Juan David,

I'm using 1.0.1 ... I'm just abotu to try and test with 1.1.0a3 ...

BTW, does that mean it's alpha software or something? is 1.1.0 stable?

If the problem comes from Localizer I suspect this patch might be the
cause:

def new_publish(request, module_name, after_list, debug=0): 
id = get_ident() 
print Localizer got thread id:  + str(id)
Publish._requests[id] = request 
print Request dict is now:  + str(Publish._requests)   
x = Publish.old_publish(request, module_name, after_list, debug) 
try:
del Publish._requests[id]
except KeyError:
# XXX 
# Some people has reported that sometimes a KeyError exception is
# raised in the previous line, I haven't been able to reproduce it.
# This try/except clause seems to work. I'd prefer to understand
# what is happening.
LOG('Localizer', PROBLEM,
The thread number %s don't has a request object associated. %
id)
except:
print Localizer got an exception 
 
return x 

What happens if an exception occurs here:

x = Publish.old_publish(request, module_name, after_list, debug) 

??

If some clean up code is supposed to occur it might be held back by the
fact
that
there's a reference to the request kept in the global dictionary ...
At least that's my theory :)
But then I'm mostly uneducated about the intricacies of exception handling
at this
level, though I've been forced to learn quickly :)

I'd be curious to know at least if you can reproduce the leak by generating
loads
of errors on a Localizer enabled site?

One option would also be to simply wrap it in a try:catch and see what
happens there
... But I noticed 1.1.0a3 uses a different model for wrapping the request,
so that
just might do the trick.

Thanks for a very cool and useful product BTW.  This is combination with
the
TranslationService is saving me loads of time.

Cheers,
J.F.

-Original Message-
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]
Behalf Of J. David Ibáñez
Sent: May 20, 2004 6:47 AM
To: Tres Seaver
Cc: [EMAIL PROTECTED]; [EMAIL PROTECTED];
[EMAIL PROTECTED]; [EMAIL PROTECTED]
Subject: [Zope-dev] Re: Preliminary findings: Zope 2.7 leakage caused by
errors



Localizer (1.0.1) dynamically patches Zope, it stores a dictionary in 
the module
ZPublisher.Publish. The keys are integers, they are the values 
returned by Python's
thread.get_ident. The values are ZPublisher.HTTPRequest.HTTPRequest
instances, they are not acquisition wrappers.

Jean-François, which Localizer version do you use? Just to be sure we 
look at
the same code.


  



-- 
J. David Ibáñez
Founder and CTO of Itaapy http://www.itaapy.com
9 rue Darwin, 75018 Paris
Tel +33 (0)1 42 23 67 45 / Fax +33 (0)1 53 28 27 88 

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists -
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )


RE: [Zope-dev] Re: Preliminary findings: Zope 2.7 leakage caused by errors

2004-05-20 Thread Dieter Maurer
[EMAIL PROTECTED] wrote at 2004-5-20 09:58 -0400:
 ...
def new_publish(request, module_name, after_list, debug=0): 
id = get_ident() 
print Localizer got thread id:  + str(id)
Publish._requests[id] = request 
print Request dict is now:  + str(Publish._requests)   
x = Publish.old_publish(request, module_name, after_list, debug) 
try:
del Publish._requests[id]
except KeyError:
 ...

This code cannot leak requests in large numbers (though it may
leak a few requests for some time).

Reason: Zope does not normally create new threads. This implies
that thread_ids are reused and thereby old requests flushed from
the dict.

-- 
Dieter

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )


RE: [Zope-dev] Re: Preliminary findings: Zope 2.7 leakage caused by errors

2004-05-20 Thread Jean-Francois . Doyon
Indeed, I was hoping the ref in the dictionary there was getting in the way
when old_publish was raising an exception, but I tried using a
weakref.proxy()
instead and that didn't help.

I managed to test without Localizer (It dawned on me that since I'm looking
at errors, it really doesn't matter if I don't have Localizer installed
since
not having it will generate errors, which is what I want!) ... And the
problem
diappeared ...

So it's definitely Localizer, but it's not the Globals() request patch ...
Or
at least not the act of patching it itself, that causes the problem.

*sigh*
J.F.

-Original Message-
From: Dieter Maurer [mailto:[EMAIL PROTECTED]
Sent: May 20, 2004 2:58 PM
To: [EMAIL PROTECTED]
Cc: [EMAIL PROTECTED]; [EMAIL PROTECTED]; [EMAIL PROTECTED];
[EMAIL PROTECTED]; [EMAIL PROTECTED]
Subject: RE: [Zope-dev] Re: Preliminary findings: Zope 2.7 leakage
caused by errors


[EMAIL PROTECTED] wrote at 2004-5-20 09:58 -0400:
 ...
def new_publish(request, module_name, after_list, debug=0): 
id = get_ident() 
print Localizer got thread id:  + str(id)
Publish._requests[id] = request 
print Request dict is now:  + str(Publish._requests)   
x = Publish.old_publish(request, module_name, after_list, debug) 
try:
del Publish._requests[id]
except KeyError:
 ...

This code cannot leak requests in large numbers (though it may
leak a few requests for some time).

Reason: Zope does not normally create new threads. This implies
that thread_ids are reused and thereby old requests flushed from
the dict.

-- 
Dieter

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )