RE: [Zope] Memory management problems

2005-04-19 Thread Pascal Peregrina
My useless comment:

In such a real world scenario, anyone would put squid between Zope and the
millions of users. :)

Pascal

-Message d'origine-
De : [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] la part de
Peter Millar
Envoyé : mercredi 20 avril 2005 02:28
À : [EMAIL PROTECTED]; Zope
Objet : [Zope] Memory management problems


I have a serious memory problem.  And would like to hear from anyone who
has large image files being served up to a large number of simultaneous
user base.

I am running plone 2.0, zope 2.7.3, python 2.3.3, on windows 2003 server
with 1.5 GB RAM.

I have a 1MB image.  I have created a little script to copy and paste
this image.  To do it 50 times takes 6 minutes and uses up 300 MB of
RAM.  If I then view the folder that contains all the images and start
viewing the zpt that serves a single image it takes 40 seconds to a
minute 20 and chews up approx 2-5MB.  If I then get a few other machines
to request the same image at the same time, it takes 20seconds for the
first machine, 40 seconds for the next machine and 1 min for the next
machine and another 20 MB of ram has been chewed up.  Each time I go to
serve a page my CPU usage jumps to 50% for the minute or so.

After doing this over and over for a few minutes sees my machine has no
RAM left.  The only way to recover is to restart zope.

My question after all of this is how in the world can anyone seriously
use plone/zope in a real world scenario with potentially millions of
users accessing a site viewing content if you have to restart your zope
service every minute.

I have spent the past two days looking at all sorts of cacheing ideas,
session ideas etc.

Is there some magic that I need to apply for this to work like any
number of web image viewers.  Or is it a case of throwing 100GB of Ram
at the machine.

 
Peter Millar 



___
Zope maillist  -  Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
**   No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope-dev )


**
This email and any files transmitted with it are confidential and
intended solely for the use of the individual or entity to whom they
are addressed. If you have received this email in error please notify
the system manager.

This footnote also confirms that this email message has been swept by
MIMEsweeper for the presence of computer viruses.

www.mimesweeper.com
**

___
Zope maillist  -  Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
**   No cross posts or HTML encoding!  **
(Related lists -
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope-dev )


Re: [Zope] Memory management problems

2005-04-19 Thread Chris McDonough
Many things could be going on here.  I don't know of any outstanding
memory leaks in Zope itself.  There may be one in Plone or another
3rd-party product you're using.  Or you may have created one within your
own code.  Tracking down memory leaks is hard and it's not clear to even
tell you where to start.  Best advice is to use binary exclusion in the
template that serves up the page (comment out half of it, serve it up,
see if that leaks, if so, divide in half again, and so on).  Once you've
tracked it down, remediation is another thing altogether, however.

On Tue, 2005-04-19 at 20:27, Peter Millar wrote:
 I have a serious memory problem.  And would like to hear from anyone who
 has large image files being served up to a large number of simultaneous
 user base.
 
 I am running plone 2.0, zope 2.7.3, python 2.3.3, on windows 2003 server
 with 1.5 GB RAM.
 
 I have a 1MB image.  I have created a little script to copy and paste
 this image.  To do it 50 times takes 6 minutes and uses up 300 MB of
 RAM.  If I then view the folder that contains all the images and start
 viewing the zpt that serves a single image it takes 40 seconds to a
 minute 20 and chews up approx 2-5MB.  If I then get a few other machines
 to request the same image at the same time, it takes 20seconds for the
 first machine, 40 seconds for the next machine and 1 min for the next
 machine and another 20 MB of ram has been chewed up.  Each time I go to
 serve a page my CPU usage jumps to 50% for the minute or so.
 
 After doing this over and over for a few minutes sees my machine has no
 RAM left.  The only way to recover is to restart zope.
 
 My question after all of this is how in the world can anyone seriously
 use plone/zope in a real world scenario with potentially millions of
 users accessing a site viewing content if you have to restart your zope
 service every minute.
 
 I have spent the past two days looking at all sorts of cacheing ideas,
 session ideas etc.
 
 Is there some magic that I need to apply for this to work like any
 number of web image viewers.  Or is it a case of throwing 100GB of Ram
 at the machine.
 
  
 Peter Millar 
 
 
 
 ___
 Zope maillist  -  Zope@zope.org
 http://mail.zope.org/mailman/listinfo/zope
 **   No cross posts or HTML encoding!  **
 (Related lists - 
  http://mail.zope.org/mailman/listinfo/zope-announce
  http://mail.zope.org/mailman/listinfo/zope-dev )
 

___
Zope maillist  -  Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
**   No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope-dev )