Re: [ZODB-Dev] Zope memory usage
Izak Burger-2 wrote: > > Dieter Maurer wrote: >> This is standard behaviour with long running processes on >> a system without memory compaction: > > Of course, I remember now, there was something about that in my > Operating Systems course ten years ago :-) I suppose the bigger page > sizes used on some architectures doesn't help. > > The zope instance in question is 2.10.5, which includes ZODB 3.7.1. Can > we simply swap that out with 3.8.0? Or should we rather do a svn diff > on the dm-memory_size_limited-cache branch (based on 3.7.0) and see if > that applies cleanly to 3.7.1 (I suspect it will)? > I'm using the 3.8 branch (that will become 3.8.1) for it's blob support happily with Plone 3.1 and Zope 2.10 Laurence -- View this message in context: http://www.nabble.com/Zope-memory-usage-tp19528989p19558656.html Sent from the Zope - ZODB-Dev mailing list archive at Nabble.com. ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] Zope memory usage
Dieter Maurer wrote: > This is standard behaviour with long running processes on > a system without memory compaction: Of course, I remember now, there was something about that in my Operating Systems course ten years ago :-) I suppose the bigger page sizes used on some architectures doesn't help. The zope instance in question is 2.10.5, which includes ZODB 3.7.1. Can we simply swap that out with 3.8.0? Or should we rather do a svn diff on the dm-memory_size_limited-cache branch (based on 3.7.0) and see if that applies cleanly to 3.7.1 (I suspect it will)? regards, Izak ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] Zope memory usage
Roché Compaan wrote at 2008-9-17 19:52 +0200:
>Thanks for the notice. We'll give this a go and report back.
>
>Do you know how exactly it is decided what stays in the cache?
The cache replacement strategy has not changed: "lru" ("least recently used").
Objects are removed from the cache in inverse "least recently used" order.
--
Dieter
___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/
ZODB-Dev mailing list - ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] Zope memory usage
Alan, I'm replying to the Zope list also, because this issue is perhaps related to other components there. I'm running into the same situation: The python process running my Plone site is steadyly growing. I'm using Zope2.9.8-final (the one which works with Plone 2.5.5). What is the plan for Zope include such feature? Best regards, Manuel. Alan Runyan wrote: > There was a recent modification to limit the ZODB cache to a set size. i.e. > Limit the size of memory usage to 128MB. > > The original feature was implemented here: > http://svn.zope.org/ZODB/branches/dm-memory_size_limited-cache/ > > You can get the feature+3.8 branch of the ZODB from: > http://svn.zope.org/ZODB/branches/zcZODB-3.8/ > > The changes are also on trunk (will be ZODB 3.9). > > The goal of the modification is to try to put upper bounds on the size of > the database cache. Before this "size limited cache" the cache would grow > and if you had very large objects in the zodb cache -- your memory usage would > be surpringsly high. > > Please use this feature in your testing at upfront. And let Roche know this > feature has landed (if we was not monitoring svn checkins). The feature needs > to be test driven. Give it a go and report your experience. > > cheers > alan > > > On Wed, Sep 17, 2008 at 5:10 AM, Izak Burger <[EMAIL PROTECTED]> wrote: >> Hi all, >> >> I'm sure this question has been asked before, but it drives me nuts so I >> figured I'll ask again. This is a problem that has been bugging me for >> ages. Why does zope memory use never decrease? Okay, I've seen it >> decrease maybe by a couple megabyte, but never by much. It seems the >> general way to run zope is to put in some kind of monitoring, and >> restart it when memory goes out of bounds. In general it always uses >> more and more RAM until the host starts paging to disk. This sort of >> baby-sitting just seems wrong to me. >> >> It doesn't seem to make any difference if you set the cache-size to a >> smaller number of objects or use a different number of threads. Over >> time things always go from good to bad and then on to worse. I have only >> two theories: a memory leak, or an issue with garbage collection (python >> side). >> >> It is possible that this is not a bug, my reasoning goes like this: The >> OS exposes a "virtual memory" picture to the application, ie, the >> application does not know how much of the RAM is real and how much is >> paged out. All it does is call malloc every now and then. Combined with >> a kernel that allows overcommit (which is in general a good thing), I >> see this going only one way. >> >> I'm hoping someone on this list has some insights into the issue. >> >> regards, >> Izak >> ___ >> For more information about ZODB, see the ZODB Wiki: >> http://www.zope.org/Wikis/ZODB/ >> >> ZODB-Dev mailing list - ZODB-Dev@zope.org >> http://mail.zope.org/mailman/listinfo/zodb-dev >> > > > ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] Zope memory usage
Izak Burger wrote at 2008-9-17 12:10 +0200: >I'm sure this question has been asked before, but it drives me nuts so I >figured I'll ask again. This is a problem that has been bugging me for >ages. Why does zope memory use never decrease? Okay, I've seen it >decrease maybe by a couple megabyte, but never by much. It seems the >general way to run zope is to put in some kind of monitoring, and >restart it when memory goes out of bounds. In general it always uses >more and more RAM until the host starts paging to disk. This sort of >baby-sitting just seems wrong to me. This is standard behaviour with long running processes on a system without memory compaction: It almost is a consequence of the "increased entropy" theorem. Memory tends to fragment over time. Some memory requests cannot be satisfied by the fragments (because the individual fragments are not large enough and compaction is not available) and therefore a new large block is requested from the operation system. >It doesn't seem to make any difference if you set the cache-size to a >smaller number of objects or use a different number of threads. Over >time things always go from good to bad and then on to worse. I have only >two theories: a memory leak, or an issue with garbage collection (python >side). The lack of compactions together with weaknesses in *nix memory management (*nix essentially provides "mmap" and "brk". "mmap" is not adequate for large numbers of small memory requests and "brk" can only allocate/release at the heap boundary). -- Dieter ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] Zope memory usage
Hi Alan Thanks for the notice. We'll give this a go and report back. Do you know how exactly it is decided what stays in the cache? -- Roché Compaan Upfront Systems http://www.upfrontsystems.co.za On Wed, 2008-09-17 at 10:26 -0500, Alan Runyan wrote: > There was a recent modification to limit the ZODB cache to a set size. i.e. > Limit the size of memory usage to 128MB. > > The original feature was implemented here: > http://svn.zope.org/ZODB/branches/dm-memory_size_limited-cache/ > > You can get the feature+3.8 branch of the ZODB from: > http://svn.zope.org/ZODB/branches/zcZODB-3.8/ > > The changes are also on trunk (will be ZODB 3.9). > > The goal of the modification is to try to put upper bounds on the size of > the database cache. Before this "size limited cache" the cache would grow > and if you had very large objects in the zodb cache -- your memory usage would > be surpringsly high. > > Please use this feature in your testing at upfront. And let Roche know this > feature has landed (if we was not monitoring svn checkins). The feature needs > to be test driven. Give it a go and report your experience. > > cheers > alan > > > On Wed, Sep 17, 2008 at 5:10 AM, Izak Burger <[EMAIL PROTECTED]> wrote: > > Hi all, > > > > I'm sure this question has been asked before, but it drives me nuts so I > > figured I'll ask again. This is a problem that has been bugging me for > > ages. Why does zope memory use never decrease? Okay, I've seen it > > decrease maybe by a couple megabyte, but never by much. It seems the > > general way to run zope is to put in some kind of monitoring, and > > restart it when memory goes out of bounds. In general it always uses > > more and more RAM until the host starts paging to disk. This sort of > > baby-sitting just seems wrong to me. > > > > It doesn't seem to make any difference if you set the cache-size to a > > smaller number of objects or use a different number of threads. Over > > time things always go from good to bad and then on to worse. I have only > > two theories: a memory leak, or an issue with garbage collection (python > > side). > > > > It is possible that this is not a bug, my reasoning goes like this: The > > OS exposes a "virtual memory" picture to the application, ie, the > > application does not know how much of the RAM is real and how much is > > paged out. All it does is call malloc every now and then. Combined with > > a kernel that allows overcommit (which is in general a good thing), I > > see this going only one way. > > > > I'm hoping someone on this list has some insights into the issue. > > > > regards, > > Izak > > ___ > > For more information about ZODB, see the ZODB Wiki: > > http://www.zope.org/Wikis/ZODB/ > > > > ZODB-Dev mailing list - ZODB-Dev@zope.org > > http://mail.zope.org/mailman/listinfo/zodb-dev > > > > > ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] Zope memory usage
There was a recent modification to limit the ZODB cache to a set size. i.e. Limit the size of memory usage to 128MB. The original feature was implemented here: http://svn.zope.org/ZODB/branches/dm-memory_size_limited-cache/ You can get the feature+3.8 branch of the ZODB from: http://svn.zope.org/ZODB/branches/zcZODB-3.8/ The changes are also on trunk (will be ZODB 3.9). The goal of the modification is to try to put upper bounds on the size of the database cache. Before this "size limited cache" the cache would grow and if you had very large objects in the zodb cache -- your memory usage would be surpringsly high. Please use this feature in your testing at upfront. And let Roche know this feature has landed (if we was not monitoring svn checkins). The feature needs to be test driven. Give it a go and report your experience. cheers alan On Wed, Sep 17, 2008 at 5:10 AM, Izak Burger <[EMAIL PROTECTED]> wrote: > Hi all, > > I'm sure this question has been asked before, but it drives me nuts so I > figured I'll ask again. This is a problem that has been bugging me for > ages. Why does zope memory use never decrease? Okay, I've seen it > decrease maybe by a couple megabyte, but never by much. It seems the > general way to run zope is to put in some kind of monitoring, and > restart it when memory goes out of bounds. In general it always uses > more and more RAM until the host starts paging to disk. This sort of > baby-sitting just seems wrong to me. > > It doesn't seem to make any difference if you set the cache-size to a > smaller number of objects or use a different number of threads. Over > time things always go from good to bad and then on to worse. I have only > two theories: a memory leak, or an issue with garbage collection (python > side). > > It is possible that this is not a bug, my reasoning goes like this: The > OS exposes a "virtual memory" picture to the application, ie, the > application does not know how much of the RAM is real and how much is > paged out. All it does is call malloc every now and then. Combined with > a kernel that allows overcommit (which is in general a good thing), I > see this going only one way. > > I'm hoping someone on this list has some insights into the issue. > > regards, > Izak > ___ > For more information about ZODB, see the ZODB Wiki: > http://www.zope.org/Wikis/ZODB/ > > ZODB-Dev mailing list - ZODB-Dev@zope.org > http://mail.zope.org/mailman/listinfo/zodb-dev > -- Alan Runyan Enfold Systems, Inc. http://www.enfoldsystems.com/ phone: +1.713.942.2377x111 fax: +1.832.201.8856 ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
[ZODB-Dev] Zope memory usage
Hi all, I'm sure this question has been asked before, but it drives me nuts so I figured I'll ask again. This is a problem that has been bugging me for ages. Why does zope memory use never decrease? Okay, I've seen it decrease maybe by a couple megabyte, but never by much. It seems the general way to run zope is to put in some kind of monitoring, and restart it when memory goes out of bounds. In general it always uses more and more RAM until the host starts paging to disk. This sort of baby-sitting just seems wrong to me. It doesn't seem to make any difference if you set the cache-size to a smaller number of objects or use a different number of threads. Over time things always go from good to bad and then on to worse. I have only two theories: a memory leak, or an issue with garbage collection (python side). It is possible that this is not a bug, my reasoning goes like this: The OS exposes a "virtual memory" picture to the application, ie, the application does not know how much of the RAM is real and how much is paged out. All it does is call malloc every now and then. Combined with a kernel that allows overcommit (which is in general a good thing), I see this going only one way. I'm hoping someone on this list has some insights into the issue. regards, Izak ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
