You can cut down on the spikes now. Just add another line to the cache.ccf wherever you are using the shrinker.
By default it is set to -1, or no limit. If you set this at a few hundred, you should fair better. jcs.region.YOURREGIONNAME.cacheattributes.MaxSpoolPerRun=250 The indexed diskcache now has a maximum number of keys that it will hold. By default it is 500,000. You can set it lower with this parameter in your diskcache defintion: jcs.auxiliary.DC.attributes.maxKeySize=10000 I'm using the commons LRUMap for now. When it initializes to allow for a huge maximum number, say a million, it eats memory. Set it at 10k or so, in most cases you shouldn't need more than that for a single region. This is only part of a longterm disk cache plan of improvement. Cheers, Aaron > -----Original Message----- > From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] > Sent: Sunday, May 09, 2004 5:04 PM > To: Turbine JCS Users List > Subject: RE: Out of memory problem > > This sounds plausible. > > Running the profiler on our production machine for 30 minutes showed heavy > memory spikes but no leakage when the memory shrinker was enabled (After > a successfull GC it always seemed to go back to a normal level). I also > checked the memory shrinker code manually, I didn't dig too deep into the > spooling but suspected this might be causing problems simply because the > rest of the memory shrinker code was so simple. > > I would have liked to run the profiler for a longer period but Jprofiler > I was using is not stable enough to leave unattended on a production > system > (it crashes the jvm sometimes). So I was unable to confirm if it was a > slow > leak or sudden spike. I have seen other libraries OutOfMemory due to heap > usage spikes where we previously suspected a leak, so it's not uncommon > (in an XML processing library). > > Good luck anyway, > Matt. > > >-- Original Message -- > >Reply-To: "Turbine JCS Users List" <[EMAIL PROTECTED]> > >Date: Sun, 09 May 2004 15:15:05 -0500 > >From: Aaron Smuts <[EMAIL PROTECTED]> > >Subject: RE: Out of memory problem > >To: 'Turbine JCS Users List' <[EMAIL PROTECTED]> > > > > > >I've been profiling JCS for days. I still have more work to do, but I > >have a few ideas. > > > >The shrinker does not maintain references to any objects. However, > >there is a potential issue involved in using the shrinker, namely, it > >causes sudden spikes in memory usage. > > > >The main purpose of the shrinker is to find elements that have exceeded > >their maximum idle memory time and then to spool these to disk. It also > >checks for expired elements in the memory cache and removes them. > > > >When the shrinker finds elements to spool it initiates the spooling > >process. This invovles putting the items in purgatory and then they are > >queued to be written for disk. The disk writing can take a while if you > >have thousands of items. The shrinker will try to spool every item in > >memory that has exceeded the maxmemory time. Basically, this can result > >in thousands of items in the queue which results in a memory spike that > >can exceed the maximum heap size. All my tests indicate that the memory > >is reclaimed. > > > >I will add a configuration option to set the maximum number of items to > >spool per run of the shrinker. This way the size of the spike can be > >minimized. > > > >There is a potential memory issue invovled in using the indexed disk > >cache. The disk cache stores the keys in memory. If an expired element > >is pulled from the disk cache or an item is removed from the cache, the > >key gets removed from the set held in memory by the disk cache. > >However, if you create millions of entries in the cache that you never > >use, many of these get pushed onto the disk, and they don't get removed, > >the in memory key size of the disk cache can grow indefinitely. > > > >This would most likely be the result of improper cache usage, since you > >shouldn't cache something that won't likely be used again. However, > >there is a high usage scenario that could result in massive disk key > >memory usage. Let's assume that you cache objects that might not get > >used very frequently, and you set them to expire after a short duration, > >let's say 30 seconds. If that item gets spooled to disk within 30 > >seconds, because either you have a small max memory size or extremely > >high usage, and there is never a get for that item, then the key will > >sit in disk memory forever or until a get. If there is a get it will be > >found on disk, checked for expiration, and then removed, but if there is > >no get, it will just hang around. > > > >For now, the way to prevent this is to increase the memory size and > >shrink the max life time. Alternatively, you can disable the disk cache > >if most of your items are of this type. > > > >In the future some sort of disk cleanup should occur, or the key sotrage > >for the disk cache could use a LRU algorithm that would limit it size. > >I think this is actually the best option. I will do something like this > >soon. > > > >I can't find any actual memory leaks, only the spike from the shrinker > >and the potential key bloat in the disk cache. > > > >I'm still hunting for more. > > > >Aaron > > > > > > > > > >> -----Original Message----- > >> From: Estefano Eduardo [mailto:[EMAIL PROTECTED] > >> Sent: Friday, May 07, 2004 2:43 AM > >> To: Turbine JCS Users List > >> Subject: RE: Out of memory problem > >> > >> We also started getting out of memory errors. Same situation, only > >after > >> we started using the jcs library (I have latest copy from main trunk). > >> > >> We did not pin point the error yet though. It did not happen straight > >> away, only after the server was running for a few days. The solution > >is > >> to restart the server at the moment. We only use memory cache and we > >do > >> use schrinkers on some regions. > >> > >> -----Original Message----- > >> From: Aaron Smuts [mailto:[EMAIL PROTECTED] > >> Sent: Thursday, 06 May, 2004 19:52 > >> To: 'Turbine JCS Users List' > >> Subject: RE: Out of memory problem > >> > >> > >> Matthew informs me that the problem seems to go away with the shrinker > >> off. > >> > >> This suggests a problem with the memory shrinker. It could be that > >the > >> thread is maintaining a reference to the keys, the spooling from the > >> shrinker is problematic, or there is a problem generating the list of > >> keys. > >> > >> I don't think any of these things is happening, so I'm sort of at a > >loss > >> for an explanation. > >> > >> I think the shrinker just sleeps for an interval. Perhaps it would be > >> better to use a timer and start the shrinker and then let it die. One > >> concern with this is that we wouldn't want more than one shrinker > >> running per region at any given time. This could happen say if the > >> number of elements was huge and the interval between shrinks was very > >> small. I can solve that by passing in a lock that will prevent a > >second > >> shrinker from running. > >> > >> Hmmn. I'll look into it again. > >> > >> Aaron > >> > >> > -----Original Message----- > >> > From: Aaron Smuts [mailto:[EMAIL PROTECTED] > >> > Sent: Thursday, May 06, 2004 9:43 AM > >> > To: 'Turbine JCS Users List' > >> > Subject: RE: Out of memory problem > >> > > >> > Send your cache.ccf and let me know how much memory you've allocated > >> for > >> > the jvm. > >> > > >> > Aaron > >> > > >> > > -----Original Message----- > >> > > From: Parvathi Rajaraman [mailto:[EMAIL PROTECTED] > >> > > Sent: Thursday, May 06, 2004 1:53 AM > >> > > To: '[EMAIL PROTECTED]' > >> > > Subject: Out of memory problem > >> > > > >> > > Hi, > >> > > > >> > > I have recently started using JCS implementation for Caching in my > >> > > project. I have noticed that every few minutes, I keep getting a > >> > > out-of > >> memory > >> > > error > >> > > and this has happened only after I have started using JCS. So is > >> this > >> > > something to do with the way I have configured it. Can you help me > >> > out. > >> > > > >> > > Regds > >> > > Parvathi > >> > > > >> > > > >> --------------------------------------------------------------------- > >> > > To unsubscribe, e-mail: > >> > [EMAIL PROTECTED] > >> > > For additional commands, e-mail: > >> > [EMAIL PROTECTED] > >> > > >> > > >> > > >--------------------------------------------------------------------- > >> > To unsubscribe, e-mail: > >> [EMAIL PROTECTED] > >> > For additional commands, e-mail: > >> [EMAIL PROTECTED] > >> > >> > >> --------------------------------------------------------------------- > >> To unsubscribe, e-mail: > >[EMAIL PROTECTED] > >> For additional commands, e-mail: > >> [EMAIL PROTECTED] > >> > >> > >> --------------------------------------------------------------------- > >> To unsubscribe, e-mail: > >[EMAIL PROTECTED] > >> For additional commands, e-mail: > >[EMAIL PROTECTED] > > > > > >--------------------------------------------------------------------- > >To unsubscribe, e-mail: [EMAIL PROTECTED] > >For additional commands, e-mail: [EMAIL PROTECTED] > > > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
