Hi, applied to the scratchpad. Own config is missing. I have to go to bed now ;)...
TIA ~Gerhard "Sorry, but my karma just ran over your dogma." >-----Original Message----- >From: Peter Hargreaves [mailto:[EMAIL PROTECTED]] >Sent: Tuesday, February 05, 2002 1:36 AM >To: [EMAIL PROTECTED] >Subject: Re: StoreJanitor new calculation [was: Re: Store Janitor Hangs >System] > > >Hi Gerhard, > >Great to hear that I've not been wasting my time :-) > >Please find attached the updated code. > >[EMAIL PROTECTED] wrote: > >> Peter, >> >> >>>From: Peter Hargreaves [mailto:[EMAIL PROTECTED]] >>> >>>Hi Gerhard, >>> >>>I've tested your latest Store Janitor improvements (using Optimizit and >>>your debug messages) and although its a very great improvement, I've >>>found some other problems :-( >>> >> >> Damn... >> >> >>>As my Optimizit trial was running out I decided to do some urgent work >>>on the Checker and now have a solution which appears to work well :-) >>> >>>Here is a summary of the problems with ideas for correcting them (some >>>are nitty gritty, and some you already know): >>> >>>*) When demand for memory is high, it is possible to run out of >>>java.memory before the store has been emptied. Might it be possible to >>>make sure the stores are empty before the memory can run out? >>> >> >> Then we don't need the StoreJanitor, or? >> >> >>>*) When memory is low, there is a risk of running out of it before the >>>next check. Why not set the interval short enough to ensure there is no >>>risk of running out of memory before the next check? >>> >> >> Be careful with Threads. When they run in a to short interval they'll >> kill your performance! >> >> >>>*) Calling the gc once every check interval seriously undermines system >>>performance because it can take several seconds to complete, which is >>>comparable to the check interval. Why not call it only when some items >>>are released from a store? Why not trust it and use its characteristics >>>to advantage? >>> >> >> That's a contradiction to previous point, I guess. On the one hand you want >> to increase the the iteration of the Thread on the other hand you want to >> decrease the GC calls. > > >Ah, but only call the GC if stores are reduced - I think you got it >lower down the page. > > >> >> >>>*) When several instances of Cocoon are running (independent .WAR apps >>>in the same servlet container), the calls to gc become very dominant. >>>Would this affect the strategy for store-janitor settings? >>> >>>*) If the percent to reduce storage is set to 10%, it fails to remove >>>any when the number of items are below 10. The number of items to be >>>removed needs rounding upwards. Why not remove a fixed number of items >>>instead of a percentage? (My idea and now I think it was wrong!!) >>> >> >> Au, my embarrsing math capabilities strikes agein ;). >> >> >>>*) It is difficult to get a good picture of what is happening by >>>observing the debug output, so it is difficult to know if it is working >>>and difficult use the debug output to optimise choice of janitor >>>settings. Might it be possible to make this easy? >>> >> >> Yes and no. That's always a problem in Cocoon. But you can define your >> own log target (own file) in the logkit.xconf for the StoreJanitor. Then >> you get rid of the noise. > > >I suspected something like this might be possible - sounds great! > > >> >> >>>The proposed, implemented and tested solution to the above works as follows: >>> >>>1) Monitor the rate at which memory is consumed during each check >>>interval. Remember the biggest rate that has been found so far, to use >>>it later. My system logs this at 2.5 megabytes per second. >>> >> >> Understand. >> >> >>>2) If heapIsBig() && freeIsLow(), then attempt to free storage. >>> >>>2.1) 'heapIsBig' means that the total memory has exceeded the parameter >>>'maxheap'. >>> >>>2.2) 'freeIsLow' means the free memory is less than the parameter 'minfree'. >>> >>>2.3) Attempt to remove from the stores, the number of items specified by >>>the parameter 'reduceby'. >>> >>>2.4) To free storage, start at the next store and remove items, moving >>>to the next again if necessary, until the specified number of items have >>>been removed or all the stores are empty. >>> >>>2.5) Then call the garbage collector, but only if some memory items were >>>freed. >>> >> >> Ok, that's nice. >> >> >>>3) Sleep for an interval half that in which the memory could run out. >>> >>>3.1) If the remaining heap could be more than half filled during the >>>interval specified by the 'maxsleep' parameter (at the max rate of >>>memory consumption) then sleep for the time it would take to only half >>>fill (freememory/maxrate)/2. Otherwise sleep for the max sleep interval. >>> >> >> Damn, that's sounds great ;). Your biggest unknown variable is the >> "could be". But I think taking the last biggest consumption is not that >> bad. But how about taking the average of all calls. That would be more exact >> then the last biggest value. > > >You got straight to the weakest point didn't you? My fear is that a >single rogue ridiculously high value might get stuck in there - I think >that's possibly what you are getting at. > >Can't average all the rates, because you need only the periods where >memory is trying to increase as fast as it can. Which ones are these? > >Could possibly, average any new highest value with the current highest. >Or average two of the current one with the new one (i.e. give a greater >weight to the current one). > >So, for the moment I've changed it to average the new highest rate with >the old - instead of just replacing it. I'm still thinking about this one. > > >> >>>The effect of the store-janitor parameters in cocoon.xconf are now >>>slightly changed, so I've changed the names of the parameters to reflect >>>this. For the moment the names are changed only inside >>>StoreJanitorImpl.java. If the solution were to be adopted then the names >>>in cocoon.xconf would also change. The changes, with notes, are as follows: >>> >> >> Naming was indeed crap. >> >> >>>'maxheap' (was 'heapsize') The maximum total heap size below which >>>stores are guaranteed to remain intact. >>> >>>'minfree' (was 'freememory') The minimum free heap required for stores >>>to remain intact. I have found this should be set to less than 10% of >>>avaliable memory, so that the jvm will respond to low memory by >>>allocating more heap (if its available), before stores are likely to be >>>reduced. Default value might be 2000000. Setting it to zero or very low >>>will effectively disable store reduction. >>> >>>'maxsleep' (was 'cleanupthreadinterval') The maximum interval between >>>checks. Should be short enough to ensure that 'rate of change' data is >>>collected before free memory runs low. Suggest 5 secs as default. >>> >>>'reduceby' (was 'percent_to_free') is the number of items to be removed >>>>>from the stores, on each attempt to reduce the stores. Each successfull >>>attempt to reduce stores results in a call to the garbage collector >>>taking several seconds. This limits the rate a which the stores can be >>>emptied. Removing several items at a time compensates for this. Using >>>the debug output, choose a number of items which results in an increase >>>of free memory between successive store reductions. Best tested with a >>>sudden high demand for memory when the stores are full. In practice this >>>setting is not critical. Suggested default is 10 items. Ideally >>>'reduceby' would be in bytes! >>> >>>Setting 'minfree' and 'reduceby' both too low can intensify gc activity >>>if sudden high demand requires all stores to be completely freed and can >>>result in memory running out before all the stores are free. Verify >>>using debug output. >>> >>>'priority' remains unchanged. >>> >>>The above solution is implemented and tested and it works very well >>>indeed for me. The parameters do not seem to be at all critical - thank >>>goodness! I see no reason why the default settings would need to be >>>changed if more memory were available. I have attached the >>>StoreJanitorImpl.java so others can test it. I find I can get a very >>>good picture of what is happening by using my editor's repeat find >>>command on the debug output in the core.log.000001 file. Optimizeit is >>>no longer necessary to prove that it works effectively. >>> >> >> Just define as mentioned above you own log target for the store.janitor. >> Take a look into logkit.xconf in the WEB-INF dir! >> >> >>>Beware! - there might still be things I don't know about that I have not >>>take into account. >>>Ideally - low memory should be detected by some sort of interrupt or >>>exception rather than by polling. >>> >> >> That's correct. But here java is a big white spot. >> >> >>>If you are subsequently interested in committing these changes into >>>Cocoon and you want me to do some more work to conform to design >>>practice etc., then please let me know. >>> >> >> Yep of course. But I switch to my picky mode now. Please reformat your >> code to our coding standards. >> - 4 indents, no tabs. >> - if(foo) >> this.isBad = true; >> That's not good coding style. Use brackets for each condition or block. >> - Make it more readable > > >O.K. done that. It now makes a thrilling read! I'm not sure about a >sequel though. > > >> >> Then I will commit your stuff into the Scratchpad area. >> (That's the normal way this days) >> > > >Peter. > --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, email: [EMAIL PROTECTED]
