Eliot, Script?
Best, Phil On Fri, Mar 3, 2017 at 5:05 PM, Eliot Miranda <[email protected]> wrote: > Hi Phil, > > On Fri, Mar 3, 2017 at 5:04 AM, [email protected] < > [email protected]> wrote: > >> >> >> Le 3 mars 2017 13:56, "Clément Bera" <[email protected]> a écrit : >> >> >> >> On Fri, Mar 3, 2017 at 12:12 PM, Cyril Ferlicot D. < >> [email protected]> wrote: >> >>> On 03/03/2017 11:56, Clément Bera wrote: >>> > Hello everyone, >>> > >>> > This morning I investigated with Vincent Blondeau a problem reported by >>> > the Moose community a while ago: loading Moose model is slower in Spur >>> > (Pharo 5+) than in pre-Spur (Pharo 4 and older). In general, this >>> > problem was present for anyone growing images to a significant size. >>> > >>> > To investigate the problem, we loaded a 200Mb[3] Moose model on a 250Mb >>> > image, growing the image to 450Mb. Loading such a model takes 2 minutes >>> > in Spur and 1m30s in pre-Spur VMs. >>> > >>> > Using the stable Pharo VM, the analysis results were the following: >>> > - total time spent to load the Model: 2 minutes >>> > - time spent in full GC: 1 minute (4 fullGCs) >>> > - time spent in scavenges[1]: 15 seconds >>> > On the 2 minutes spent, we have 50% of the time spent in full GCs, >>> 12.5% >>> > in scavenges, 37.5% executing code. >>> > >>> > We then used the latest VM that features the new compactor (VM from >>> > beginning of March 2017 and over). The full GC execution time went down >>> > from 1 minute to 2 seconds. >>> > >>> > In addition, we increased the size of Eden[2] from 4Mb to 12Mb. Time >>> > spent in scavenges decreased from 15 seconds to 5 seconds. >>> > >>> > Overall, loading the model is now taking ~50 seconds instead of 2 >>> minutes. >>> > >>> > To increase Eden size, one needs to run a script similar to: >>> > >>> > | currentEdenSize desiredEdenSize | >>> > currentEdenSize := Smalltalk vm parameterAt: 44. >>> > desiredEdenSize := currentEdenSize * 4. >>> > Smalltalk vm parameterAt: 45 put: desiredEdenSize. >>> > >>> > _*And then restart the image.*_ >>> > >>> > I hope this report can be useful for some of you. I will try to make a >>> > blog post out of it, detailing other GC settings one can change from >>> the >>> > image to improve performance. >>> > _* >>> > *_ >>> > Best, >>> > >>> > Clement >>> > >>> > [1] A scavenge is basically the garbage collection of only young >>> objects >>> > [2] Eden is basically the space where objects are initially allocated. >>> > [3] All numbers in the report are order of magnitudes and not precise >>> > numbers >>> > >>> > >>> > >>> >>> Hi, >>> >>> This is great! We will probably try it soon on our models. >>> >>> Guillaume had a question also, what is the counterparty if we let the >>> EdenSize at this size when we are not loading/exporting a MSE? >>> >> >> There are 2 main counterparts: >> - You waste a bit of memory. If you increase from 4Mb to 12Mb, you waste >> 8Mb. >> - The user-pauses for scavenges may be more significant. >> >> There are customers using 64Mb Eden in production. It improves their >> performance, they do not care about wasting 60Mb on machine with 16Gb RAM >> and their application does heavy computation and does not need to be that >> responsive. >> >> However for UI applications (typically the IDE), the scavenge pauses may >> become significant enough to be noticed by the programmer. Maybe not at >> 12Mb, but certainly at 64Mb. >> >> For your case: >> - Do you care that your application use some more Mb of RAM ? >> - Do you care if your application takes a couple extra ms to answer a >> request ? (In any case, the full GC takes much more time and also delays >> the answer right now) >> If you don't care, you can use larger Eden. In any case, an Eden of 12 or >> 16 Mb should be safe. >> >> There are other settings that can be useful in your case. I will try to >> write a post about it. >> >> >>> Because in our case we deploy a server that might need to read some MSE. >>> We cannot restart it with our current solution. In that case it would be >>> good to have more info to select the best EdenSize for the server. >>> >>> Thank you! >>> >> >> I have images that do not shrink with the new compactor and as there are >> no leaks I am wondering if there is a way for me to diagnose why this >> happens. I am routinely loading 100MB XML in the image but these are only >> transient. >> > > First of all run the attached shell script on your image and report back > what the "freespace" field value us. In the old compactor this would > usually be tens to hundreds of kilobytes. With the new compactor this > should be zero or perhaps a few hundred bytes (and I suspect the few > hundred bytes should only show up in Newspeak images). > > >> Shrinking works with a Pharo6 image but not with a 5.0 based. Strange. >> > > Look at the vm parameters for growth and shrinkage. > > >> Anyone willing to pair over the internet to have a look? >> > > Yes, but sometime next week. > > >> It is annoying because I am providing those images to customers and an >> ever growing thing is not really great especially with a non standard tech >> I am trying to push forward there. >> >> Phil >> >> >>> -- >>> Cyril Ferlicot >>> >>> http://www.synectique.eu >>> >>> 2 rue Jacques Prévert 01, >>> 59650 Villeneuve d'ascq France >>> >>> >> >> > > > -- > _,,,^..^,,,_ > best, Eliot >
