On 05/12/2011, at 5:28 PM, Kris De Volder wrote: > ----- Original Message ----- >> >> >> >> >> On 02/12/2011, at 3:33 PM, Peter Niederwieser <[email protected]> >> wrote: >> >>>> I don't necessarily have a problem with defaults if they are >>>> documented >>>> and explained to the user via a good error message if the daemon >>>> OOMs. >>>> >>> >>> I'm more concerned about the opposite: Wasting memory by setting >>> high >>> defaults in an attempt to prevent OOMs for most users. >> >> I don't think this is much of an issue. The OS will page out the >> unused JVM heap. > > Sorry but I beg to differ. That's certainly not my experience (on Linux). As > soon as I see any memory spilling over into swap, performance really goes > down the drain fast. I don't think the JVM heap is really all that > friendly/amenable to virtual memory management. > > Not to mention that if swap is in use, you don't get much memory being > allocated to OS-level file caches, I notice that this has a major impact on > startup times for something like Eclipse / STS (for re-starts, initial start > isn't impacted as much). > > I didn't do any systematic testing, this is just from casual observation > while working with Gradle/Eclipse and keeping an eye on my memory usage. > > Nevertheless, please don't make light of memory usage thinking that 'it is > free', it really is not free.
The issue at hand is how to think about default settings for memory usage, particularly heap. If we are going to provide defaults, we'll have to either be conservative to avoid the JVM using too much memory or be more generous to allow the defaults to be more useful for more builds. My contention was that we should err on the side of generousness. The daemon will use little heap when idle. Then it will receive a job, memory usage will grow while it performs the build, then the build will finished and it will go back to very little heap use when idle (ignoring leaks). So using a more generous max heap comes in to play when you run a large build, then run smaller builds after that. Since no JVM gives unused heap back to the OS after it has claimed it, we've now got this memory for the life of the daemon which could be all day if the user runs a build every couple of mins. So what happens to this large chunk of memory we've claimed from the OS but don't go near again? My point was that the OS will probably page it out because we don't touch it. This is assuming quite a bit though. At no point did I say memory was free. My contention was that, given the memory profile of the daemon, we should be generous instead of conservative as being able to do more builds without having to change the defaults is more appealing than being memory conservative because of paging. >> Side thought: I wonder if we should call System.gc() after a daemon >> build. > > No... calling gc is probably a bad idea. Especially if swap is being used. It > will cause massive paging. I can't find it online, but I am sure I read at one point that it's a good idea to call System.gc() in a Swing app when the user minimises the app as it's a hint that the user is no longer using the app so it's a good time to spend cycles on gc. This is the same usage pattern here. We'd call it after the user has finished interacting with the daemon so it would in no way affect user experience. Theoretically, it may improve performance for the next build because it may prevent GC happening in the middle of the build where we could have done it when the daemon was idle. Again, a lot of assumptions though. Perhaps we should just leave it alone. -- Luke Daley Principal Engineer, Gradleware http://gradleware.com --------------------------------------------------------------------- To unsubscribe from this list, please visit: http://xircles.codehaus.org/manage_email
