On 18/01/2011, at 7:48 PM, Andrew Schetinin wrote: > Hi, > > A minor note - one of the items in the plans for 1.0 is to make the Gradle > daemon being a default. > > In my experience, this daemon process, while really improving the execution > speed, is very memory-consuming - it was sitting there in background and > eating more than 400 MB of memory. I know Gradle in general is memory > intensive, but there is a difference between consuming 20% of RAM (in my > case) for a minute while building, and consuming the same amount forever - > other applications are more likely to be pushed to swap. > > I'm not complaining, don't get me wrong :-) I'm just pointing out that there > is a kind of trade-off between memory and speed, and I'd at least warn a user > about it in the documentation, or on daemon startup.
This is a good point. At this stage, we haven't done any memory profiling of the daemon, so there are almost certainly some improvements we can do there. There are a bunch of things we might do: * We do cache quite a bit of stuff in memory. Most of the code that does this dates from before the daemon existed, and so assumes that the caching is only for the lifetime of a single build and never bothers to clean anything up. So, we might have the daemon progressively clean these caches out, based on the size of the cache, the likelihood of it being used in the future, available memory on the machine, etc. * We do load some (in theory) unbounded collections into memory. We might instead spool these to the file system when they exceed a certain size, to keep the peak heap size down. * Split the work across multiple processes. This way, the peak usage may be higher, but we will be able to progressively expire worker processes to quickly release the memory back to the system. For example, we might do compilation in a second process, which we can reuse multiple times, but can kill off when memory is tight. Or, we might give you some way to specify that a task should execute in a forked process, so that its heap is reclaimed quickly. For example, javadoc or docbook generation might happen in a forked process which we throw away as soon as it is finished with. * Move some in-memory storage out of the heap, and into non-heap buffers, so we can release the memory quickly when we're finished with it. There might be some interesting opportunities to share this between worker processes, too. * Build remotely, or across multiple machines, so that you can use some other machine's memory instead of your own machine's. * Add some way to specify the command-line args for the daemon, for the cases where we get the above wrong. -- Adam Murdoch Gradle Developer http://www.gradle.org CTO, Gradle Inc. - Gradle Training, Support, Consulting http://www.gradle.biz
