On 11 Apr 2014, at 7:33 am, Xavier Ducrohet <x...@google.com> wrote:
> Hey Adam, > > Long term improvements in this will be very welcome. > > In the short term do you have any idea what is taking some much time outside > of the configuration step and the dependency resolution? Looking at the > profile results there's 30 sec (or 2/3 of the time) that's unaccounted for. > This means that not creating the tasks (while useful) wouldn't have a huge > impact. Right now it's 8.4 second configuration out of 49 second. Even > halving that doesn't get us much. There are a few things that aren’t called out separately in the profile report. For example, starting or connecting to the daemon. Can you send me the profile report, and also the output of running the same build with -d? I’ll see if there’s anything obvious. > > > On Wed, Apr 9, 2014 at 3:23 PM, Adam Murdoch <adam.murd...@gradleware.com> > wrote: > > On 10 Apr 2014, at 1:07 am, Xavier Ducrohet <x...@google.com> wrote: > >> Hi, >> >> We have a large (for us?) projects with 122 sub-projects. With a warmed up >> daemon it takes about 49 seconds to run "gradlew projects". >> >> Trying to figure out what's taking time we ran it with --profile and the >> results were strange: >> >> - 1.6 seconds of startup/settings/buildSrc/loading up projects. >> - 8.4 seconds of project configurations >> - 6.9 seconds of dependency resolution >> - 0.1 seconds of task execution. >> >> This adds up to 17 seconds out of 49 seconds. >> >> My original intent was to see if I could improve configuration time by >> reducing the amount of work done during configuration and moving it to the >> execution phase, but it seems 2/3 of the time is spend somewhere else. >> >> Does it look like a typical profiling output? anything we can do to speed >> things up? (I tried with --parallel and got the same results) > > I’d say that’s pretty typical. Configuration time is a real problem once you > get a moderate number of projects involved (more than 20 or so, which is > pretty tragic). > > At this stage, there’s really only configure-on-demand that you can use to > make things faster. The —parallel option doesn’t affect configuration, only > task execution. Both these features are still somewhat experimental, because > of edge cases they don’t deal with very well, but they're both stable for the > common use cases. > > To make things faster, we need to make some pretty deep changes to the > configuration model. We’re working on these, but it’ll take time. > > There are a few parts to this. The main strategy is to roll configuration > rules out. This basically means that we’ll be able to pull the same kinds of > tricks we can for tasks: we’ll be able to configure only those things > (projects, source sets, variants, tasks, etc) that are required for whatever > the build is doing, and we’ll know the dependencies between things so we’ll > be able to do configuration in parallel. > > Longer term, the plan is that configuration rules will allow us to do > incremental configuration as things change, rather than reconfiguring things > from scratch each time. I think this will have a massive impact on IDE > integration, in particular. > > Another angle is to push configure-on-demand more, so that it allows parallel > configuration at the project level. > > Then there’s also making configuration more efficient. I suspect we could > find some low-hanging fruit for configuration and dependency resolution time > for your build. If you like, you could take some cpu and memory snapshots in > a profiler and we can see if there’s anything obvious there. > > > -- > Adam Murdoch > Gradle Co-founder > http://www.gradle.org > VP of Engineering, Gradleware Inc. - Gradle Training, Support, Consulting > http://www.gradleware.com > > Join us for Gradle Summit 2014, June 12th and 13th in Santa Clara, CA: > http://www.gradlesummit.com > > -- Adam Murdoch Gradle Co-founder http://www.gradle.org VP of Engineering, Gradleware Inc. - Gradle Training, Support, Consulting http://www.gradleware.com Join us for Gradle Summit 2014, June 12th and 13th in Santa Clara, CA: http://www.gradlesummit.com
