You can try "gradle androidDep" to see the dependency tree. On Jan 17, 2014 7:05 AM, "Ashton Cummings" <prolink...@gmail.com> wrote:
> It is a dependency spaghetti. =P > > It is a fairly deep dependency graph. With lots of libraries depending on > other libraries. > > I am going to put a tool together real quick to get you the dependency > tree. I will just rename all the projects and libraries. I will try to get > it to you today some time. > > Thanks for your continued interest and assistance, it is greatly > appreciated. > > On Thursday, January 16, 2014 3:55:10 PM UTC-6, Xavier Ducrohet wrote: >> >> (pre)dexing is a known issue. I'm not sure how much faster it's going to >> be in 0.8 with parallel dexing since you have a lot of projects and you can >> use parallelization there already. >> >> 18secs for a manifest merge seems like a crazy amount of time. do you >> have a lot of manifest data to merge? >> >> I'm going to build a fake multi-project setup with 100s of project to >> look at scaling issues. Can you tell me a bit more about your inter-project >> dependencies? Basically do you have 1 apps and 139 flat libs that don't >> depend on each other, or do you have a fairly deep dependency graph with >> lots of transitive dependencies? >> >> >> On Thu, Jan 16, 2014 at 12:09 PM, Ashton Cummings <proli...@gmail.com>wrote: >> >>> *"clean build" with no changes = 3h1m53.88s* >>> >>> Top 15 tasks (roughly the same for each project) >>> :projectA:preDexRelease 1m26.29s >>> :projectA:preDexDebug 1m25.14s >>> :projectA:processDebugManifest 18.452s >>> :projectA:processReleaseManifest 17.705s >>> :projectA:lint 10.699s >>> :projectA:mergeReleaseResources 8.264s >>> :projectA:dexDebug 7.742s >>> :projectA:dexRelease 7.449s >>> :projectA:mergeDebugResources 4.303s >>> :projectA:compileDebugJava 1.672s >>> :projectA:processReleaseResources 1.662s >>> :projectA:processDebugResources 1.325s >>> :projectA:compileReleaseJava 1.219s >>> :projectA:packageDebug 1.179s >>> :projectA:packageRelease 0.923s >>> >>> >>> *clean assembleDebug with pre-dex turned off = 58m58.29s* >>> >>> Top 15 tasks (roughly the same for each project) >>> :projectA:dexDebug 56.567s >>> :projectA:mergeDebugResources 5.135s >>> :projectA:processDebugManifest 3.517s >>> :projectA:compileDebugJava 1.579s >>> :projectA:packageDebug 1.174s >>> :projectA:processDebugResources 0.972s >>> :projectA:processDebugJavaRes 0.373s UP-TO-DATE :projectA: >>> prepareLibrary1UnspecifiedLibrary 0.340s >>> :projectA:prepareLibrary2UnspecifiedLibrary 0.218s >>> :projectA:compileDebugAidl 0.057s >>> :projectA:compileDebugRenderscript 0.046s >>> :projectA:prepareLibrary3UnspecifiedLibrary 0.045s >>> :projectA:prepareLibrary4UnspecifiedLibrary 0.043s >>> :projectA:prepareLibrary5UnspecifiedLibrary 0.029s >>> :projectA:prepareLibrary6UnspecifiedLibrary 0.028s >>> >>> I can not find my data for the parallel run. I will do another one of >>> those runs and post the data. Let me know if this is helpful for you. >>> >>> Thanks >>> >>> >>> On Thursday, January 16, 2014 1:46:45 PM UTC-6, Ashton Cummings wrote: >>>> >>>> I have been using --profile to figure out the bottlenecks. However, due >>>> to the nature of our project, i can not share that information. =/ >>>> >>>> I can give you times and stuff, just not project names and details. So, >>>> unless i modify the data substantially, i can not share the profile info. >>>> Sorry. >>>> >>>> However, i will give you the build time improvements and where the >>>> bottlenecks are. If you need any specifics, let me know. I will try and get >>>> you details that i can. >>>> >>>> I will get the info to you as soon as i can. I have so much profile >>>> data now, i am going to have to run through it and see if i can find the >>>> correct data. Might just do the runs again to make sure to get you the >>>> correct data. >>>> >>>> On Thursday, January 16, 2014 12:21:46 PM UTC-6, Xavier Ducrohet wrote: >>>>> >>>>> Do you remember the time gain you had when disabling pre-dexing? >>>>> >>>>> If you wouldn't mind. I'd love to get the output of running gradle >>>>> with --profile. It will create a report in the build folder of the main >>>>> project which will show me build time for all projects and all tasks in >>>>> all >>>>> projects. >>>>> >>>>> >>>>> On Thu, Jan 16, 2014 at 10:06 AM, Ashton Cummings >>>>> <proli...@gmail.com>wrote: >>>>> >>>>>> I hope that we can. Our alternatives that i am testing are Maven and >>>>>> ANT. Right now our maven approach is showing times that are comparable to >>>>>> our ANT. I personally want to stick with gradle, however, the decision is >>>>>> not up to me. I just give the higher ups my results and conclusions. =) >>>>>> >>>>>> I will keep an eye out for those performance improvements and will >>>>>> keep our people informed. >>>>>> >>>>>> Thank you for your time and assistance. If you figure out anything >>>>>> else or have any other suggestions, please continue to contact me. I >>>>>> really >>>>>> hope to convince them to move to gradle. >>>>>> >>>>>> >>>>>> On Thursday, January 16, 2014 11:13:32 AM UTC-6, Xavier Ducrohet >>>>>> wrote: >>>>>> >>>>>>> BTW, we are still a ways off from 1.0. I totally understand not >>>>>>> wanting to switch to Gradle right now (especially given your build >>>>>>> times), >>>>>>> but if you decide to not switch right now, please don't plan to stick >>>>>>> with >>>>>>> Ant for 2+ years (or some other random long duration). >>>>>>> The Gradle build system will see a lot of changes in the next 6 >>>>>>> months. I would recommend revisiting it mid-year at least :) >>>>>>> >>>>>>> >>>>>>> On Thu, Jan 16, 2014 at 9:10 AM, Xavier Ducrohet >>>>>>> <x...@android.com>wrote: >>>>>>> >>>>>>>> http://tools.android.com/tech-docs/new-build-system contains our >>>>>>>> changelog. >>>>>>>> >>>>>>>> We generally don't include Gradle specific changes in it even when >>>>>>>> we start requiring newer versions of Gradle, but performance fixes will >>>>>>>> probably be indicated in it because we know it's a pain point. >>>>>>>> >>>>>>>> >>>>>>>> On Thu, Jan 16, 2014 at 8:54 AM, Ashton Cummings < >>>>>>>> proli...@gmail.com> wrote: >>>>>>>> >>>>>>>>> Xavier, >>>>>>>>> >>>>>>>>> Thanks for the information. We are coming to a point now where the >>>>>>>>> decision to switch our build process is coming to an end. We would >>>>>>>>> really >>>>>>>>> like to move to gradle, hopefully you guys are able to get these >>>>>>>>> optimizations in soon. Once we make the decision, we are going to be >>>>>>>>> stuck >>>>>>>>> with it for a while. >>>>>>>>> >>>>>>>>> Where can i keep up to date with the exact changes that are going >>>>>>>>> into each build of the gradle plugin? I need to know when these >>>>>>>>> changes are >>>>>>>>> in place as soon as possible. >>>>>>>>> >>>>>>>>> Thanks >>>>>>>>> >>>>>>>>> >>>>>>>>> On Thursday, January 16, 2014 10:48:52 AM UTC-6, Xavier Ducrohet >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> Right now each project will pre-dex its dependencies on its own. >>>>>>>>>> This means 2 components depending on the same library will both run >>>>>>>>>> pre-dex >>>>>>>>>> on that library's classes.jar which is silly. We're looking at >>>>>>>>>> fixing this. >>>>>>>>>> >>>>>>>>>> Another thing we are looking at is parallelization at the task >>>>>>>>>> level. Right now Gradle only supports it at the projects level. >>>>>>>>>> It'll build >>>>>>>>>> 2 sub projects in parallel (if there are no dependencies between the >>>>>>>>>> two), >>>>>>>>>> but in a single project it won't parallelize its task. We are >>>>>>>>>> working with >>>>>>>>>> the Gradle developers to change this. >>>>>>>>>> >>>>>>>>>> Of course the tasks themselves can do their work in parallel, and >>>>>>>>>> most of our tasks do already. Pre-dexing doesn't though, so if a >>>>>>>>>> project >>>>>>>>>> has more than one dependencies it'll run pre-dex one by one. This is >>>>>>>>>> fixed >>>>>>>>>> in the next Gradle version. >>>>>>>>>> >>>>>>>>>> We do take performance very seriously, but we also wanted to >>>>>>>>>> provide a lot of features and flexibility in our build. This means >>>>>>>>>> we do >>>>>>>>>> some extra work. For instance our resource merger is extra work that >>>>>>>>>> aapt >>>>>>>>>> does in a way but differently (more efficiently in some ways, less in >>>>>>>>>> others, but with much less flexibility). We are working to do more to >>>>>>>>>> optimize each task and, as I mentioned above, provide generic >>>>>>>>>> performance >>>>>>>>>> improvements as well. >>>>>>>>>> >>>>>>>>>> I'm a bit surprised your build went from 1:20 to only 0:58 when >>>>>>>>>> running assembleDebug vs build. 'build' does both debug and release >>>>>>>>>> but >>>>>>>>>> also runs lint. It should be a lot more work than just >>>>>>>>>> assembleDebug. Also, >>>>>>>>>> do you have flavors? >>>>>>>>>> >>>>>>>>>> One other thing to consider is incremental build. Pre-dexing for >>>>>>>>>> instance makes clean build clearly slower but incremental builds >>>>>>>>>> faster. >>>>>>>>>> The rest of the build is also very good at only doing what needs to >>>>>>>>>> be done >>>>>>>>>> and nothing extra, which Ant wasn't very good at. >>>>>>>>>> >>>>>>>>>> Disabling pre-dexing on all projects without touching each >>>>>>>>>> project's build.gradle doesn't seem to be easy. The issue is that >>>>>>>>>> doing (in >>>>>>>>>> the main build.gradle): >>>>>>>>>> >>>>>>>>>> subprojects { >>>>>>>>>> /// some config >>>>>>>>>> } >>>>>>>>>> >>>>>>>>>> this gets applied before all the other projects' own build.gradle >>>>>>>>>> are applied. So unless you can apply the 'android' or >>>>>>>>>> 'android-library' in >>>>>>>>>> there, you won't be able to use our DSL. I need to look into this. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Thu, Jan 16, 2014 at 6:51 AM, Ashton Cummings < >>>>>>>>>> proli...@gmail.com> wrote: >>>>>>>>>> >>>>>>>>>>> My Stack post <http://stackoverflow.com/q/21125302/427763> >>>>>>>>>>> >>>>>>>>>>> Using gradle 1.9-rc-3 >>>>>>>>>>> When building with gradle on a multi-project setup containing >>>>>>>>>>> roughly 140 projects/libraries, the build time took 1 hour and 22 >>>>>>>>>>> minutes. >>>>>>>>>>> And i was using "--parallel". And our ANT build takes less than >>>>>>>>>>> 20 minutes without parallel building. >>>>>>>>>>> >>>>>>>>>>> Here is what i was initially doing: >>>>>>>>>>> >>>>>>>>>>> ./gradlew clean >>>>>>>>>>>> ./gradlew build --parallel >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> This took 1 hour and 22 minutes, roughly. I did a little testing >>>>>>>>>>> it seems like the dexing is taking the longest amount of time. Is >>>>>>>>>>> there a >>>>>>>>>>> way to get the gradle process to re-use the stuff it has already >>>>>>>>>>> dexed? If >>>>>>>>>>> the libraries have already been built, it should re-use the already >>>>>>>>>>> dexed >>>>>>>>>>> libraries. >>>>>>>>>>> >>>>>>>>>>> I saw the option --no-rebuild, but when i run with that option >>>>>>>>>>> it says the following: >>>>>>>>>>> >>>>>>>>>>>> File '/path/to/project/build/libs/project.aar' specified for >>>>>>>>>>>> property 'bundle' does not exist. >>>>>>>>>>>> >>>>>>>>>>>> I replaced the path and the project name with generic stuff. >>>>>>>>>>> >>>>>>>>>>> *** >>>>>>>>>>> >>>>>>>>>>> After some assistance i added "preDexLibraries = false" to all >>>>>>>>>>> of my "build.gradle" files. However, i still would like to know a >>>>>>>>>>> centralize place that i can put that entry and it affect all the >>>>>>>>>>> other >>>>>>>>>>> "build.gradle" files. Because having to add that to all of the >>>>>>>>>>> "build.gradle" files is annoying and if i ever have to remove it, >>>>>>>>>>> it will >>>>>>>>>>> be more annoying. This did not reduce the time to what we are >>>>>>>>>>> needing. >>>>>>>>>>> >>>>>>>>>>> After some more assistance, i ran "assembleDebug" instead of >>>>>>>>>>> "build". This way it will only do debug builds instead of both >>>>>>>>>>> release and >>>>>>>>>>> debug. This still did not bring us down to the time we are getting >>>>>>>>>>> with >>>>>>>>>>> ANT. With all these changes we are only down to 58 minutes. And our >>>>>>>>>>> ANT >>>>>>>>>>> process is less than 20 minutes. Our ANT process is what android >>>>>>>>>>> provides >>>>>>>>>>> only modified to support large multi-project setups and to re-use >>>>>>>>>>> dexed >>>>>>>>>>> libraries. The dex process in the gradle process is taking 1+ >>>>>>>>>>> minute for >>>>>>>>>>> each project. It is by far the largest time consumer. >>>>>>>>>>> >>>>>>>>>>> *** >>>>>>>>>>> >>>>>>>>>>> So, how can we speed the process up more? Is gradle re-using the >>>>>>>>>>> dexed libraries after they have been dexed? What are your >>>>>>>>>>> suggestions? For >>>>>>>>>>> us to move over to gradle, it has to be able to compete with the ANT >>>>>>>>>>> process. >>>>>>>>>>> >>>>>>>>>>> Thanks >>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> You received this message because you are subscribed to the >>>>>>>>>>> Google Groups "adt-dev" group. >>>>>>>>>>> To unsubscribe from this group and stop receiving emails from >>>>>>>>>>> it, send an email to adt-dev+u...@googlegroups.com. >>>>>>>>>>> >>>>>>>>>>> For more options, visit https://groups.google.com/groups/opt_out >>>>>>>>>>> . >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> Xavier Ducrohet >>>>>>>>>> Android SDK Tech Lead >>>>>>>>>> Google Inc. >>>>>>>>>> http://developer.android.com | http://tools.android.com >>>>>>>>>> >>>>>>>>>> Please do not send me questions directly. Thanks! >>>>>>>>>> >>>>>>>>> -- >>>>>>>>> You received this message because you are subscribed to the Google >>>>>>>>> Groups "adt-dev" group. >>>>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>>> send an email to adt-dev+u...@googlegroups.com. >>>>>>>>> For more options, visit https://groups.google.com/groups/opt_out. >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> Xavier Ducrohet >>>>>>>> Android SDK Tech Lead >>>>>>>> Google Inc. >>>>>>>> http://developer.android.com | http://tools.android.com >>>>>>>> >>>>>>>> Please do not send me questions directly. Thanks! >>>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Xavier Ducrohet >>>>>>> Android SDK Tech Lead >>>>>>> Google Inc. >>>>>>> http://developer.android.com | http://tools.android.com >>>>>>> >>>>>>> Please do not send me questions directly. Thanks! >>>>>>> >>>>>> -- >>>>>> You received this message because you are subscribed to the Google >>>>>> Groups "adt-dev" group. >>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>> send an email to adt-dev+u...@googlegroups.com. >>>>>> For more options, visit https://groups.google.com/groups/opt_out. >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> Xavier Ducrohet >>>>> Android SDK Tech Lead >>>>> Google Inc. >>>>> http://developer.android.com | http://tools.android.com >>>>> >>>>> Please do not send me questions directly. Thanks! >>>>> >>>> -- >>> You received this message because you are subscribed to the Google >>> Groups "adt-dev" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to adt-dev+u...@googlegroups.com. >>> For more options, visit https://groups.google.com/groups/opt_out. >>> >> >> >> >> -- >> Xavier Ducrohet >> Android SDK Tech Lead >> Google Inc. >> http://developer.android.com | http://tools.android.com >> >> Please do not send me questions directly. Thanks! >> > -- > You received this message because you are subscribed to the Google Groups > "adt-dev" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to adt-dev+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/groups/opt_out. > -- You received this message because you are subscribed to the Google Groups "adt-dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to adt-dev+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
