Hi, Dan 在 2020年3月31日星期二 UTC+8上午1:07:08，Dan Willemsen写道： > > Interesting, 20 minutes still seems super excessive for that 😕 > > You'll probably save some amount of time sharing a single out directory > between devices, but not necessarily if you switch between them frequently, > as some of the device-specific files will clobber each other (this was > Soong trying to be smarter about sharing device-side code, until we came to > the point that we had too many configurations options to make that > reasonable 😞). And soong itself will re-run every time you switch devices > (kati may not, since we cache multiples of all of its outputs) > > The one thing that would help you (at least until that github issue is > resolved) is to move the output directory outside of your source directory: > OUT_DIR=../out, etc. Then the find emulator wouldn't discover it. > > Will try with your suggestion here with the use of OUT_DIR first, to see if the problem still there.
> Using a single output directory vs multiple output directories shouldn't > change the findemulator slowdown you're seeing as long as they're still in > the source directory. But the one thing a single output directory can slow > down is the initial ninja startup speed -- usually we expect to see that at > <=1s (depending on the machine), but that can grow if it needs to load > particularly large .ninja_log and .ninja_deps files, which will be shared > across all devices in a single output tree. It looks like yours is taking > ~4s based on the trace. So your minimum incremental build times are > probably ~2x as long as they could be. > > Thanks for all the kind explanation! Best Regards, Yongqin Liu > On Mon, Mar 30, 2020 at 7:58 AM Yongqin Liu <[email protected] > <javascript:>> wrote: > >> Hi, Dan >> >> Thanks for the explanation! >> >> 在 2020年3月29日星期日 UTC+8上午3:15:52，Dan Willemsen写道： >>> >>> So that time was mostly spent initializing the find emulator: >>> >>> verbose: *kati*: init find emulator time: 1311.781312 >>> >>> What that's doing is walking through your source tree and building up a >>> data structure to more efficiently (normally!) run `find` commands. This >>> can become very slow if you've added a symlink into your source tree >>> pointing to elsewhere on your filesystem (or a network filesystem). We've >>> also had reports that this can get very slow when you have a lot of output >>> trees under your source tree (being discussed in >>> https://github.com/google/kati/issues/184), but 20 minutes seems >>> excessive for that -- it claims that there are ~3M nodes in the cache: >>> >> >> Having a lot of output trees is my case. >> I build for following products under the same AOSP tree. >> beagle_x15/ >> db845c/ >> generic/ >> generic_arm64/ >> hikey/ >> hikey960/ >> vsoc_arm64 >> >> I thought that would help to save time on the host tools/java files >> building... >> >> verbose: *kati*: 3071468 find nodes >> >>> >>> One of my AOSP trees (after removing out/) is closer to 1M nodes (~2s), >>> and after a hikey960 build it has ~1.5M nodes (3.4s). >>> >> I checked log for another build which was done after out directory >> removed, the time is OK, >> >> Then what's your suggestion here: >> 1. use separate aosp tree for less product, like 2 or 3, my has 7 and >> cts, vts in the host out directory >> 2. I have not checked the details, and not verified it yet, do you think >> specifying the OUT out of the AOSP tree helps? still build multiply >> projects in the same out put directory. >> 3. I do not have data yet, just want to ask you here first, do you think >> build multiple products with the same output directory helps to save some >> time? >> maybe my thought that it saves time on building the host binaries >> and java objects was wrong:( >> >> >> Thanks, >> Yongqin Liu >> >> >> >> >>> On Sat, Mar 28, 2020 at 8:36 AM Yongqin Liu <[email protected]> wrote: >>> >>>> Hi, All >>>> >>>> Anyone knows what's the problem with the step of "including >>>> out/soong/Android-hikey960.mk ..." >>>> and how to work around this problem? >>>> >>>> it took about 20 minutes to include during the build, which is almost >>>> half of the entire build time. >>>> >>>> The build was an incremental build after did repo sync. >>>> >>>> here is some information from the out/verbose.log.gz >>>> >>>> out/soong/make_vars-hikey960.mk was modified, regenerating... >>>> verbose: *kati*: regen check time: 0.079827 >>>> verbose: *kati*: Stack size: 8376320 bytes >>>> [287/287] initializing build system ... >>>> verbose: *kati*: slow included makefiles (4.618866): build/make/core/ >>>> base_rules.mk >>>> verbose: *kati*: slow included makefiles (4.618982): build/make/core/ >>>> soong_cc_prebuilt.mk >>>> verbose: *kati*: slow included makefiles (4.604702): build/make/core/ >>>> base_rules.mk >>>> verbose: *kati*: slow included makefiles (4.605250): build/make/core/ >>>> soong_cc_prebuilt.mk >>>> verbose: *kati*: slow included makefiles (43.432355): >>>> out/soong/Android-hikey960.mk >>>> [288/491] including out/soong/Android-hikey960.mk ... >>>> verbose: *kati*: 3071468 find nodes >>>> verbose: *kati*: init find emulator time: 1311.781312 >>>> verbose: *kati*: slow included makefiles (1311.898633): art/build/ >>>> Android.cpplint.mk >>>> verbose: *kati*: slow included makefiles (1312.676836): art/Android.mk >>>> [289/491] including art/Android.mk ... >>>> >>>> >>>> For details, please check the attached build log. >>>> >>>> >>>> Thanks in advance! >>>> >>>> >>>> Best Regards, >>>> Yongqin Liu >>>> >>>> -- >>>> -- >>>> You received this message because you are subscribed to the "Android >>>> Building" mailing list. >>>> To post to this group, send email to [email protected] >>>> To unsubscribe from this group, send email to >>>> [email protected] >>>> For more options, visit this group at >>>> http://groups.google.com/group/android-building?hl=en >>>> >>>> --- >>>> You received this message because you are subscribed to the Google >>>> Groups "Android Building" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to [email protected]. >>>> To view this discussion on the web visit >>>> https://groups.google.com/d/msgid/android-building/eebd86b2-83c0-460e-a26d-6cd5af31fb20%40googlegroups.com >>>> >>>> <https://groups.google.com/d/msgid/android-building/eebd86b2-83c0-460e-a26d-6cd5af31fb20%40googlegroups.com?utm_medium=email&utm_source=footer> >>>> . >>>> >>> -- >> -- >> You received this message because you are subscribed to the "Android >> Building" mailing list. >> To post to this group, send email to [email protected] >> <javascript:> >> To unsubscribe from this group, send email to >> [email protected] <javascript:> >> For more options, visit this group at >> http://groups.google.com/group/android-building?hl=en >> >> --- >> You received this message because you are subscribed to the Google Groups >> "Android Building" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected] <javascript:>. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/android-building/5afd3044-8e12-4c7d-9878-d2408ff0283b%40googlegroups.com >> >> <https://groups.google.com/d/msgid/android-building/5afd3044-8e12-4c7d-9878-d2408ff0283b%40googlegroups.com?utm_medium=email&utm_source=footer> >> . >> > -- -- You received this message because you are subscribed to the "Android Building" mailing list. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/android-building?hl=en --- You received this message because you are subscribed to the Google Groups "Android Building" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/android-building/adb51fc2-4887-4867-a05b-f5cb5c89287e%40googlegroups.com.
