Re: [android-building] Re: Soong confusion

2018-10-01 Thread John Kaye

If I'm integrating a large package (Samba, for example), I go through many 
clean and (re)build cycles until I get a working result. I would typically 
use (-B) for that. Is there a ninja way to clean a specific package/target 
rather than the entire build? Thanks, -John

On Tuesday, September 25, 2018 at 10:35:08 AM UTC-7, Colin Cross wrote:
>
> Ninja (which is what does the actual build command execution now) does not 
> support -B.  What is your use case?  Incremental builds should be reliable 
> enough that -B is not necessary for correctness.  I'd suggest touching the 
> relevant files if you want to force them to rebuild, for example:
> touch $(find . -name "*.java")
>
> On Tue, Sep 25, 2018 at 12:31 AM John Kaye  > wrote:
>
>>
>> I'm also a long-time pre-8.0 developer who has added many packages and 
>> heavily customized the framework. How does one unconditionally make (-B) 
>> all targets within a given package using the new build system?
>>
>> Thanks,
>> -John
>>
>>
>>
>> On Tuesday, December 12, 2017 at 2:08:35 PM UTC-8, Colin Cross wrote:
>>>
>>>
>>>
>>> On Mon, Dec 11, 2017 at 1:50 PM, Jacob Abrams  
>>> wrote:
>>>
>>>> Hello,
>>>>
>>>> I would like to voice protest over the AOSP build system switch from 
>>>> Make to Soong. Make is not a perfect tool but it is well documented and 
>>>> extremely stable. The introduction of ninja into AOSP was seamless and 
>>>> acceptable. However, migrating away from Make to a totally new tool with 
>>>> zero history is a serious set back. Why not instead choose Bazel, Tup, 
>>>> Gradle, Buck or simply stick with what works?
>>>>
>>>> I read the following statements from one of the developers of Soong 
>>>> build:
>>>>
>>>> > One of our goals for build health is to reduce the number of different
>>>> > ways we build modules.  Adding too many build flags makes it harder to
>>>> > tell if a change will break the build, and hard to run tests.  We
>>>> > would much rather compiling everything the same on all devices, and
>>>> > then determine which parts to use at runtime.
>>>>
>>>> It is unclear what is meant by "reduce the number of different ways we 
>>>> build modules.", nor is it clear what is meant by "We would much rather 
>>>> compiling everything the same on all devices". This seems to conflict with 
>>>> the example of LLVM where the build basically consists of completely 
>>>> custom 
>>>> go code: 
>>>> https://android.googlesource.com/platform/external/llvm/+/master/soong/llvm.go
>>>>
>>>> Clearly this custom go code does not reduce the number of different 
>>>> ways modules are built.
>>>>
>>>  
>>> Some teams have existing flows where they want to locally modify the way 
>>> they build, and we've supported those through the custom go code for those 
>>> modules.  In general we still try to avoid them.
>>>  
>>>
>>>> I assume this is an attempt to improve build performance yet again, but 
>>>> it ends up wasting thousands of engineering hours across the globe. 
>>>> Engineers must figure out a new system that likely contains numerous bugs 
>>>> and could possibly be destined for the dustbin if it is not maintained 
>>>> properly or turns out to be inferior. If the goal is to improve build 
>>>> performance perhaps Google engineers could explore an under-the-hood 
>>>> contribution to Make itself?
>>>>
>>>
>>> As Glenn pointed out, the purpose for Soong is not primarily 
>>> performance, it is correctness and reliability.  Before Soong (and the 
>>> conversion to Ninja was part of Soong), incremental builds were completely 
>>> unreliable, requiring significant knowledge of the internals of the Android 
>>> build for platform developers to get anything done.  Wiping the entire 
>>> output directory and rebuilding was common.  Incremental builds are now 
>>> reliable enough to be used in our continuous build infrastructure.
>>>
>>> Debugging typos in Android.mk files was also very painful.  LOCAL_CFALGS 
>>> instead of LOCAL_CFLAGS gets silently ignored, deleting a module that still 
>>> has users doesn't break incremental builds but breaks clean builds, 
>>> overwriting variables that are being used by other modules, subtle 
>>> differenc

Re: [android-building] Re: Soong confusion

2018-09-25 Thread John Kaye

I'm also a long-time pre-8.0 developer who has added many packages and 
heavily customized the framework. How does one unconditionally make (-B) 
all targets within a given package using the new build system?

Thanks,
-John



On Tuesday, December 12, 2017 at 2:08:35 PM UTC-8, Colin Cross wrote:
>
>
>
> On Mon, Dec 11, 2017 at 1:50 PM, Jacob Abrams  > wrote:
>
>> Hello,
>>
>> I would like to voice protest over the AOSP build system switch from Make 
>> to Soong. Make is not a perfect tool but it is well documented and 
>> extremely stable. The introduction of ninja into AOSP was seamless and 
>> acceptable. However, migrating away from Make to a totally new tool with 
>> zero history is a serious set back. Why not instead choose Bazel, Tup, 
>> Gradle, Buck or simply stick with what works?
>>
>> I read the following statements from one of the developers of Soong build:
>>
>> > One of our goals for build health is to reduce the number of different
>> > ways we build modules.  Adding too many build flags makes it harder to
>> > tell if a change will break the build, and hard to run tests.  We
>> > would much rather compiling everything the same on all devices, and
>> > then determine which parts to use at runtime.
>>
>> It is unclear what is meant by "reduce the number of different ways we 
>> build modules.", nor is it clear what is meant by "We would much rather 
>> compiling everything the same on all devices". This seems to conflict with 
>> the example of LLVM where the build basically consists of completely custom 
>> go code: 
>> https://android.googlesource.com/platform/external/llvm/+/master/soong/llvm.go
>>
>> Clearly this custom go code does not reduce the number of different ways 
>> modules are built.
>>
>  
> Some teams have existing flows where they want to locally modify the way 
> they build, and we've supported those through the custom go code for those 
> modules.  In general we still try to avoid them.
>  
>
>> I assume this is an attempt to improve build performance yet again, but 
>> it ends up wasting thousands of engineering hours across the globe. 
>> Engineers must figure out a new system that likely contains numerous bugs 
>> and could possibly be destined for the dustbin if it is not maintained 
>> properly or turns out to be inferior. If the goal is to improve build 
>> performance perhaps Google engineers could explore an under-the-hood 
>> contribution to Make itself?
>>
>
> As Glenn pointed out, the purpose for Soong is not primarily performance, 
> it is correctness and reliability.  Before Soong (and the conversion to 
> Ninja was part of Soong), incremental builds were completely unreliable, 
> requiring significant knowledge of the internals of the Android build for 
> platform developers to get anything done.  Wiping the entire output 
> directory and rebuilding was common.  Incremental builds are now reliable 
> enough to be used in our continuous build infrastructure.
>
> Debugging typos in Android.mk files was also very painful.  LOCAL_CFALGS 
> instead of LOCAL_CFLAGS gets silently ignored, deleting a module that still 
> has users doesn't break incremental builds but breaks clean builds, 
> overwriting variables that are being used by other modules, subtle 
> differences between := and =, or ifdef blah and ifneq(,$(blah)).  All of 
> these problems are fundamental to the way that Make works and can't be 
> fixed.
>
> We've explored various options with Make (for a while we had a modified 
> version of Make that would cache its build rules).  The conversion to Ninja 
> (and all of the speed and reliability improvements that came with it) was 
> done by using Kati instead of Make, and we've continued to invest in new 
> features there.  But most of the improvements have come from moving the 
> very complex build code out of the terrible Make language and into a high 
> level, maintainable, testable language.
>
> Android is mature; it deserves a mature build system.
>>
>> Jacob Abrams
>>
>> --
>> --
>> You received this message because you are subscribed to the "Android 
>> Building" mailing list.
>> To post to this group, send email to android-...@googlegroups.com 
>> 
>> To unsubscribe from this group, send email to
>> android-buildi...@googlegroups.com 
>> For more options, visit this group at
>> http://groups.google.com/group/android-building?hl=en
>>
>> ---
>> You received this message because you are subscribed to the Google Groups 
>> "Android Building" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to android-buildi...@googlegroups.com .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>
> On Mon, Dec 11, 2017 at 1:50 PM, Jacob Abrams  > wrote:
>
>> Hello,
>>
>> I would like to voice protest over the AOSP build system switch from Make 
>> to Soong. Make is not a perfect tool but it is well documented and 
>> extremely stable. The introduction of ninja into AOSP was seamless and 
>> acceptable.