Re: [gradle-dev] Incremental builds and ASLR

[Rene Groeschke]

Adam Murdoch 26 Feb 2014 05:51 You sure that’s the case? ASLR is applied at execution time, not compile time (it’s completely pointless to do this at compile time). There are apparently some bugs in older versions of Clang that prevent it producing deterministic output, which looks more likely the problem we’re hitting based on that diff you posted above. The clang version that comes with Ubuntu 12.04 was affected, which means any test running on agent1 - agent3 is probably going to fail some times. Have you seen this test fail on the build vms? I know there where bugs fixed in clang related to deterministic builds, but none of them seem to be related to the issue I have seen. I've tested this with also on the build vms with the same behaviour. I ran tests compiling the same source 10 (20) times and got different kinds of object files generated. When disabling ASLR by calling "setarch `uname -m` -R /bin/bash upfront, this test passes and always generates exactly the same object file (on ubuntu 12.04 and 13.10). The test I used for this can be found here: https://github.com/gradle/gradle/blob/master/subprojects/cpp/src/integTest/groovy/org/gradle/nativebinaries/language/objectivec/ObjectiveCLanguageIncrementalBuildIntegrationTest.groovy#L51 Best regards, Rene -- Rene Groeschke Principal Engineer, Gradleware Inc. - Gradle Training, Support, Consulting rene.groesc...@gradleware.com http://gradleware.com -- Adam Murdoch Gradle Co-founder http://www.gradle.org VP of Engineering, Gradleware Inc. - Gradle Training, Support, Consulting http://www.gradleware.com Rene Groeschke 25 Feb 2014 08:32 Hey guys, while working on the gradle native support I've noticed some issues with incremental builds. I've added an implementation of the AbstractLanguageIncrementalBuildIntegrationTest For ObjectiveC and Objective-CPP, which works without problems with gcc and with clang on osx, but I noticed randomly failing tests when using clang on ubuntu. After some digging I noticed that the generated object files can differ, even when created from exactly the same sources/flags. When comparing the according assembler code, I see that the address space is chosen randomly (see. diff here: http://pastebin.com/uY1LERGX). This behavior comes from a security feature called ASLR (Address space layout randomization). Toget our tests to pass on every invocation, I needed to disable ASLR by globally configuring it through /proc/sys/kernel/randomize_va_space (0 for disabling, 2 for enabling, 1 for "conservative randomization) or by disabling it locally via "> setarch `uname -m` -R /bin/bash". So, ASLR breaks our incremental builds because the generated object files can differ. I havn't yet figured out why ASLR currently does not affect our c / c++ integration tests with clang. There are also a bunch of raised bugs that prevents/prevented llvm to be complete deterministic. Some of them were already solved in newer llvm versions. To get the incremental build running we would need to be able to disable ASLR in our native toolchains. Of course this comes with the cost of less secure output that ASLR introduced. I see 3 options at the moment: - don't deal with ASLR at all with the cost of possibly loosing incremental build support. - always disable ASLR if possible to have deterministic builds (with the cost of loosing ASLR introduced security). - model this in our toolchains to allow enabling/disabling ASLR support. Thoughts? --------------------------------------------------------------------- To unsubscribe from this list, please visit: http://xircles.codehaus.org/manage_email