I seem to remember a tool called `make` that was pretty good at this. On Tue, Nov 28, 2017 at 10:47 AM, Lukasz Cwik <lc...@google.com> wrote:
> Its been well shown that a build system that uses input/output set change > detection can correctly implement incremental builds. Build systems are not > tied to knowing the internal details of how Java compiles things. Knowing > that there are some inputs, a process, and some outputs is enough to know > when the process needs to be rerun. > > On Mon, Nov 27, 2017 at 9:53 PM, Romain Manni-Bucau <rmannibu...@gmail.com > > wrote: > >> Hmm, no. >> >> Incremental build is never correctly implemented cause there is just no >> way to detect some dependencies statically with java code - or any dynamic >> language. >> >> Side note: same applies for gradle daemon usage BTW. >> >> After if the list is not maintained it is a bug at the same level than >> coding a toString() with "null.toString()". This is not very hard to handle >> the list of modules and worse case a mvnextension can make it coded if you >> feel more comfortable with this kind of solution. >> >> Le 27 nov. 2017 23:12, "Lukasz Cwik" <lc...@google.com.invalid> a écrit : >> >>> Manually whitelisting/blacklisting sub-modules is error prone since it >>> hides issues due to incorrectly maintaining that list is the same >>> argument >>> as if the build process doesn't correctly invoke an incremental build >>> process. >>> >>> On Mon, Nov 27, 2017 at 1:45 PM, Romain Manni-Bucau < >>> rmannibu...@gmail.com> >>> wrote: >>> >>> > Well for validation builds- pre PR, incremental support is pointless >>> since >>> > it easily hides issues die to caching so a solution saving half of the >>> > build without loosing anuyhing would still be good IMHO. >>> > >>> > Le 27 nov. 2017 21:12, "Lukasz Cwik" <lc...@google.com.invalid> a >>> écrit : >>> > >>> > > Incremental builds aren't correctly setup right now so your likely >>> to see >>> > > Python/Go rebuild even if there were no changes. See >>> > > https://issues.apache.org/jira/browse/BEAM-3253 >>> > > >>> > > On Mon, Nov 27, 2017 at 11:46 AM, Romain Manni-Bucau < >>> > > rmannibu...@gmail.com> >>> > > wrote: >>> > > >>> > > > that was the goal: validate there was no side effect of the >>> changes on >>> > > > the whole project. Now the "not java" part of the build will not be >>> > > > impacted by java changed so this is the part i want to skip since >>> it >>> > > > takes a lot of time and I have guarantees it is safe to skip them. >>> > > > >>> > > > Romain Manni-Bucau >>> > > > @rmannibucau | Blog | Old Blog | Github | LinkedIn >>> > > > >>> > > > >>> > > > 2017-11-27 20:28 GMT+01:00 Lukasz Cwik <lc...@google.com.invalid>: >>> > > > > Romain, that will build the entire project. I think you want to >>> > execute >>> > > > > (from the root of the project): >>> > > > > ./gradlew :beam-sdks-parent:beam-sdks-python:build >>> > > > > >>> > > > > On Mon, Nov 27, 2017 at 11:25 AM, Romain Manni-Bucau < >>> > > > rmannibu...@gmail.com> >>> > > > > wrote: >>> > > > > >>> > > > >> gradle build --no-daemon >>> > > > >> >>> > > > >> (with gradle 4.2) >>> > > > >> >>> > > > >> Romain Manni-Bucau >>> > > > >> @rmannibucau | Blog | Old Blog | Github | LinkedIn >>> > > > >> >>> > > > >> >>> > > > >> 2017-11-27 20:21 GMT+01:00 Kenneth Knowles >>> <k...@google.com.invalid >>> > >: >>> > > > >> > What is the gradle command you are using to build just the >>> Python >>> > > SDK? >>> > > > >> > >>> > > > >> > On Mon, Nov 27, 2017 at 11:19 AM, Romain Manni-Bucau < >>> > > > >> rmannibu...@gmail.com> >>> > > > >> > wrote: >>> > > > >> > >>> > > > >> >> Hmm, >>> > > > >> >> >>> > > > >> >> issue is the same with gradle (locally python build takes >>> 15mn >>> > > alone >>> > > > >> >> which is as much as the java build and it is not >>> parallelized I >>> > > > think) >>> > > > >> >> >>> > > > >> >> pl is not as smooth since it means doing it on each command >>> > whereas >>> > > > >> >> the proposal is automatically activated through settings.xml >>> > > > >> >> >>> > > > >> >> Romain Manni-Bucau >>> > > > >> >> @rmannibucau | Blog | Old Blog | Github | LinkedIn >>> > > > >> >> >>> > > > >> >> >>> > > > >> >> 2017-11-27 20:07 GMT+01:00 Kenneth Knowles >>> > <k...@google.com.invalid >>> > > >: >>> > > > >> >> > I think you can already mostly do this with mvn -pl >>> sdks/XYZ >>> > -am >>> > > > >> -amd. I >>> > > > >> >> > think that we have other work (gradle support) underway >>> that >>> > will >>> > > > make >>> > > > >> >> this >>> > > > >> >> > a non-issue since gradle automatically does even better >>> than >>> > the >>> > > > >> profile >>> > > > >> >> or >>> > > > >> >> > -am -amd. >>> > > > >> >> > >>> > > > >> >> > On Mon, Nov 27, 2017 at 11:01 AM, Romain Manni-Bucau < >>> > > > >> >> rmannibu...@gmail.com> >>> > > > >> >> > wrote: >>> > > > >> >> > >>> > > > >> >> >> Hi guys, >>> > > > >> >> >> >>> > > > >> >> >> java/python/go/xxx support is great but as a developer you >>> > > rarely >>> > > > >> hack >>> > > > >> >> >> on them all. >>> > > > >> >> >> >>> > > > >> >> >> For that reason I opened https://github.com/apache/ >>> > > beam/pull/4173 >>> > > > . >>> > > > >> >> >> >>> > > > >> >> >> Goal is to give each developer a way to build the whole >>> > project >>> > > > and >>> > > > >> >> >> all the code he can impact at once but without caring of >>> the >>> > > code >>> > > > he >>> > > > >> >> >> doesn't modify at all - other languages. >>> > > > >> >> >> >>> > > > >> >> >> Wdyt? >>> > > > >> >> >> >>> > > > >> >> >> Romain Manni-Bucau >>> > > > >> >> >> @rmannibucau | Blog | Old Blog | Github | LinkedIn >>> > > > >> >> >> >>> > > > >> >> >>> > > > >> >>> > > > >>> > > >>> > >>> >> >
