Davide Grandi created DERBY-7052: ------------------------------------ Summary: Reordering and (mildly) reorganizing build.xml Ant targets around buildsource Key: DERBY-7052 URL: https://issues.apache.org/jira/browse/DERBY-7052 Project: Derby Issue Type: Improvement Components: Build tools Affects Versions: 10.15.1.3 Reporter: Davide Grandi
Module dependencies should be reflected by Ant targets dependencies. And Ant targets execution follows the rules stated in https://ant.apache.org/manual/targets.html so a dependency could be covered (being redundant) by applying transitivity rule to every dependency's dependecies (please look at the dot+pdf diagrams). -------------------------------------------------------------------------------- Modules dependencies, by the docs, are : ||module depends||on|| |org.apache.derby.client|org.apache.derby.commons| |org.apache.derby.engine|org.apache.derby.commons| |org.apache.derby.locale_*|org.apache.derby.commons| | | | |org.apache.derby.tools|org.apache.derby.client| |org.apache.derby.tools|org.apache.derby.engine| | | | |org.apache.derby.optionaltools|org.apache.derby.tools| | | | |org.apache.derby.server|org.apache.derby.tools| | | | |org.apache.derby.runner|org.apache.derby.server| | | | |org.apache.derby.engine|org.osgi.framework| but actual ant dependencies are (partial target => module map) : ||target||depends||and module||depends on|| |buildsource|client| | | |buildsource |shared| | | |buildsource|felixStubs| |org.osgi.framework| |engine|shared|org.apache.derby.engine|org.apache.derby.commons| |client|shared|org.apache.derby.client|org.apache.derby.commons| |tools|engine|org.apache.derby.tools|org.apache.derby.engine| |drda|engine| | | |runner|optional| | | |optional|engine| | | so we expect that building 'engine' target alone will fails due to the lack of 'felixstubs' dependency. And it fails ("01 ant engine.log") quoting "requires static org.osgi.framework;" So the best try is to delete (on copy "build-01.xml" of "build.xml") the dependency buildsource => felixstubs and to put it as last dependency of 'engine' target Result : success ("02 ant -f build-01.xml engine.txt") Similarly we can expect that also 'tools' fails, since it really depends on 'client' but this is not explicited in "build.xml". And it fails ("03 ant -f build-01.xml tools.log") with: [javac] requires static org.apache.derby.client; So the best try is to switch the 'client' dependency (in copy "build-02.xml" of "build-01.xml") from 'buildsource' to 'tools'. Result: success ("04 ant -f build-02.xml tools.log"). The same thing for 'optional' that should depends on 'tools' but depends only on 'engine'. And it fails ("05 ant -f build-02.xml optional.log") with [javac] requires org.apache.derby.tools; So we switch the 'tools' dependency from 'buildsource' to 'optional' (in copy "build-03.xml" of "build.02.xml") And it succeeds ("06 ant -f build-03.xml optional.log"). Same as 'drda' (aka server) that require 'tools' ("07 ant -f build-03.xml drda.log") [javac] requires org.apache.derby.tools; that succeds after adding 'tools' dependency to 'drda' ("08 ant -f build-04.xml drda.log") and also for 'runner' ("09 ant -f build-04.xml runner.log") that requires 'drda' [javac] requires org.apache.derby.server; and succeeds after adding it into copy "build-05.xml" of "build-04.xml". ("10 ant -f build-05.xml runner.log") After graphing "build-05.xml" (in "build-05.xml.pdf") one can notice that, apart from redundant prerequisites (covered later), there's an extra dependency on 'runner', that in module hierararchy needs only 'drda'. After duplicating "build-05.xml" in "build-06.xml" and edited, removing 'runner' -> 'optional' prerequisite, build of 'runner' succeeds ("11 ant -f build-06.xml runner.log") The next step is removing all the extra dependencies. Given the transitivity of dependencies in this set A => C (a depends on C) A => B (a depends on B) B => C (B depends on C) the first dependendency is already covered by second and third one, and ant will ignore it. After graphing "build-06.xml" (in "build-06.pdf") and focusing on the subset from 'buildsource' to 'shared' we can reduce it (in "build-07.xml")applying the previous rule and deleting these dependencies: - buildSource => shared - buildsource => drda - buildsource => engine - drda => engine - optional => engine before continuing there are some test that can be passed : - single target checking : a serie of deep clean + single target builds e.g. : ant -f build-07.xml clobber svn st --no-ignore del/rmdir items marked with "I" rmdir /Q /S classes.pptesting rmdir /Q /S jars del tools\java\junit.jar ant -f build-07.xml shared ... for shared, felixstub, engine, client, tools, drda, storeless, optional, runner so the full build list, refactored in a windows batch command, is -------------------------------------------------------------------------------- @echo off call ant clobber -file %1 -quiet -silent > nul 2>&1 if errorlevel 1 ( echo error %ERRORLEVEL% during first clobber exit /B 1 ) for %%t in ( shared felixStubs engine client tools drda storeless optional runner buildsource ) do ( call ant clobber -file %1 -quiet -silent > nul 2>&1 rmdir /Q /S classes.pptesting > nul 2>&1 rmdir /Q /S jars > nul 2>&1 del tools\java\junit.jar > nul 2>&1 rmdir /Q /S classes > nul 2>&1 rmdir /Q /S generated > nul 2>&1 rmdir /Q /S release > nul 2>&1 svn st --no-ignore | find "I " echo ant -file %1 %%t call ant -file %1 -quiet -silent %%t if errorlevel 1 exit /B 1 echo. ) -------------------------------------------------------------------------------- Executing "ant -f build-07.xml -quiet felixstubs" we get an unexpected fail (but tests exist for this) and looking at "13 ant -f build-07.xml felixStubs.log" we can notice that the root cause is an undefined variable. adding a dependency on 'state' to 'felixStubs' (on "build-08.xml") the build succeed ("14 ant -f build-08.xml felixStubs.log") Graphing "build-08.xml" (in "build.08.pdf") we notice that there's still a redundant dependency : shared => init covered by shared => state => showenv => init and, just on the right, another redundancy init => makeOutDirs already covered by init => setCompilerProperties => prebuild => makeOutDirs and, finally, a dependency engine => state already covered by engine => felixStubs => state and engine => shared => state so these can be removed. Building "build-09.xml" succeeds, and so the run test for all the targets ("15 test targets.cmd build-09.xml") -------------------------------------------------------------------------------- The last tests are a) log comparison b) file comparison we define a cleaning sequence -- call ant clobber -f build-09.xml -quiet -silent rmdir /Q /S classes.pptesting rmdir /Q /S jars rmdir /Q /S classes del tools\java\junit.jar svn st --no-ignore -- and, for the standard build sequence -- <cleaning sequence> ant -quiet all ant -quiet buildjars -- we generate a standard log, a verbose log <cleaning sequence> ant all -f build.xml > "all - build.xml.log" ant buildjars -f build.xml >> "all - build.xml.log" <cleaning sequence> ant all -f build.xml -verbose > "all - build.xml - verbose.log" ant buildjars -f build.xml -verbose >> "all - build.xml - verbose.log" <cleaning sequence> ant all -f build-09.xml > "all - build-09.xml.log" ant buildjars -f build-09.xml >> "all - build-09.xml.log" <cleaning sequence> ant all -f build-09.xml -verbose > "all - build-09.xml - verbose.log" ant buildjars -f build-09.xml -verbose >> "all - build-09.xml - verbose.log" all differences seems ok. -------------------------------------------------------------------------------- After looking at "build-09.xml.pdf" we notice that buildsource => init is already covered by many path, eg buildsource => optional => tools => client => shared => state => showenv => => init and also buildsource => prebuild is covered by ... => init => setCompilerProperties => prebuild and also buildsource => setCompilerProperties by the same path(s), so 'init', 'prebuild' and 'setCompilerProperties' can be deleted from 'buildsource' prerequisites in "build-10.xml". Tests and binary compare are ok. -------------------------------------------------------------------------------- Looking at "build-10.xml.pdf" and "build-10.buildsource-expanded-01.pdf" we notice that 'build' target, 5th buildsource dependency, is ant-called anyway from 'engine' and can be deleted from 'buildsource' dependencies in "build-11.xml". Tests and binary compares pass. The only problem remaining is the 'generateSecurityPolicies' target that fails if built alone (due to a deep buried dependency) -------------------------------------------------------------------------------- Summary of included files (maybe zipped) : ||File||Content|| |bu...@1863766.buildsource.dot|actual build graphing - focused on buildsource| |bu...@1863766.buildsource.pdf | | |bu...@1863766.buildsource-expanded.dot|same as above but with external build.xml expanded| |bu...@1863766.buildsource-expanded.pdf| | |bu...@1863766.xml|actual build.xml| |bu...@1863766.xml.dot|rendered in dot+pdf| |bu...@1863766.xml.pdf| | | | | |build-11.buildsource.dot|final build-11 graphing| |build-11.buildsource.pdf| | |build-11.buildsource-expanded.dot|same as above but with external build.xml expanded| |build-11.buildsource-expanded.pdf| | |build-11.xml|final build-11.xml| |build-11.xml.dot|rendered in dot+pdf| |build-11.xml.pdf | | | | | |patch.txt|the tiny 65 lines patch| ... I'll attach asap ... -- This message was sent by Atlassian JIRA (v7.6.14#76016)