---------- Forwarded message ----------
From: Christoph Lofi <[EMAIL PROTECTED] >
Date: Sep 13, 2005 1:04 PM
Subject: Re: [HACKYSTAT-DEV-L] RFC: modularized build process
To: Aaron Kagawa <[EMAIL PROTECTED]>
Hi!
Well, I don't know. I think generating the build file is the easiest and least dangerous way to go.
I also think that the debugugging problem isn't that severe as long as the build file generation is transparent (which is the case when using template languages like velocity).
But Aarons Option 2 also sounds cool too me.
Determining the module build order should not be too hard (isn't that some kind of stupid standard graph theory problem? "serialization of a paralell tree" or something like that?). The implementation of that isn't not too different from the generating alternative (crawaling for the module definition files, checking if all dependencies are there and then generating eitther a build file or some kind of build order description file.)
Have a sunny day,
Christoph
On 9/13/05, Aaron Kagawa <[EMAIL PROTECTED]
> wrote:
I agree with the current problems with Hackystat's Ant Build process. I've just done some Ant hacking for some other complicated projects and thought I could base it on Hackystat's build. However, because of the dependency problems I felt that Hackystat's build.xmls were pretty messy and pretty hard to understand (for newbies).
I like the idea of the definition file. But, how would the generated file know which module to build first? One would think that hackyKernel should go first, then followed by hackyInstaller, hackyStatistics, hackyReport, etc. Would we be able to determine the order of the modules from the definition file?
Not to mention, I just seen that the checkModuleAvailability needs to execute the checks in alphabetical order according to the comment.. "Modules in dailybuild. Keep in alphabetical order, please!" on line 125.
When it finds them, it reads in the data, and uses that information to _generate_ a build.xml file that includes the checkModuleAvailability, doInternal_<module>, and doAll tasks with all known modules.
(I'm just thinking out loud here. Feel free to stop reading).
Not sure if I like the idea of the generated build.xml file. Would that be hard to debug? I say that because, we would need to make sure that the generated file works and that the generator works too.
Option 2: I'm not sure we would even need to have a generated checkModuleAvailability, doInternal, doUtil, etc. It seems to me that if we can generate a list of modules in the order that it should be built, based on the definition files, then we should also be able to write targets that dynamically calls each module. What we could do is:
1) come up with a xml file that contains the order of the modules and the dependencies
2) figure out how to use that order in dynamic targets. this could involve custom ant tasks.
3) then after all these to steps see if we can generate the order xml file from the definition xml files. this could involve custom ant tasks.
i suppose option 2 is actually harder and has the same problems i explained earlier, but it seems cooler. :)
Option 3: use includes, macrodef, and subant.. see < http://ant.apache.org/manual/CoreTasks/subant.html>.
<project>The only thing is the ordering of the "modules" in the fileset. I think we could write an extension to the fileset to order the modules. Or use antfile="hackyKernel/local.build.xml, hackyInstaller/local.build.xml, hackyStatistics/local.build.xml, ....", which could be a string that is generated.
<macrodef name="iterate">
<attribute name="target"/>
<sequential>
<subant
target="@{target}">
<fileset
dir="modules" includes="*/build.xml"/>
</subant>
</sequential>
</macrodef>
<target name="compile">
<iterate target="compile"/>
</target>
<!-- more targets -->
</project>
Anyway.. this is a hard problem.
One last thing, there seems to be a need for module specific targets like installEclipseSensor (although this might be removed) and installInstaller.
thanks, aaron
At 05:09 PM 9/12/2005, Philip Johnson wrote:
Greetings, all,
As part of the Version 7.0 release planning, I wanted to try to address a build issue that has bugged me for a long time.
Consider the situation of someone who wants to develop their own Hackystat module (hackyFoo) and build a local version of the system with their hackyFoo extension incorporated. Currently, that process is (a) difficult and (b) undocumented. Essentially, you have to hack our build.xml file in several places:
- The "checkModuleAvailability" task must be hacked to check for the availability of the new hackyFoo module.
- A new "doInternal_hackyFoo" task must be written for the new module, indicating its dependencies on other modules.
- The "doAll" task must be hacked to include the new "doInternal_hackyFoo" task.
Of course, each time we release a new version of Hackystat, with possible changes to the build.xml, the person who has implemented hackyFoo has to go in and manually edit the new build.xml to include the hackyFoo changes.
This is totally bogus. The basic problem is that our build.xml file, as currently written, hard-wires in knowledge about all of the known modules for Hackystat. What we really need is a build.xml file that "discovers" the set of Hackystat modules at Ant invocation time. Consider the following idea:
Right now, each module must include its own local.build.xml file, which provides module-specific tasks to support compilation and installation. This is A Good Thing and works very well. Let's extend this to say that each module also needs to include a local.module.definition.xml file, which might look like this:
<hackystat-module>
<name>hackyFoo</name>
<available>on</available>
<dependent-modules>
<module>hackyKernel</module>
<module>hackyStdExt</module>
</dependent-modules>
</hackystat-module>
As you can see, a module definition is simple: it just specifies the module name, whether it is "available" (this replaces the hackyFoo.available property in hackystat.properties), and what modules it depends upon (this replaces all of the information now hard-wired into build.xml).
Now, let's apply the "autoconf" pattern to the problem. A simple way to solve things would be to say that when someone wants to build Hackystat for the first time with a new or changed module definition, one must do:
ant config
what that does is search all of the "sibling" directories to hackyBuild, looking for occurrences of local.module.definition.xml files. When it finds them, it reads in the data, and uses that information to _generate_ a build.xml file that includes the checkModuleAvailability, doInternal_<module>, and doAll tasks with all known modules.
At this point one can invoke all of the normal set of commands. Furthermore, each time one runs a "normal" command, there is first a quick check of the timestamps associated with all known local.module.definition.xml files, and if their timestamps are later than the timestamp associated with build.xml, then the invocation fails with a notice that the developer needs to re-run 'ant config' to get an up to date version of build.xml.
This particular scheme seems quite feasible to me, since it requires no changes to the current build process other than the creation of a "configuration" step.
Questions for discussion:
- There might be ways to simplify the resulting build.xml file by using recent ant features like macrodefs and so forth.
- Is there a more radical way to re-invent the build process to accomplish this same goal?
I'd really like to get this fixed in the near future; I think it will significantly simplify external involvement in hackystat development.
Let me know what you think.
Cheers,
Philip
