Sorry about not putting the proper prefix on the subject but I wasn't sure
what it should be.
The following is a fairly lengthy overview of requirements that we've found
would be useful additions or fix existing issues with Ant.
Design suggestions for Ant 2.0
==============================
Prepared by Wind River Systems Inc.
author: Julian Bromwich
email: [EMAIL PROTECTED]
Submitted for March 21, 2001
Contents
========
- Summary
- Fundamental design changes to make Ant more powerful:
1) Scoping is definitely the biggest problem I can see.
2) Targets need to be considered more as "methods".
3) Flow control is the second biggest problem after scoping.
4) Data types need to be more powerful.
5) XInclude needs to be implemented.
6) Class-level dependency checking.
7) The Ant documentation needs to be better.
8) Nested elements...
- More standard tasks
=======================================================================
Summary
-------
Ant's original goal was to be a platform independent "make" utility.
It has far surpassed this goal and hints of being a very powerful,
extensible language. It does, however, have several shortcomings
which, at present, prohibit it from reaching this state. I have
prototyped some fundamental changes to Ant to address some issues.
I cover these changes as well as some changes that I would like to
see below.
Many of the problems with Ant are most visible in the scenarios
where one build file calls another (potentially identical) build
file, or recursively calls itself. Scoping is the biggest problem
here, and I will go into that more later.
Also, many problems I encountered in creating my own Ant tasks
centered around the fact that many components of Ant weren't made to
be easily extended. Many things were private that should probably
have been protected or even public. In contrast, some things such
as the "properties" and "userproperties" hashtables were essentially
public and really should have just had accessor methods to retrieve
individual entries.
In Ant 2.0, the architecture needs to be carefully designed to allow
maximum flexibility to the Task writer while still properly
protecting access to core components.
If you would like any of the source code to the extensions that we
have done, please just let me know and I will send them to you.
Good luck in your Ant 2.0 design cycle. I will try to participate
as much as I can... if I am given time. :)
Fundamental design changes to make Ant more powerful
----------------------------------------------------
1) Scoping is definitely the biggest problem I can see.
Scoping currently is the cause for all kinds of un-expected
behaviour. The one which comes to mind is demonstrated in this
following over-simplified example:
<target name="myself" unless="base_case" >
<testbasecase property="base_case" />
<antcall target="myself" />
</target>
Quite obviously, this is intended to recurse until the base-case is
true. As it stands, you can't really do this in Ant for a several
reasons:
1) Child call always inherit the parent's variables
2) The child can't re-assign those variables
3) "if/unless" only check for the existence of a variable,
not its value.
For now, I can get around this problem with my "variable" task
which allows me to re-assign or completely destroy a property...
a hack indeed. :)
What is really needed is scoped data.
- A build file should be considered much like a class.
- A target should be considered more as a method.
- Local variables (within a target) should be distinguished from
globals.
This implies the following scoping rules:
- A target can see it's local variables and the globals within the
current build file.
- When one target calls another, the called target only has access to
the globals that are within its build file, not the globals of the
calling target's build file.
The "property" and "UserProperty" hashtables are just weird. Who's
idea was that!? That needs to be re-thought.
Perhaps think about structuring the data space more in the way that
a language would with a heap for each project and a stack for each
target invocation.
The scoping problem doesn't look to me like it will be easily
solved, but I think it is very important to spend the time to come
up with a good solution now that won't have to be changed in the
future.
2) Targets need to be considered more as "methods".
- The ability to return values to the caller is needed.
Suggested syntax:
Calling target:
<invoke antfile="some_build_file.xml" target="some_target"
property="my_return_val" />
Called target:
<return> value="some string" </return>
"Invoke" would replace both the "Ant" and "AntCall" tasks since
the "antfile" parameter would default to the current build file
if not specified.
Upon returning from calling "invoke", in this case, the property
"my_return_val" in the local scope would have the value "some
string", or would not be set if the called target did not return
a value, or if the "property" parameter was not specified in the
calling "invoke" task.
Again, this is something that I have prototyped and have been
using with good results for a while now. I also consider this
feature to be vital.
3) Flow control is the second biggest problem after scoping.
- There need to be conditionals.
- A target should also have the option of a return statement for
early abort.
- Conditionals would be nice too. There are a couple incompatible
options here:
1) <and> and <or> tags
2) <if test="(A | B) & ~C" > where a standard expression syntax
is used.
(I prefer the second option myself, since there are already
standards and supporting libraries out there for this.)
Maybe something like this just off the top of my head, or maybe
just use one of the many existing standards. XSLT does something
like this and would be worth a look.
<target name="example">
<if test="some_test">
<echo message="early return" />
<return/>
</if>
<if test="(A | B) & ~C" >
<echo message="normal return" />
<return value="2" />
</if>
<echo message="end of target" />
</target>
- There are several standards for how to implement conditionals
in XML.
The XSLT spec is at http://www.w3.org/Style/XSL/
It allows for conditionals such as <if> and case statements:
<choose> <when> <otherwise> with the use of a test="conditional"
syntax.
4) Data types need to be more powerful.
This can only be properly accomplished if the "scoping" issue above
is first addressed.
I have added a "variable" and a "list" datatype which I consider to
be absolutely necessary for Ant to be powerful. The "list"
datatype opens up whole new areas for Task creation and has been
very useful.
I had the situation where I needed to create a software load from
several components. Each component compiled in exactly the same
way but had different dependencies on other components. To compile
component "A", it was most convenient to create a task to determine
the list of all the components that needed to be compiled first.
By calling each component's "build.xml" file (which returns a
classpath as a "list"), I was able to dynamically build the
classpath that component "A" needed to compile. This was most
often a multi-layered process and the "list" datatype allowed the
returning of an extensive classpath list back through several Ant
call-stacks. At the top of the stack, it is eventually converted
from a "list" (which is a Vector internally) to a path string via
the list's "tostring" parameter. The reason I kept the classpath
as seperate elements in a list rather than combined into a single
colon-separated path, was that I needed to easily be able to check
that I was not adding repeat paths into the list. (see
unique="true" below).
Examples:
<!-- Overwrite the previous value of language with a new one -->
<variable name="language" value="english" />
<!-- Destroy the skip_clean variable (a work-around hack) -->
<variable name="skip_clean" remove="true" />
The "remove" parameter destroys the variable (removes it from the
hashtable), but this is a hack to get around the fact that the Ant
"if" cannot check for anything other than the existence of a
property. It also allows me to get around the scoping problems.
Normally, once I set "skip_clean", all targets that are called in
other build files also have "skip_clean" set by inheritance, and
this is not the desired behaviour.
<list create="jar_list" />
<list name="${jar_list}" addlists="${invoke_return_list}"
unique="true" tostring="path" property="classpath" />
<list name="circular_dependencies" add="${thisfile}" unique="true"/>
<list name="${import_list}" containslist="circular_dependencies"
property="circular_error" />
<list name="circular_dependencies" remove="${thisfile}" />
<list name="${jar_list}" add="${bundlejar}" />
The "create" functionality is another hack to get around the lack
of scoping in Ant. It just returns a unique name since the "list"
dataspace is global and there would be a collision if the script
was recursive.
Other than XInclude, I have found the "list" datatype to be the
most useful thing I have added to Ant's repertoire.
It also facilitates many other useful tasks such as these examples:
<!-- Set skip_clean if ${myjar} is more recent than everything in
the ${dependency_list} -->
<uptodatelist property="skip_clean" targetfile="${myjar}"
list="${dependency_list}" />
<!-- find all build files within the BUNDLE_PATH, and return a
list of them in "buildfile_list" -->
<find path="${BUNDLE_PATH}" name="build.xml" prune="true"
property="buildfile_list" />
<!-- Invoke build on all dependencies and put their return values
in "classpath_list" -->
<invokelist list="${import_list}" target="${buildtarget}"
property="classpath_list" />
<!-- To convert the list "classpath_list" into a path string called
"classpath" do this... -->
<list name="classpath_list" tostring="path" property="classpath" />
5) XInclude needs to be implemented.
I have implemented XInclude for Ant, by allowing Ant to classload
and run a specified pre-processor (-Dpp="Xincluder"), but I think
Xinclude should be directly supported by Ant.
As an example of what this looks like, here's a snipit of a build
file which includes the definitions for some of the tasks that I
have written:
<!-- Wind River Ant extensions -->
<xinclude:include href="file:/${WIND_HOME}/ant/windtasks/windtasks.xml"
parse="xml" >
<xinclude:children />
</xinclude:include>
The above block is replaced by Ant's pre-processor with all
thechild elements of the xml file called windtasks.xml. Only the
<taskdef> elements are included since the xinclude specifies
<xinclude:children />.
Here's a shortened version of windtasks.xml (which is being
included above).
<?xml version="1.0"?>
<antTaskDefinitions>
<taskdef name="abort"
classname="com.windriver.antx.taskdefs.Abort" />
<taskdef name="canonical"
classname="com.windriver.antx.taskdefs.Canonical" />
<taskdef name="getlocalhost"
classname="com.windriver.antx.taskdefs.GetLocalHost" />
<taskdef name="httpserver"
classname="com.windriver.antx.taskdefs.HTTPServer" />
<taskdef name="invoke"
classname="org.apache.tools.ant.taskdefs.Invoke" />
<taskdef name="invokelist"
classname="org.apache.tools.ant.taskdefs.InvokeList" />
<taskdef name="list"
classname="com.windriver.antx.taskdefs.List" />
<taskdef name="uptodatelist"
classname="com.windriver.antx.taskdefs.UpToDateList" />
<taskdef name="variable"
classname="com.windriver.antx.taskdefs.Variable" />
<taskdef name="xslt"
classname="com.windriver.antx.taskdefs.Xslt" />
</antTaskDefinitions>
Here are some of the uses that I have found for XInclude:
- The "build.xml" for a particular project can "extend" a base
"project.xml".
- By specifying targets in "build.xml" of the same name as those
in "project.xml", before the Xinclude statement, I can override
particular targets from their default. (This is SO USEFUL
since we compile many "nearly-identical" components that all
share the same complicated build process, but often need to
customize the process slighty.)
- (as above) Easily include all taskdefs for my Ant tasks.
XInclude is another standard from w3.org, though I have extended
this standard somewhat to allow for partial includes of an XML
tree. Ant 2.0 should probably support the XInclude standard as
described w3c (once they get it finalized) and then perhaps I will
try and get my extension supported as part of the w3c XInclude
standard.
Our XInclude syntax also allows for the following include which
specifies only parts of an XML file to be included:
The original source file:
-------------------------
<?xml version="1.0" encoding="iso-8859-1"?>
<project xmlns:xinclude="http://www.w3.org/1999/XML/xinclude";>
<xinclude:include href="file:/${WIND_HOME}/include/header.xml"
parse="xml">
<A name="2"/>
<A name="1">
<A1/>
</A>
<B>
<xinclude:children/>
</B>
</xinclude:include>
</project>
The file to include ("header.xml"):
-----------------------------------
<?xml version="1.0" encoding="iso-8859-1"?>
<childmaintag>
<A name="1">
<property name="1A" value="1" />
<property name="1B" value="2" />
<A1>
<property name="1C" value="3" />
</A1>
</A>
<A name="2" bob="3">
<property name="2A" value="1" />
<property name="2B" value="2" />
<A1>
<property name="2C" value="3" />
</A1>
</A>
<B name="1">
<property name="B1" value="1" />
<property name="B2" value="2" />
<B1>
<property name="B3" value="3" />
</B1>
</B>
</childmaintag>
The original source file after the XInclude is processed:
---------------------------------------------------------
<?xml version="1.0" encoding="iso-8859-1"?>
<project>
<A name="2" bob="3">
<property name="2A" value="1" />
<property name="2B" value="2" />
<A1>
<property name="2C" value="3" />
</A1>
</A>
<A1>
<property name="1C" value="3" />
</A1>
<property name="B1" value="1" />
<property name="B2" value="2" />
<B1>
<property name="B3" value="3" />
</B1>
</project>
6) Class-level dependency checking.
This needs to be addressed. We haven't explored this enough yet to
make too many suggestions on how this is accomplished. Our
developers have asked me several times about this, but I don't know
how to easily provide it in a compiler-independent way.
7) The Ant documentation needs to be better.
- Subtle behaviours of Ant are not properly documented. For
instance, it does not say (see the example below) which runs
first A or B, or if "A" runs at all if "B" fails, etc... I had
to experiment quite a bit to determine how it behaves, since I
was setting the value of "B" in the task "A". There just needs
to be a bit more explanation in the docs on how some of this
"core-Ant" stuff works.
<target name="example" depends="A" if="B" unless="C" >
</target>
- There also need to be more examples, especially showing how
nested elements work.
8) Nested elements...
- As it stands, nested elements only work if the task supports that
nested element specifically, but how would we get this example to
work?
<if test="condition>
<some_arbitrary_block>
</if>
The <if> task (if indeed it is ever made) would not know about
all tasks which might be within it. This needs some careful
thought... and I haven't given it much yet. :)
Nevertheless, I often wish I could do something like the above
example, and I'll bet others do too.
More standard tasks
-------------------
Here are some tasks that I implemented and found very useful and may
warrant including. (I will provide source code to any of these that
you wish of course)
- abort
System.exit if condition true.
<abort if="circular_error"
message="Circular dependency detected in ${thisfile}" />
- canonical
Expands path relative to current build file by default, or
relative to path of original invocation if callerpath="true".
(This is unbelievably useful when you have a lot of ant files
calling each other in various directories).
<canonical name="thisfile" value="${ant.file}" />
- getlocalhost
Get IP address of current machine (This can be problematic in
Java (host with multiple interfaces) so we should make sure that
this task supports the full capability.).
<getlocalhost property="ip_address" />
- httpserver
Registers files in a small http server.
(addFile="virtual_name, actual_name")
<httpserver base="${base_url}"
addFile="/${bundlename}, ${bundlejar}" />
- invoke
Calls an target (optional) within an antfile (optional) and
allows for a return value.
<invoke antfile="${WIND_HOME}/windcore/osgicore/build.xml"
property="jar_list" />
- invokelist
Same as invoke, but invokes a list of antfiles, with a
particular target.
<invokelist list="${import_list}"
property="invoketarget.return"
target="${buildtarget}" />
- list
Creates and processes lists.
<list name="${jar_list}" addlists="${invoke_return_list}"
unique="true" tostring="path" property="classpath" />
- uptodatelist
Just like "uptodate" but on a list (Current "uptodate" task
should really instead be able to process a list rather than
create this new task).
<uptodatelist property="skip_clean"
targetfile="${bundlejar}" list="${jar_list}" />
- variable
Just like "property" but re-writable.
<variable name="base_url"
value="/com/windriver/internal/bundle" />
- xslt
Calls xslt (XML Stylesheet translation).
<xslt input="file:/${manifest_xml}" output="${manifest_mime}"
xsl="file:/${manifest_stylesheet}" />
- find
Like Unix "find" command but starts from multiple base
directories that are specified in "path".
<find path="${BUNDLE_PATH}" name="build.xml" prune="true"
property="buildfile_list" />
