Over the past week, I've had a bunch of ideas for enhancing
the Gradle's command line.
My goals are as follows:
[1] Allow directory and/or file tasks to be specified naturally on
the command line, without any weird clutter. They should't
be second-class citizens in a world where Gradle one day
handles C++ builds.
[2] Make it easy to specify a list of tasks within different
namespaces (subprojects), and which tasks in the list
to execute (or exclude) recursively.
[3] Provide a simple rule for resolving conflicts in whether to
run a task or not in a convenient, expressive, and
well-defined way.
[4] Allow tasks to be specified (or skipped) by regex.
[5] Provide a syntax for skipping any task that depends upon
a given task (or task regex) directly or indirectly.
[6] Provide a way to tunnel a set of args to a task via the
command line.
This email is pretty long, but I think it goes a long way
to addressing goals [1]-[6]. Hopefully, others will think
the goals themselves matter, but I'd really like feedback
on both the goals and the ideas I've presented below for
addressing them.
Cheers,
-Jon
------------------------------------------------------------------
Gory Details:
------------------------------------------------------------------
The easy one first...
[1] Directory (or file) tasks names should just be equal
to their own path name. Thus, to run the 'a/b/c'
task on all subprjects, just type:
% gradle a/b/c
Note: This one isn't much of a proposal, but I wanted to
include explicit support for file/dir tasks without
weird name mangling rules. File/dir task names should
be first class citizens!
------------------------------------------------------------------
[2] To get fine-grained control over what tasks are recursive
or non-recursive, adding new options to the following syntax
isn't quite powerful enough:
% gradle [options] [tasks]
Thus, exploring a "decorator" syntax for tasks seems like
the way to go. Thinking analogously for a moment:
o In shells (and in ant), something like: '**/*.c'
means '*.c in all subdirs, recursively'.
o In many languages, the colon (':') or double colon
('::') is associated with namespaces.
Let's see where it leads if we:
o Denote recursion in subprojects via the token '::'
(note that subprojects are a sort of namespace).
o Denote non-recursion via ':'
o Make the last namespace given on the command line remain
in effect until another namespace specifier is encountered;
this lets us avoid giving the same long subproject path
name over and over again if we've got a list of tasks.
Trivial examples
----------------
If the default namespace is the empty string followed by '::',
and if we make whitespace between a namespace and the task(s)
within it insignifigant, then the following commands are identical:
% gradle foo # normal shorthand form
% gradle ::foo # '<the-empty-string>::foo' == 'foo'
% gradle :: foo # whitespace after :: does not matter
If we had a list of tasks ('foo' and 'bar') we can extend this
trivial example a bit further:
% gradle foo bar # normal shorhand form
% gradle ::foo ::bar # longhand form
% gradle ::foo bar # :: isn't overriden, so bar == ::bar
% gradle :: foo bar # prettier formatting for a list
Typically though, we'd just say:
% gradle foo bar # no different from what we do today!
So far, so what, right? In the typical case, it just looks like
the same old command line! Well, let's move on to some fancier
examples & see where this takes us.
Fancier examples
----------------
Let's say you wanted to execute the following tasks:
o The foo task (starting from the current project)
o The 'moo', 'cow', and 'egg' tasks recursively
within the relatively-specified subproject x/y
o The 'c' and 'd' tasks non-recursively
within the absolutely-specified project /a/b
To do that with the proposed syntax you could say:
% gradle foo x/y:: moo cow egg /a/b: c d
Note that the (recursive) 'x/y::' namespace remains in effect
until overridden by the (non-recursive) 'a/b:' namespace. Thus,
the "longhand" version of this would be:
# This is just for illustration purposes!
% gradle ::foo x/y::moo x/y::cow a/b:c a/b:d
If task skipping is denoted by a '-' before the task name, then
% gradle /a/b:: moo -cow
would mean:
o Recursively run the moo task within the
absolutely-specified /a/b subproject.
o If running moo anywhere implies running
a cow task anywhere, skip the cow task.
Taking this further:
% gradle /a/b:: moo /a/b/c: -cow
would mean:
o Recursively run the moo task within the
absolutely-specified /a/b subproject.
o If running moo anywhere implies running
the cow task within /a/b/c, skip that cow task.
If the cow task is implied elsewhere, then it
won't be skipped. Notice that '/a/b/c: -cow'
isn't a recursive skip because of the ':'.
One very nice thing about representing namepaces as
slash-seperated dirs (with a terminating ':' or '::')
is that it lets you use tab completion!
------------------------------------------------------------------
[3] The decision rule when two contradictory instructions
are given should be "the last rule wins". Thus:
% gradle foo a/b:: -bar a/b/d:: bar
would mean:
o Recursively run the foo task
o Recursively prohibit the bar task from running
within the a/b subproject
o Recursively override the prohibition against
running the bar task in the subproject a/b/c
Thus, bar would not run in a project like a/b/c or a/b/e
because of the 'a/b::-bar' goal, but it would run in a/b/d,
a/b/d/xyz, a/b/d/foo/bar (and so on) because of the later
p'a/b/d::bar' recursive override.
There are a few different alternatives to "last rule wins",
but in considering them, it seems like end up being confusing,
not very expressive, or ambiguous. This becomes clearer
when you consider task regexes (see the next section).
------------------------------------------------------------------
[4] Allowing tasks to be specified (or skipped) by some sort
of regex or globbing syntax would be great.
Several issues spring to mind:
o Glob vs regex:
Glob is usually easier/better for file patterns,
but could potentially be annoyingly limited when
it comes to task name patterns.
o If we use "real" regexes, do we use find, match,
or leading substring pattern semantics?
o Scope of regex/glob: task vs namespace vs global
o Interactions with other task decorators like + or -
and delimiting the end of decorators with the
start of the pattern itself.
My current view:
o Because subproject paths correspond to filesystem
paths, some kind of globbing syntax seems natural.
for specifying namespaces. There are many flavors
of globbing out there, but I'm thinking of something
fairly limited like:
* The '*' character for zero or more characters
* The '?' character for any single character
* The character class ([...]) where you can have a specify
individual characters (via a hyphen). For example,
'[aeiou3-7]' represents the set of all vowels plus the
characters '3', '4', '5', '6', and '7'.
* Alternation: '(...|...|...)'
For example, the glob: 'moo(xxx|yyy)cow'
is matched by both 'mooxxxcow' and 'mooyyycow'.
Note: you'd probably need to quote globs
to avoid interactions with your command
line shell.
o By default use globbing on task names as well.
Consider the following command:
% gradle 'hello*xyz'
I think that should mean to run any task that starts with
the string 'hello' and ends with 'xyz'. If the default
were
a regex gramamr, then you'd need to say something more like:
% gradle '^hello.*xyz$' # Yuck.
Full regexes are powerful, but they're fairly ugly;
most of the time, you'd never need all the power they
provide anyway. Thus I favor treating tasks on the
command line as glob patterns, not regexes.
Still, it would be cool if we *could* do real regexes. I belive
it's possible if we a sequence like '+~' for regex inclusion
and '-~' regex exclusion. Because '+' is optional, you should
also be able to have a bare '~', as long as it's quoted; otherwise
a shell might confuse this with a homedir.
Thus the following are all equivalent:
% gradle +~test # regex
% gradle '~test' # regex with bare '~' (suitably quoted)
% gradle '*test*' # glob syntax for "match test anywhere"
Here's a nice illustration of where a real regex could be handy:
% gradle '~(?i)test$'
This would mean: recursively run any task that ends with
the string 'test', matched in a case-insensitive way.
That's not too far-fetched of an example. Who knows
what significance people might give to capitalization
in a set of naming conventions they use.
Of course you could support stuff that really is far-fetched too. :)
The one trick to all of this is that the '~' should always come
immediately before the task pattern if you've got multiple task
decorators. For example, to exclude a pattern like "contains
the string 'bar'", you'd say:
% gradle foo -~bar
not:
% gradle foo ~-bar
With that one rule, there's never any ambiguity as to whether
the '-' is part of the pattern (bar) or a negation indicating
a skipped task.
------------------------------------------------------------------
[5] Sometimes, you're building on a machine that simply can't
support a particular task. For example, some tasks
might require other programs that aren't present
(or resources like network access, licence keys, etc.)
Maybe only a special "golden build box" has everything
required to run every single task.
Example:
Let's say we have two tasks 'foo', and 'bar', each
of which have a long chains of dependencies:
foo->a->b->c->nasty->d->e->f->g->h->i
bar->j->k->l->m->n->o->p
Suppose that the build machine you're on can't do
task 'nasty' for some reason (e.g.: it may require
something platform specific, licence keys, etc.).
You'd like to be able to say:
% gradle foo bar ...something_magical....
Where ...something_magical... allows gradle to
be smart enough to *cleanly* skip task 'foo'
without first doing all the work of building
i,h,g,f,e, & d, then failing on 'nasty'.
The thing is, I want to be able to do this without digging
through what 'foo' depends on by hand. I want some way for
gradle to do that "...something magical..." bit for me!
Here's an idea:
What if '^' denoted "is wedged in there somewhere".
In the case given above, we'd be able to say:
% gradle foo bar -^nasty
This would mean:
Recursively run 'foo' as long as it does not directly
or indirectly depend on the 'nasty' task.
Recursively run 'bar' as long as it does not directly
or indirectly depend on the 'nasty' task.
More generally:
-^taskName Excludes any task that ultimately depends
upon taskName
+^taskName Includes any task that ultimately depends
upon taskName
^taskName Same as +^taskName -- the '+' is optional
This works nicely with recursive/non-recursive namespaces, negation,
glob, and regex syntax.
For example, consider:
% gradle foo a/b:-^~test
That would mean:
Recursively run the 'foo' task, but skip any task that
directly or indirectly depends upon any task that includes
the string 'test' in the subproject a/b.
NOTE:
o If 'foo' in subproject x/y/z depended on 'mytest'
in subproject a/b/c, it WOULD run -- 'a/b:' is
non-recursive.
o If foo in in subproject p/d/q were dependent upon
a task named 'your_test' in subproject 'a/b'
then foo would NOT run, as it would match
the rule a/b:-^~test
A nice side-feature for skipping would be something like
% gradle -n --show-skipped foo -bar ...
Where '-n' could be like the standard "don't really do anything"
flag (like Gnu Make), and the --show-skipped flag would emit
a list of whatever tasks were skipped.
The algorithm to make all this would wouldn't even be that hard:
just process the rules in-order. Discovering whether a task
ultimately depends on something could be handled by doing a
no-op "visitor pattern" style dag traversal. You could cache
the result and if necessary, perform a topological sort before
executing anything for real.
------------------------------------------------------------------
[6] Providing a way to tunnel a set of args to a task via the
command line would be nice, because Gradle tasks could be
used as a sort of dependency-aware program launcher.
Thinking analogously:
o Many utilities use '-' to indicate stdin
o A common way of saying "no more switches" is '--'
What if '_' meant "command line argument for task"
Consider the following:
% gradle mytask _ 'foo bar' # case 1
% gradle mytask _ '[foo, bar]' # case 2
% gradle mytask _ '[foo:bar, moo:cow]' # case 3
Suppose that when a task executes, it gets a set of parameters
in a variable called 'arg'. Further, let's say that gradle
parsed that arg like a string, array, or map (cases 1,2, and 3
respectively) accordingly. Thus in your build.gradle you
could have:
createTask("mytask")
{
if ( it.arg ...)
{
...
}
}
This could then launch your program (or do whatever else you wanted)
with all the power of resolve() and friends to set classpaths,
ensure preconditions, and so forth. In effect, it would let
gradle become a lightweight workflow engine (of sorts), with
the ability to parameterize ad-hoc actions directly on the command
line, rather than force everything into resource files. If gradle
is to be a dsl for dependency management (and not just a build system),
this seems like it could be a pretty big win for very little effort.
Hans and I talked about this a bit yesterday, and he was wondering
about also adding something like:
% gradle mytask _ '{ ...code... }'
It's a bit unclear to me what the best values of 'this'
should be within '{...code...}', and whether {...code...}
should replace action, or just be a way to configure 'mytask'.
After writing all this stuff down, I think it's time for
me to step back for a while. :)
Sick with a cold in Boston,
-Jon
---------------------------------------------------------------------
To unsubscribe from this list, please visit:
http://xircles.codehaus.org/manage_email
