Very nice, and I like it a lot,
2009/3/14 <[email protected]>
>
>
> Over the past week, I've had a bunch of ideas for enhancing
> the Gradle's command line.
>
> My goals are as follows:
>
>
> [1] Allow directory and/or file tasks to be specified naturally on
> the command line, without any weird clutter. They should't
> be second-class citizens in a world where Gradle one day
> handles C++ builds.
>
> [2] Make it easy to specify a list of tasks within different
> namespaces (subprojects), and which tasks in the list
> to execute (or exclude) recursively.
>
> [3] Provide a simple rule for resolving conflicts in whether to
> run a task or not in a convenient, expressive, and
> well-defined way.
>
> [4] Allow tasks to be specified (or skipped) by regex.
>
> [5] Provide a syntax for skipping any task that depends upon
> a given task (or task regex) directly or indirectly.
>
> [6] Provide a way to tunnel a set of args to a task via the
> command line.
>
>
> This email is pretty long, but I think it goes a long way
> to addressing goals [1]-[6]. Hopefully, others will think
> the goals themselves matter, but I'd really like feedback
> on both the goals and the ideas I've presented below for
> addressing them.
>
> Cheers,
> -Jon
>
>
> ------------------------------------------------------------------
> Gory Details:
> ------------------------------------------------------------------
>
> The easy one first...
>
> [1] Directory (or file) tasks names should just be equal
> to their own path name. Thus, to run the 'a/b/c'
> task on all subprjects, just type:
>
> % gradle a/b/c
>
> Note: This one isn't much of a proposal, but I wanted to
> include explicit support for file/dir tasks without
> weird name mangling rules. File/dir task names should
> be first class citizens!
>
> ------------------------------------------------------------------
>
>
> [2] To get fine-grained control over what tasks are recursive
> or non-recursive, adding new options to the following syntax
> isn't quite powerful enough:
>
> % gradle [options] [tasks]
>
> Thus, exploring a "decorator" syntax for tasks seems like
> the way to go. Thinking analogously for a moment:
>
> o In shells (and in ant), something like: '**/*.c'
> means '*.c in all subdirs, recursively'.
>
> o In many languages, the colon (':') or double colon
> ('::') is associated with namespaces.
>
> Let's see where it leads if we:
>
> o Denote recursion in subprojects via the token '::'
> (note that subprojects are a sort of namespace).
>
> o Denote non-recursion via ':'
>
> o Make the last namespace given on the command line remain
> in effect until another namespace specifier is encountered;
> this lets us avoid giving the same long subproject path
> name over and over again if we've got a list of tasks.
>
> Trivial examples
> ----------------
> If the default namespace is the empty string followed by '::',
> and if we make whitespace between a namespace and the task(s)
> within it insignifigant, then the following commands are
> identical:
>
> % gradle foo # normal shorthand form
> % gradle ::foo # '<the-empty-string>::foo' ==
> 'foo'
> % gradle :: foo # whitespace after :: does not
> matter
>
> If we had a list of tasks ('foo' and 'bar') we can extend this
> trivial example a bit further:
>
> % gradle foo bar # normal shorhand form
> % gradle ::foo ::bar # longhand form
> % gradle ::foo bar # :: isn't overriden, so bar ==
> ::bar
> % gradle :: foo bar # prettier formatting for a list
>
>
> Typically though, we'd just say:
> % gradle foo bar # no different from what we do
> today!
>
> So far, so what, right? In the typical case, it just looks like
> the same old command line! Well, let's move on to some fancier
> examples & see where this takes us.
>
>
>
> Fancier examples
> ----------------
> Let's say you wanted to execute the following tasks:
>
> o The foo task (starting from the current project)
>
> o The 'moo', 'cow', and 'egg' tasks recursively
> within the relatively-specified subproject x/y
>
> o The 'c' and 'd' tasks non-recursively
> within the absolutely-specified project /a/b
>
> To do that with the proposed syntax you could say:
>
> % gradle foo x/y:: moo cow egg /a/b: c d
>
> Note that the (recursive) 'x/y::' namespace remains in effect
> until overridden by the (non-recursive) 'a/b:' namespace. Thus,
> the "longhand" version of this would be:
>
>
> # This is just for illustration purposes!
> % gradle ::foo x/y::moo x/y::cow a/b:c a/b:d
>
>
> If task skipping is denoted by a '-' before the task name, then
>
> % gradle /a/b:: moo -cow
>
> would mean:
>
> o Recursively run the moo task within the
> absolutely-specified /a/b subproject.
>
> o If running moo anywhere implies running
> a cow task anywhere, skip the cow task.
>
> Taking this further:
>
> % gradle /a/b:: moo /a/b/c: -cow
>
> would mean:
>
> o Recursively run the moo task within the
> absolutely-specified /a/b subproject.
>
> o If running moo anywhere implies running
> the cow task within /a/b/c, skip that cow task.
> If the cow task is implied elsewhere, then it
> won't be skipped. Notice that '/a/b/c: -cow'
> isn't a recursive skip because of the ':'.
>
>
> One very nice thing about representing namepaces as
> slash-seperated dirs (with a terminating ':' or '::')
> is that it lets you use tab completion!
>
>
> ------------------------------------------------------------------
>
> [3] The decision rule when two contradictory instructions
> are given should be "the last rule wins". Thus:
>
> % gradle foo a/b:: -bar a/b/d:: bar
>
> would mean:
>
> o Recursively run the foo task
>
> o Recursively prohibit the bar task from running
> within the a/b subproject
>
> o Recursively override the prohibition against
> running the bar task in the subproject a/b/c
>
>
> Thus, bar would not run in a project like a/b/c or a/b/e
> because of the 'a/b::-bar' goal, but it would run in a/b/d,
> a/b/d/xyz, a/b/d/foo/bar (and so on) because of the later
> p'a/b/d::bar' recursive override.
>
> There are a few different alternatives to "last rule wins",
> but in considering them, it seems like end up being confusing,
> not very expressive, or ambiguous. This becomes clearer
> when you consider task regexes (see the next section).
>
>
> ------------------------------------------------------------------
>
>
> [4] Allowing tasks to be specified (or skipped) by some sort
> of regex or globbing syntax would be great.
>
> Several issues spring to mind:
>
> o Glob vs regex:
> Glob is usually easier/better for file patterns,
> but could potentially be annoyingly limited when
> it comes to task name patterns.
>
> o If we use "real" regexes, do we use find, match,
> or leading substring pattern semantics?
>
> o Scope of regex/glob: task vs namespace vs global
>
> o Interactions with other task decorators like + or -
> and delimiting the end of decorators with the
> start of the pattern itself.
>
> My current view:
>
> o Because subproject paths correspond to filesystem
> paths, some kind of globbing syntax seems natural.
> for specifying namespaces. There are many flavors
> of globbing out there, but I'm thinking of something
> fairly limited like:
>
> * The '*' character for zero or more characters
>
> * The '?' character for any single character
>
> * The character class ([...]) where you can have a
> specify
> individual characters (via a hyphen). For example,
> '[aeiou3-7]' represents the set of all vowels plus the
> characters '3', '4', '5', '6', and '7'.
>
> * Alternation: '(...|...|...)'
> For example, the glob: 'moo(xxx|yyy)cow'
> is matched by both 'mooxxxcow' and 'mooyyycow'.
> Note: you'd probably need to quote globs
> to avoid interactions with your command
> line shell.
>
> o By default use globbing on task names as well.
> Consider the following command:
>
> % gradle 'hello*xyz'
>
> I think that should mean to run any task that starts with
> the string 'hello' and ends with 'xyz'. If the default
> were
> a regex gramamr, then you'd need to say something more
> like:
>
> % gradle '^hello.*xyz$' # Yuck.
>
> Full regexes are powerful, but they're fairly ugly;
> most of the time, you'd never need all the power they
> provide anyway. Thus I favor treating tasks on the
> command line as glob patterns, not regexes.
>
>
> Still, it would be cool if we *could* do real regexes. I belive
> it's possible if we a sequence like '+~' for regex inclusion
> and '-~' regex exclusion. Because '+' is optional, you should
> also be able to have a bare '~', as long as it's quoted; otherwise
> a shell might confuse this with a homedir.
>
> Thus the following are all equivalent:
>
> % gradle +~test # regex
> % gradle '~test' # regex with bare '~' (suitably
> quoted)
> % gradle '*test*' # glob syntax for "match test
> anywhere"
>
> Here's a nice illustration of where a real regex could be handy:
>
> % gradle '~(?i)test$'
>
> This would mean: recursively run any task that ends with
> the string 'test', matched in a case-insensitive way.
> That's not too far-fetched of an example. Who knows
> what significance people might give to capitalization
> in a set of naming conventions they use.
>
> Of course you could support stuff that really is far-fetched too.
> :)
>
> The one trick to all of this is that the '~' should always come
> immediately before the task pattern if you've got multiple task
> decorators. For example, to exclude a pattern like "contains
> the string 'bar'", you'd say:
>
> % gradle foo -~bar
> not:
> % gradle foo ~-bar
>
> With that one rule, there's never any ambiguity as to whether
> the '-' is part of the pattern (bar) or a negation indicating
> a skipped task.
>
>
> ------------------------------------------------------------------
>
> [5] Sometimes, you're building on a machine that simply can't
> support a particular task. For example, some tasks
> might require other programs that aren't present
> (or resources like network access, licence keys, etc.)
> Maybe only a special "golden build box" has everything
> required to run every single task.
>
> Example:
> Let's say we have two tasks 'foo', and 'bar', each
> of which have a long chains of dependencies:
>
> foo->a->b->c->nasty->d->e->f->g->h->i
> bar->j->k->l->m->n->o->p
>
> Suppose that the build machine you're on can't do
> task 'nasty' for some reason (e.g.: it may require
> something platform specific, licence keys, etc.).
>
> You'd like to be able to say:
>
> % gradle foo bar ...something_magical....
>
> Where ...something_magical... allows gradle to
> be smart enough to *cleanly* skip task 'foo'
> without first doing all the work of building
> i,h,g,f,e, & d, then failing on 'nasty'.
>
> The thing is, I want to be able to do this without digging
> through what 'foo' depends on by hand. I want some way for
> gradle to do that "...something magical..." bit for me!
>
> Here's an idea:
> What if '^' denoted "is wedged in there somewhere".
> In the case given above, we'd be able to say:
>
> % gradle foo bar -^nasty
>
> This would mean:
> Recursively run 'foo' as long as it does not directly
> or indirectly depend on the 'nasty' task.
>
> Recursively run 'bar' as long as it does not directly
> or indirectly depend on the 'nasty' task.
>
> More generally:
>
> -^taskName Excludes any task that ultimately depends
> upon taskName
>
> +^taskName Includes any task that ultimately depends
> upon taskName
>
> ^taskName Same as +^taskName -- the '+' is optional
>
>
> This works nicely with recursive/non-recursive namespaces, negation,
> glob, and regex syntax.
>
> For example, consider:
>
> % gradle foo a/b:-^~test
>
>
> That would mean:
> Recursively run the 'foo' task, but skip any task that
> directly or indirectly depends upon any task that includes
> the string 'test' in the subproject a/b.
>
> NOTE:
>
> o If 'foo' in subproject x/y/z depended on 'mytest'
> in subproject a/b/c, it WOULD run -- 'a/b:' is
> non-recursive.
>
> o If foo in in subproject p/d/q were dependent upon
> a task named 'your_test' in subproject 'a/b'
> then foo would NOT run, as it would match
> the rule a/b:-^~test
>
> A nice side-feature for skipping would be something like
>
> % gradle -n --show-skipped foo -bar ...
>
> Where '-n' could be like the standard "don't really do anything"
> flag (like Gnu Make), and the --show-skipped flag would emit
> a list of whatever tasks were skipped.
>
> The algorithm to make all this would wouldn't even be that hard:
> just process the rules in-order. Discovering whether a task
> ultimately depends on something could be handled by doing a
> no-op "visitor pattern" style dag traversal. You could cache
> the result and if necessary, perform a topological sort before
> executing anything for real.
>
>
> ------------------------------------------------------------------
>
> [6] Providing a way to tunnel a set of args to a task via the
> command line would be nice, because Gradle tasks could be
> used as a sort of dependency-aware program launcher.
>
> Thinking analogously:
> o Many utilities use '-' to indicate stdin
> o A common way of saying "no more switches" is '--'
>
> What if '_' meant "command line argument for task"
>
> Consider the following:
>
> % gradle mytask _ 'foo bar' # case 1
> % gradle mytask _ '[foo, bar]' # case 2
> % gradle mytask _ '[foo:bar, moo:cow]' # case 3
>
> Suppose that when a task executes, it gets a set of parameters
> in a variable called 'arg'. Further, let's say that gradle
> parsed that arg like a string, array, or map (cases 1,2, and 3
> respectively) accordingly. Thus in your build.gradle you
> could have:
>
> createTask("mytask")
> {
> if ( it.arg ...)
> {
> ...
> }
> }
>
> This could then launch your program (or do whatever else you wanted)
> with all the power of resolve() and friends to set classpaths,
> ensure preconditions, and so forth. In effect, it would let
> gradle become a lightweight workflow engine (of sorts), with
> the ability to parameterize ad-hoc actions directly on the command
> line, rather than force everything into resource files. If gradle
> is to be a dsl for dependency management (and not just a build
> system),
> this seems like it could be a pretty big win for very little effort.
>
> Hans and I talked about this a bit yesterday, and he was wondering
> about also adding something like:
>
> % gradle mytask _ '{ ...code... }'
>
> It's a bit unclear to me what the best values of 'this'
> should be within '{...code...}', and whether {...code...}
> should replace action, or just be a way to configure 'mytask'.
>
For this to be handy I think we should use the same that javadoc does for
long commandlines, use @ for commandline expansion by a file, so:
% gradle mytask _ @mytask.custom.code
with @mytask.custom.code contain the ...code... part.
I think it would make sense to allow commandline expansion with @ in
general. When large projects are using all the features of the command line
mentioned here they probably would find this usefull and create a couple of
commandline files that they can easily execute with @... .
> After writing all this stuff down, I think it's time for
> me to step back for a while. :)
>
>
>
>
> Sick with a cold in Boston,
> -Jon
>
>
> ---------------------------------------------------------------------
> To unsubscribe from this list, please visit:
>
> http://xircles.codehaus.org/manage_email
>
>
>
