Hey Adam, this looks good. I'm sure there's some non-android implication that I'm missing but it looks like it would server our needs. thanks.
On Thu, Nov 29, 2012 at 12:39 AM, Adam Murdoch <[email protected]>wrote: > > On 29/11/2012, at 8:14 AM, Xavier Ducrohet wrote: > > Hello all, > > as it’s my first post on this list, some context: I’m the lead for the > Android dev tools at Google and I’m working on our new build system based > on Gradle. I’ve worked with Adam and Hans in the last few months to get the > project started. > > We have a concept of build variants in our plugin which allow a project to > build different versions of the same app (either different flavor of it, or > different builds of it -- debug vs release for instance). Each variants has > its own source sets, it own dependencies and its own output. One additional > complication: some of the source sets could be used by multiple variants. > > IDE integration is critical to us and we need to make sure that the > Tooling APIs can surface this to the IDEs. The idea is that the developer > would choose which variant to work on, thus enabling a specific set of > source sets in the IDE, setting up the current classpath, and selecting the > proper output to run when requested. > > Additionally, we’d like to have other plugins be able to act on Android > projects. For instance the FindBugs plugin would have to be able to know > what sourcesets to use together, where the compilation output is, etc... > > I’d like to propose that Gradle adds native support for build variants, > and that plugins should be able to “register” variants (our variants are > automatically created from items defined in the build.gradle file), and the > API should let other plugins (and the IDE) query for variants and their > configuration (dependencies, source sets, output, etc...) > > I think the biggest challenge is handling sourcesets. The current Java > SourceSet is very specific to the Java plugin and includes not only the > source but also the output. Also, creating a SourceSet automatically > creates a set of tasks to compile this SourceSet, which was not good for us. > Unfortunately, this is what most other plugins rely on, querying Java > sourcesets and their output (for instance the STS plugin does this, I’m > sure the FindBug plugin does this as well). > > > This is all good stuff to do. Plus there are other nice things Gradle can > do in the Java/JVM space once we detangle our plugins from the idea that > there is exactly one output from a jvm-based project. > > I'd see a model something like this: > > * A project can produce one or more "jvm components" - things that can run > on a JVM. Some examples: > * A library > * A command-line application > * A web application > * An Android application > * An Android library > * Each jvm component can be "packaged" in one or more ways, depending on > its type: > * A directory (or directories) containing classes and resources. > * A jar. > * A dex file. > * An apk. > * An Android library. > * A native shared library, executable, or .net assembly, etc. > * Every packaging has some associated meta-data, such as runtime > dependencies, byte code version, platform api version, etc. > * Not every packaging makes sense for every component, but every jvm > component can be packaged as a directory containing classes (and almost > always, the final packaging is some transformation of this packaging). A > given packaging may not necessarily be usable. > * Not every packaging represents a component. For example, the unit tests > for a component can be packaged in one or more ways as well. > * Every packaging is built from some source files. > * Source files are grouped into source sets. > * Source sets contain source files and can contain other source sets. They > have some meta-data. > * Some source sets are language specific, containing only source files of > a certain type plus meta-data specific to that type of source. > * Language plugins add support for taking a language specific source set > and transforming it in some way (e.g. compiling or copying or generating > resources or whatever) into the appropriate packaging. > * Some source sets represent a logical grouping, such as main or unit test > source. Exactly which logical groupings make sense for a given component > depends on its type. Usually a logical source set will contain source from > several different languages. > > I've avoided using the term 'variant', as we're still sorting out exactly > what 'variant' means. Quite possibly the same thing as 'packaging' above. > Certainly each variant would be represented by a packaging, but possibly > not every packaging is a variant. > > I don't think we need to care too much about components or variants > initially. We could probably start with something like: > > Assuming that: > * The Java/Scala/Groovy plugins add a 'main' and a 'test' logical source > set. > * The Android plugins add a 'main' and a 'test' logical source set, plus > one for each product flavor and build type. > > Then, we can: > * Add some container of class directory packagings. > * The Java/Scala/Groovy plugins would add 1 class directory packaging for > each logical source set. The Android plugins would add 1 packaging per > variant. > * Add some way to attach source sets to class directory packagings. The > base language plugins would know how to take a java source set and compile > it into the classes directory, or take a resource source set and copy it to > the classes directory, and so on. > * The Java/Scala/Groovy plugins would attach each logical source set to > its corresponding packaging. The Android plugins would attach the source > sets for a variant (main or test, product flavor and build type, etc) to > the corresponding packaging. > * Add some way to create a source set that contains generated source. > * The Android plugin would create a source set for the source it generates > for each variant, and attach this source set to the appropriate packaging. > * The Antlr plugin would create a source set for the Java source it > generates, and attach this to the appropriate logical source set (which > would mean it would work with the Android plugins). > * The code quality plugins would continue to inspect each logical source > set. We'd need to do something to choose the appropriate classes to pass to > those tools that work off the byte code. > * Add some way to attach source sets to the various IDE source paths and > class paths. > * The Java/Scala/Groovy plugins would add each logical source set to the > Eclipse source and class paths, and would add the 'main' and 'test' logical > source sets to the IDE source and class paths. > * The Android plugins would add the source sets for the selected variant > to the Eclipse and IDEA source and class paths. > * It might make sense to add a logical source set for each Android > variant, that is a composite made up of the various source sets for the > variant. > > Then later, we can build more stuff on top of this, introducing components > and variants and so on. > > > > In our case we have our plugin do things a bit differently. We don’t use > the normal SourceSet and have rolled up our own to decouple the output from > the source and not create tasks we don’t need. Instead we create our own > JavaCompile tasks and wire the input and output manually based on the > current variant and its sourcesets. > We’d like to see if we can separate the notion of source folders from the > output (and the creation of the tasks). > > On top of Java we also have JNI code as well as other languages (that are > specific to Android). I’m not sure if we are going to use the CppSourceSet > just yet as we are likely to simply call out to our make-based build system > for those for now. > In general though, we’d like to make sure the tooling API is flexible and > that the concept of SourceSet is abstract enough so that we can use our > own, while still benefiting from the ecosystem of Gradle plugins > (Disclaimer: I’m not super familiar with the Tooling API yet) > > > I would see the JNI stuff working the same way as the Java stuff: A JNI > library can be packaged as a shared library or a set of object files, a > packaging is built from source files grouped into source sets, some of the > source is generated, base plugins know how to take a C or C++ source set > and compile it to object files, and how to take object files and link them > into shared libraries, and so on. > > > > We are starting to look at IDE integration, and there are a lot of changes > to make there, but we’re going to need these changes in Gradle to be able > to make progress on the IDE side. > > I realize this is a lot of work there. Adam and I already talked about it > a bit. We're happy to help but you guys should probably figure out the > internal design first :) > > > I'll start putting a spec together for this. > > > -- > Adam Murdoch > Gradle Co-founder > http://www.gradle.org > VP of Engineering, Gradleware Inc. - Gradle Training, Support, Consulting > http://www.gradleware.com > >
