On 21/01/2013, at 12:24 AM, Adam Murdoch <[email protected]> wrote:
>
> On 21/01/2013, at 8:51 AM, Adam Murdoch wrote:
>
>>
>> On 17/01/2013, at 12:10 PM, Adam Murdoch wrote:
>>
>>>
>>> Another question is how to group source sets and packagings.
>>>
>>> For source sets, we currently use two approaches:
>>>
>>> - For Java, Scala, Groovy, ANTRL and resources, we use the functional
>>> source sets, adding the source to `sourceSets.${function}.${language}`.
>>> - For C++, we use language specific source sets, adding the source to
>>> `cpp.sourceSets.${function}.source` and
>>> `cpp.sourceSets.${function}.exportedHeaders`.
>>>
>>> I'd like to come up with a consistent pattern here, which can allow
>>> arbitrary groupings of source files by language and by function. I can see
>>> three options. For all these options, assume that all source sets are
>>> composable to some degree, so, for example, you can add a given Java source
>>> set to another Java source set, or you can add a given Java source set to a
>>> composite source set.
>>>
>>> 1. Java plugin style, where the primary grouping is functional:
>>> `sourceSets.${function}.${language}`:
>>>
>>> sourceSets {
>>> main {
>>> cpp { srcDirs = '…' }
>>> cppHeaders { srcDirs = '…' }
>>> javaScript { srcDirs = '…' }
>>> }
>>> }
>>>
>>> 2. C++ plugin style, where the primary grouping is by language:
>>> `${language}.sourceSets.${function}`
>>>
>>> java {
>>> sourceSets {
>>> main { srcDirs = '…' }
>>> }
>>> }
>>> groovy {
>>> sourceSets {
>>> main { srcDirs = '…' }
>>> }
>>> }
>>> resources {
>>> sourceSet {
>>> main { srcDirs = '…' }
>>> }
>>> }
>>> javaScript {
>>> sourceSets {
>>> main { srcDirs = '…' }
>>> }
>>> }
>>>
>>> 3. Both, where defining `sourceSets.${function}.${language}` also defines
>>> `${language}.sourceSets.${function}` and vice versa.
>>>
>>> 4. A polymorphic container of source sets. You use whatever groupings you
>>> like, and can add language specific or composite source sets to this
>>> container. The opinionated language plugins would continue to group by
>>> function and add a `main` and `test` composite source set.
>>>
>>> sourceSets {
>>> main(CompositeSourceSet) { // possibly the default type
>>> java { srcDirs = '…' }
>>> cpp { srcDirs = '…' }
>>> resources { srcDirs = '…' }
>>> }
>>> test(GroovySourceSet) {
>>> srcDirs = '...'
>>> }
>>> }
>>>
>>
>> My preference at this stage is to go with option #1. Let's dig into this a
>> bit more.
>>
>> The goal, regardless of whichever grouping option we choose, is to introduce
>> language specific source sets as a layer underneath this. So, we'd add types
>> like JavaSourceSet, GroovySourceSet, CppSourceSet, and so on. These types
>> would have some stuff common - mainly just a set of source files - and some
>> meta-data about the source files:
>>
>> - For a Java source set, this would include the Java language level and Java
>> API that the source is written against, and the compile and runtime
>> dependencies of the source.
>> - Same for Groovy and Scala source sets, except with the Groovy and Scala
>> languages and runtimes. The Java API is also relevant for this source, I
>> guess. These source sets also have some way to declare the compiler
>> macros/AST transforms that the source expects to be available, probably as a
>> set of dependencies on the implementation libraries.
>> - For a C/C++ source set, this would include the language dialect that the
>> source is written against, and the compile, link and runtime dependencies of
>> the source.
>> - For an ANTLR source set, this would include the ANTLR language version
>> that the grammars are written to.
>> - For a Javascript source set, this would include the runtime dependencies
>> of the source.
>>
>> One question is how to model source sets that are related to each other at
>> the language level:
>>
>> - C/C++ source files and their public headers and private headers.
>> - Java (Scala/Groovy) source and their resource source files.
>> - Jointly compiled Java/Scala/Groovy source files.
>>
>> Currently, the C++ plugin groups the C++ source files and headers into a
>> CppSourceSet, with separate SourceDirectorySets for the source and for the
>> headers. The Jvm language plugins group the Java/Scala/Groovy and resources
>> into a SourceSet, with separate SourceDirectorySets for each language. So,
>> these plugins are effectively grouping the source based on the target
>> platform, or by target output component, depending on your view.
>>
>> Do we keep these typed groupings (or something similar), or do we model this
>> as an untyped composite source set that contains a bunch of atomic language
>> source sets? There are some problems with the current groupings:
>>
>> - The ANTLR source is currently attached to the 'jvm' group, but ANTLR can
>> generate C, C#, and bunch of other languages.
>> - Java source and JVM byte code can be compiled to native code.
>> - Groovy, Scala and ANTLR source are added in dynamically via a convention
>> object, so don't make use of the typing anyway. It would be the same for
>> native languages other than C/C++ as well. So you've got this first-and
>> second-class thing going on. It feels like a good solution should not treat
>> certain languages specially.
>> - Sometimes multiple groups of source in a given language make up a logical
>> group. Eg API + impl, java 5 + java 6, windows + posix, etc.
>>
>> Let's say we remove the typed groupings (in a backwards compatible way, as
>> always). It might work something like this:
>>
>> - Some basic 'language' plugin adds the concept of (composite) source sets.
>> You can define source sets and add whichever language source sets you like.
>> - A 'jvm-language' plugin adds a rule that adds a resources source set to
>> each source set, adds the concept of JVM packagings, and a rule that knows
>> how to copy the resource files that are inputs to a JVM packaging.
>> - A 'java-language' plugin adds a rule that adds a Java source set to each
>> source set, and a rule that knows how to compile the Java source that are
>> inputs to a JVM packaging.
>> - Groovy and Scala language plugins, as above.
>> - A 'standard-source-sets' plugin adds the 'main' and 'test' source sets.
>> - The 'java-base' plugin adds a rule that adds a classes dir packaging for
>> each source set, and wires up the source set as input to the packaging.
>> - The 'java', 'groovy' and 'scala' plugins just apply various combinations
>> of the above plugins.
>> - The 'android' plugin adds an Android resources source set to each source
>> set (these are different to the JVM resource files above), and APK
>> packaging, and a rule that knows how to compile the resources and classes
>> packaging into an APK packaging.
>> - A 'native-language' plugin adds the concept of native packagings.
>> - A 'cpp-language' plugin adds a C/C++ source set, a public headers source
>> set and a private headers source set to each source set, and a rule that
>> compiles C/C++ source that are inputs to a native packaging.
>> - The 'cpp-lib' and 'cpp-exe' plugins just apply various combinations of the
>> above plugins.
>> - An 'assembly-language' plugin adds an assembly source set to each source
>> set and a rule that compiles assembly source that are inputs to a native
>> packaging.
>> - A 'javascript-language' plugin adds a Javascript source set to each source
>> set, and the concept of a Javascript packaging.
>
> Let's tweak this a bit. In the above, adding support for a language also adds
> one or more language source sets to every source set, which is isn't quite
> right. Some common cases where that doesn't reflect reality:
>
> - A Java project uses Groovy for testing.
> - A project uses ANTLR to generate production code but not test code.
> - A Java project bundles a JNI library, so that it has Java and C production
> code, but all the tests are written in Java.
>
> It might be better to flip things around and infer the language source sets
> based what we need to build, and on convention:
>
> - Each of the JVM -language plugins add rules that can compile the language
> source that forms inputs to a JVM packaging. And that can generate the API
> docs from the language source that forms inputs to an API documentation
> packaging.
> - Each of the native -language plugins add rules that can compile the
> language source that forms inputs to a native packaging.
> - The antlr-language plugin adds rules that can generate source from ANTLR
> source sets that form inputs to a language source set.
> - An opinionated 'jvm-library' plugin states that for each JVM library
> component 'n', there is a jar packaging 'n', which has an input classes
> packaging 'n', which has as input a production source set 'n', which includes
> a source set for each supported JVM language.
> - This plugin may defer adding the language source set until it is
> either referenced (to be configured) or the conventional source dir is not
> empty.
> - This would mean, in turn, that we wouldn't add compile tasks for
> languages that are not required to build the classes packaging.
> - An opinionated 'jvm-unit-tests' plugin that states that each project has a
> single 'test' classes packaging, which has as input a test source set 'test',
> which includes a source set for each supported JVM language. Plus adds the
> appropriate test task.
> - The java plugin applies the the java-language, jvm-library and unit-tests
> plugins, and defines a single 'main' JVM library.
> - Similarly, an opinionated 'native-component' plugin states that for each
> native component 'n', there is a binary packaging 'n', which as as input
> object file packaging 'n', which has as input a production source set 'n',
> which includes a source set of each supported native language.
> - An opinionated 'native-unit-tests' plugin, does the same kind of thing as
> the 'jvm-unit-tests' plugin.
> - The cpp-lib and cpp-exe plugins apply the cpp-language, native-component
> and native-unit-tests plugins and define a 'main' library or executable.
>
> In other words:
>
> - The 'capability' plugins add classes of things, and rules to build a thing
> from its input things.
> - The 'opinionated' plugins that add instances of things, and rules that
> state what a given thing's inputs are.
That seems quite right to me.
Could we build on the mutable polypmorphic container idea for this?
Ignore backwards compatibility for a second…
A project has a…
interface SourceSetContainer extends NamedDomainObjectCollection<SourceSet> {}
interface SourceSet extends Named {
LanguageSourceSetContainer getLanguages()
}
interface LanguageSourceSetContainer extends
NamedDomainObjectCollection<SourceSet> {}
interface LanguageSourceSet extends Named {}
So…
sourceSets {
main {
languages {
java { }
}
}
}
If that's too much nesting, we could always offer other views, but I think
that's the data structure.
I imagine we could use rules (or something like it) for the defaulting…
sourceSets.all {
languages.addRule("Java Language Source Set") {
if (it == "java") {
languages.add("java", JavaLanguageSourceSet) {
// do conventional stuff
}
}
}
}
What happens if there are two bunches of Java to go into the same functional
source set? Can't think of why that would be the case, and we can probably just
say that that's not what it's used for. I guess there's nothing stopping
someone creating a component that is the .class file output from two different
source sets.
>
>
>>
>>
>>> Thoughts? Other options?
>>>
>>> On 16/01/2013, at 5:06 PM, Adam Murdoch wrote:
>>>
>>>> Hi,
>>>>
>>>> To better support building Android apps (and other things), we want to
>>>> rework the jvm language plugins so that they can handle building multiple
>>>> outputs from a given set of source files.
>>>>
>>>> I've made a bit of a start on a spec here, but's pretty rough:
>>>> https://github.com/gradle/gradle/blob/master/design-docs/building-multiple-outputs-from-jvm-languages.md
>>>>
>>>> I need some suggestions for terminology:
>>>>
>>>> 1. A term for the things that Gradle builds. With this work, plus
>>>> publications, components, reports and distributions work that is
>>>> happening, we're starting to model more of the things that Gradle can
>>>> build. It feels like we should have a term for this. So far we've been
>>>> calling these things 'things' and sometimes 'outputs'. I kind of like the
>>>> term 'build item' from abuild.
>>>>
>>>> 2. A term for a thing that runs on a JVM. This is the focus of the spec
>>>> above. The spec calls these things a 'packaging', but this doesn't really
>>>> work that well. Note that this isn't the logical thing - we're calling
>>>> them 'components' - but the physical thing. Initially, there are 2 types
>>>> of this thing - a class directory packaging and a JAR packaging. If we
>>>> come up with a good term for 'things' in #1 above, we could just shove
>>>> 'jvm' in front of it as a name for these things (e.g. 'jvm build item').
>>>>
>>>> 3. A term for a set of source of a specific language, that forms the input
>>>> to a compilation step. This is different to what we call 'source set' at
>>>> the moment - that's a logical grouping of source. The spec calls these
>>>> things a 'language source set'.
>>>>
>>>> 4. A term for a logical set of source. Currently we call these a 'source
>>>> set'. The spec calls these things a 'functional source set'.
>>>>
>>>> Suggestions?
>>>>
>>>> --
>>>> Adam Murdoch
>>>> Gradle Co-founder
>>>> http://www.gradle.org
>>>> VP of Engineering, Gradleware Inc. - Gradle Training, Support, Consulting
>>>> http://www.gradleware.com
>>>>
>>>
>>>
>>> --
>>> Adam Murdoch
>>> Gradle Co-founder
>>> http://www.gradle.org
>>> VP of Engineering, Gradleware Inc. - Gradle Training, Support, Consulting
>>> http://www.gradleware.com
>>>
>>
>>
>> --
>> Adam Murdoch
>> Gradle Co-founder
>> http://www.gradle.org
>> VP of Engineering, Gradleware Inc. - Gradle Training, Support, Consulting
>> http://www.gradleware.com
>>
>
>
> --
> Adam Murdoch
> Gradle Co-founder
> http://www.gradle.org
> VP of Engineering, Gradleware Inc. - Gradle Training, Support, Consulting
> http://www.gradleware.com
>
--
Luke Daley
Principal Engineer, Gradleware
http://gradleware.com
---------------------------------------------------------------------
To unsubscribe from this list, please visit:
http://xircles.codehaus.org/manage_email
