On 03/07/2012, at 10:45 PM, Adam Murdoch wrote: > > On 28/06/2012, at 11:40 AM, Daz DeBoer wrote: > >> For java code, it seems like there are at least 2 axes that could contribute >> to the artifact 'type': >> 1) client vs api vs test vs core >> 2) code vs source vs javadoc >> >> Would these all be wrapped up under the single concept of 'type'? It seems >> like they are both valuable things to model explicitly. > > Right. We should treat them separately. The idea is that an artefact's type > defines the format of the artefact, and how the artefact can be used. A jar, > for example, can really only be used in a classpath. A set of c++ headers can > really only be used for c++ compilation.
I don't think this is going to work, “type” is just not rich enough. > The artefact type doesn't say anything about the purpose of the artefact, > just its format. It doesn't answer: what's the meaning of the stuff inside > the artefact? ie. which class paths does this particular jar belong in? > > My thought was to model this at the component level, rather than the artefact > level. A component type would have a set of 'usages' associated with it. A > usage would imply a set of zero or more of the component's artefacts plus a > set of zero or more dependencies on other components. I think a component is a graph. A “usage” is just a particular query of the graph. > We could potentially model this at the artefact level as well, as the > artefact's 'purpose'. For a given artefact, there is a many-to-many > relationship with the places it needs to be used. For example, if I package > my component in a single jar, then that jar must be included in the > consumer's compile classpath, runtime classpath, test compilation classpath, > and so on. Having the thing specify how it should be used is not right I think. Instead, it should just clearly express what it is (via advertised facts) and the consumer can query appropriately. > If I package my component as an api jar + impl jar, then the api jar must be > included in the consumer's compile classpath, both jars in the runtime > classpath, and so on. The issue is that this is going to be domain specific. Resolving jars will be different to js libraries etc. I don't think we can try and abstract over all of this successfully. I think we need to consider radical changes to succeed here. For example, forget abstraction and embrace dependency sets and adapting. I don't think this is the right time to go into this, but I just want to make the point that I think what we currently do (which is what we inherited) is so deeply flawed that gradual evolution may not be appropriate. I think we need a ground up rethink, then work out how to get there. > The question is whether a source or javadoc artefact is a 'format' or a > 'purpose'. I'd say it's both: a source zip has a particular format (i.e. it's > a zip), and a particular purpose (i.e. it contains source). You could package > up the source in any archive format and still have a source artefact (e.g. as > a tgz in for a c++ based component). Its relationship to something else is what makes it a “source” artefact. It then has further characteristics such as whether it's a zip, jar etc. I may be able to handle either. The relationships are completely missing from our current data structures, and they are crucially important. It's the same for variants. > > >> Daz >> >> On 19 June 2012 16:59, Adam Murdoch <[email protected]> wrote: >> Hi, >> >> There are a couple of dependency management issues we want to fix by >> introducing the concept of an artefact type. >> >> Currently, artefacts are opaque generic things that we know nothing about. >> Which means we can't do anything particularly useful other than copy them >> around or download and plonk them in a cache directory. There are some >> problems with this: >> * Native executables need to follow a particular platform-specific naming >> scheme (or at least, should) and need to have the execute bit set. >> * Native libraries need to follow a particular platform-specific naming >> scheme, and the name should honour the soname/install_path baked into the >> binary. >> * Some types of artefacts, such as c++ headers or javascript source files or >> api documentation, need to be arranged in a particular hierarchy relative to >> each other (or, some artefacts represent a file tree, rather than a file). >> * There's no well-defined way to find the source and documentation of a >> component that we're using. >> * There's no good identifier for an artefact. Currently, you can use any >> attribute from the union of those supported by ivy and maven. Which means >> that you have to couple your dependency declarations to how the dependency >> happens to be published. >> >> Instead, I think we want to change things so that we explicitly model >> artefact types and define a set of well-known types. >> >> 1. An artifact will have a name and a type. >> 2. The artefact (name, type) must be unique for a given module. >> 3. The artefact extension and classifier attributes will be deprecated and >> removed from our DSL. >> 4. When publishing to an Ivy repository, the type will be used to map the >> name to an Ivy artefact with (name, extension, type, m:classifier), which >> then feeds into the artefact pattern to end up with a destination location >> for the artefact. >> 5. When publishing to a Maven repository, the type will be used to map to >> name to a Maven artefact with (artifactId, extension, classifier), which >> then is used to determine the destination location for the artefact. >> >> Things get more interesting when resolving. >> >> A Maven dependency declaration has a 'type' attribute associated with it, >> which defaults to 'jar'. Maven maps this type to an (extension, classifier) >> that it uses to look for the target artefact. One question is how we >> interpret the type attribute. Do we: >> * Map (group, artifactId, version, type) to Gradle module (group, >> artifactId, version) and artifact (name, type), then do an exact match on >> the name + type from the artefacts of the target module? >> * Map (group, artifactId, version, type) to Gradle module (group, >> artifactId, version) and artefact type, then select the artefacts of the >> target module that have the specified type? >> >> The result will be the same when the dependency resolves to a module in a >> maven repository. It may be different when the dependency resolves to a >> module in an ivy repository. I think the second option works better, as it >> gives the consumer the opportunity to state what type of things it is after, >> and the producer the opportunity to decide how the artefacts are structured. >> >> Another question is how we interpret a missing type attribute in a maven >> dependency declaration. Do we: >> * Always look for artefacts of type 'jar'? >> * Look for artefacts of a type that is specific to the resolve we're doing? >> That is, look for jars when resolving the Java compile classpath, look for >> c++ headers when resolving the C++ compile paths, and so on. >> >> I like the second option, but it's probably not the right way to go here. It >> might be useful, in that it would allow us to publish single variant non-jar >> projects to an existing maven repository and consume these Gradle-produced >> artefacts from another Gradle build via the Maven repository. >> >> An ivy dependency declaration allows you to request one or more artefacts >> identified by (name, type, extension). We would map the (type, extension) to >> a Gradle artefact type, and then back to the repository specific type when >> looking for the artefact in a particular repository. >> >> An ivy dependency also allows you to filter the artefacts by name, type >> and/or extension. We would map the repository specific type to an ivy >> artefact type and apply the filters. >> >> What this means is that, except for some adapters in the resolver, our >> resolve and publish code would deal with the Gradle concept of an artefact >> rather than the merged Ivy + maven concept that we use now. We would also >> expose the Gradle artefact in the resolve results. >> >> >> -- >> Adam Murdoch >> Gradle Co-founder >> http://www.gradle.org >> VP of Engineering, Gradleware Inc. - Gradle Training, Support, Consulting >> http://www.gradleware.com >> >> >> >> >> -- >> Darrell (Daz) DeBoer >> Principal Engineer, Gradleware >> http://www.gradleware.com >> > > > -- > Adam Murdoch > Gradle Co-founder > http://www.gradle.org > VP of Engineering, Gradleware Inc. - Gradle Training, Support, Consulting > http://www.gradleware.com > -- Luke Daley Principal Engineer, Gradleware http://gradleware.com
