On 1 Aug 2014, at 9:18 pm, Szczepan Faber <szcze...@gmail.com> wrote:
> More thoughts on conflict resolution, Adam-style :)
>
> 1. The starting use case is avoiding conflicting guava and
> google-collections on the same classpath. We'll enable that by
> offering an API to declare that a module (group+name)
> replaces/supersedes/deprecates a different module (group+name). This
> use is a starting point to more interesting use cases which I will
> describe in later. While designing the DSL and the behavior we need to
> be cautious about those other use cases, so that we can develop this
> feature incrementally. We will use guava an collections as an example.
>
> 2. In a standalone project, a guava-collections issue is not that
> problematic, it can be sorted out by editing the build file (change
> the dependencies, add an 'exclude', add force rule, etc.). The use
> case is getting interesting in on an enterprise level, where the
> organisation wants to solve the problem in the shared plugin suite,
> instead of getting each team solve the problem on their own.
> Well-known deprecations like collections-guava could be even encoded
> in Gradle out of the box, but that's a different topic.
Absolutely we should. There are some backwards compatibility implications if we
were to add more meta-data about certain modules over time.
>
> 3. Current APIs for influencing the dependency resolution (forcing
> versions, dependency resolve rules) offer only "unconditional"
> manipulation to the dependencies that are being resolved. Let's map
> the collections-guava problem to the current API: we could force
> collections to become guava via dependency resolve rule. The effect
> is:
> - only collections in graph - guava is used
> a) only guava in graph - guava is used
> b) both collections in graph - guava is used (collections are forced
> into guava)
> c) only collections in graph - guava is used
> This is what I mean by 'unconditional' manipulation - if collections
> appear in graph it is _always_ replaced with guava. And this is not
> quite what we want...
>
> 4. We want the replacement rule to get activated only when both
> collections and guava are in the graph.
I would think about it somewhat differently. We don’t want to conditionally
activate some rule here. Instead, we want to use some facts about collections
and guava to both detect conflicts between the two and also to resolve such
conflicts.
So, collections was replaced by guava. This is a fact that is true regardless
of whether one or both or neither of them are in the graph.
This fact implies a few further facts:
- Collections and guava conflict with each other. They provide the same API and
shouldn’t both be present in a classpath. When resolving a dependency graph, we
have to select one of these to use.
- Guava is newer than collections, so if we happen to resolve conflicts by
selecting the newer of two conflicting modules, we should select the guava
module over the collections module.
We’d use these facts in other places. For example, when reporting on
dependencies that are out-of-date, we should consider versions of guava as
candidate upgrades for any dependencies on collections. Same when we’re
offering suggestions in the IDE or when generating a dependency lock file.
For this use case, what we’re after is a DSL to let you state these facts about
various modules. We don’t want an DSL where you provide imperative code that
implements the mechanics of detecting or resolving the conflicts. This allows
us to implement everything efficiently and also to reuse these facts in other
contexts beyond resolving a dependency graph.
> If only collections appear in
> the graph - there is no need to replace it with guava. Perhaps guava
> and collection is not the best example because early guava is
> completely backwards compatible with latest collections. So in theory,
> we could unconditionally replace collections with an early version of
> guava via existing dependency resolve rule API. However, let's imagine
> that replacement source and target are not that quite compatible or
> that someone is using an _early_ version of collections which is not
> quite compatible with _early_ guava.
Compatibility is a different concern.
You can think of the different versions of a given module in the same way as
replacement of one module by another. If I have A version 1.2 and A version
1.3, then we can say 'A version 1.2 was replaced by A version 1.3’, with the
same implications.
All we’re doing with replacement is saying something about the history of some
thing. This doesn’t say anything about its compatibility over time. Or, more
accurately, this doesn’t say anything more than what the version of a thing
says about compatibility.
Similar to the history of a thing, there are also facts about the compatibility
of a thing over time that we want to use as input to resolving a dependency
graph, as part of resolving conflicts. If I have A and B in the dependency
graph and they conflict, and A is newer than B, then I can select A only if A
is compatible with B. Otherwise I have to either fail the resolve or find some
C that is newer than A and B and that is compatible with both of them.
We’d also use facts about compatibility when resolving a graph where things are
provided - eg the android SDK or the Windows API or whatever: If I have A and B
is in the dependency graph and they conflict, and A is provided (and so can’t
be substituted) and A is not compatible with B, then I have to fail the resolve.
Facts about compatibility are useful in other places beyond dependency
resolution. For example, when updating a dependency lock file, we might skip
candidates that are not compatible with what we’re currently using. Or when
offering suggestions in the IDE we might distinguish between candidates that
are compatible and not compatible.
>
> 5. The implementation is tricker than our current forcing/dependency
> resolve rules. Consider dependency graph traversal scenarios:
> - first we encounter collections, we happily use it and continue traversing.
> - then we encounter guava - oups - we need to evict already resolved
> module collections and continue traversing. (It's more complicated
> than that, as we resolve conflicts only when we cannot traverse any
> further).
>
> In fact, it does not really differ from the standard, version-based
> conflict resolution. I wanted to point out the difference between
> existing dependency resolve rules implementation. The latter does not
> need to consider already-resolved modules.
That’s right. The things we currently call ‘dependency resolve rules’ are
dependency (edge) replacement rules. They don’t have anything with conflict
handling. They’re very much not ‘everything that affects dependency resolution’
rules.
>
> 6. Other features to consider:
> - module is replaced with a set of modules ('spring' ->
> 'spring-core', 'spring-aop', etc.)
> - set of modules is replaced with a single module (I don't know any
> real example for that)
> - api module and impl module use consistent version
>
> 7. Given that there are more ways to influence the dependency
> resolution, and specifically, to replace/force certain dependencies
> with others we should think about improving diagnostics on why certain
> dependency ended up in the graph and why this particular version is
> used. Our reports and API already provides 'selection reason'
> information but I'm afraid it's not enough. A dependency can be
> manipulated during resolution by different rules, applied by different
> plugins and a flat information like 'dependency resolve rule was used'
> is not enough IMHO. There can be a stream of actions that affects the
> selected version of a dependency.
This is something the general-purpose model rules has to address, and so the
plan is that the ad hoc rule handling that we currently have in a few places in
dependency management will be replaced by the general-purpose stuff, and will
pick up the insight and reporting goodness that it provides.
>
> 8. I made up my mind about the DSL. I'm easy and I can totally be
> convinced to something else. Yet, here's my current preference:
>
> a) I would reuse some existing low-level, imperative API, like the
> component rules or dependency resolve rules, for example:
>
> eachDependency { details ->
> if (details.target.name == 'google-collections') {
> details.replacedBy ‘com.google.guava:guava'
The eachDependency() method is used to replace one dependency edge with
another. It doesn’t make any sense to say something about the history of a
module here.
If we were to add a low level mechanism, it should be given the meta-data for a
given module and allow the rules to say things like:
- This module conflicts with modules with these ids.
- This module is newer than modules with these ids.
- This module is or is not compatible with modules with these ids.
> }
> }
>
> b) I would add some higher level api that would use a). For example:
>
> dependencies {
> components {
>
> modules("com.google.collections:google-collections").deprecatedBy("com.google.guava:guava")
> //or replacedBy(), supersededBy()
> }
> }
>
> Thoughts?
We should only add the high level API.
>
> On Fri, Aug 1, 2014 at 11:30 AM, Szczepan Faber <szcze...@gmail.com> wrote:
>> This is a typo, the spec is work in progress. Initially we thought
>> about 'replaces' but we've changed that into 'replacedBy' but I forgot
>> to reverse the source and target :)
>>
>> So:
>>
>> google-collections is replaced by guava
>> guava replaces google-collections
>>
>> Thanks for pointing this out. I'll do more speccing and get back to
>> the mailing list.
>>
>> Cheers!
>>
>> On Fri, Aug 1, 2014 at 9:38 AM, Heinrich Filter
>> <heinrich.fil...@mmiholdings.co.za> wrote:
>>> Hi Szczepan,
>>>
>>> I'm new around the Gradle community but would love to start contributing
>>> more. Please feel free to point out any "conventions" that I miss.
>>>
>>> For me the "replacedBy" keyword is a bit confusing. In your example:
>>> 'guava' was replace by 'google-collections'. In my mind that implies that
>>> 'google-collection' is the *newer* dependency and not, as you state in the
>>> doc, the older version.
>>>
>>> I would think that 'guava' rather *replaces* or *supersedes*
>>> 'google-collections'.
>>>
>>> Or am thinking about this the wrong way?
>>>
>>> Regards,
>>> Heinrich
>>>
>>> On 31 Jul 2014, at 3:27 PM, Szczepan Faber <szcze...@gmail.com> wrote:
>>>
>>>> Hey,
>>>>
>>>> I've started speccing improvements to conflict resolution here:
>>>> https://github.com/gradle/gradle/blob/master/design-docs/conflict-resolution-spec.md
>>>> (there's a big DSL mock-up down the bottom).
>>>>
>>>> It would be good to start discussing the DSL additions here on the
>>>> mailing list. Please give feedback about the DSL or the use cases,
>>>> etc. I haven't decided myself on the DSL :) I'll keep you in loop on
>>>> how the design goes.
>>>>
>>>> Cheers!
>>>> --
>>>> Szczepan Faber
>>>> Core dev@gradle; Founder@mockito
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe from this list, please visit:
>>>>
>>>> http://xircles.codehaus.org/manage_email
>>>>
>>>>
>>>
>>>
>>> **********************************************************************
>>> This email and all content are subject to the following disclaimer
>>>
>>> http://content.momentum.co.za/content/legal/disclaimer_email.htm
>>>
>>> **********************************************************************
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe from this list, please visit:
>>>
>>> http://xircles.codehaus.org/manage_email
>>>
>>>
>>
>>
>>
>> --
>> Szczepan Faber
>> Core dev@gradle; Founder@mockito
>
>
>
> --
> Szczepan Faber
> Core dev@gradle; Founder@mockito
>
> ---------------------------------------------------------------------
> To unsubscribe from this list, please visit:
>
> http://xircles.codehaus.org/manage_email
>
>
--
Adam Murdoch
Gradle Co-founder
http://www.gradle.org
CTO Gradleware Inc. - Gradle Training, Support, Consulting
http://www.gradleware.com
