On Dec 7, 2012, at 2:10 AM, Adam Murdoch <[email protected]> wrote:
> > Here are a couple of interesting cases we'll need to consider in whatever > plan we come up with for dependency notations. > > 1. BuildSrc as a regular project. > > The plan to get rid of BuildSrc was to allow project dependencies in the > build script classpath, and Gradle take care of building those projects > before running the build scripts that need them. Just to be clear. We don't want to get rid of buildSrc but no longer make it a special case and make the concept much more flexible. Hans > > If we decide to use the same namespace for project and external dependencies, > then we'll have to have some way to know at script compilation time which > build script dependencies are local and which are not. Which means running > the scripts to find out what each project produces. > > I think the caching approach still works here. We'd need to defer the > configuration of a given project until we know which of the script > dependencies are local and which are not, or the project has no build script > dependencies. Things might get tricky when the build time project build > scripts have external dependencies (not the build time projects, just their > build scripts). Maybe we'd need some guidance from the settings.gradle in > this instance or perhaps just not support this case. > > Or maybe we change the plan. > > 2. Substituting a partial dependency graph > > Say we have the following dependencies: > > lib-A -> lib-B -> lib-C > > Let's say I want to build lib-A and lib-C locally, but download lib-B (for > example, let's say 'lib-B' is actually 150 separate things and I don't really > want to build them, because I'm not changing the API of lib-C). > > If we decide to use the same namespace for project and external dependencies, > then in order to decide which projects to configure, and which things to > build, we effectively have to do a full resolve of the dependencies of lib-A. > Which is fine - We need to do this at some point in the build anyway. The > issue here is more around when and how this happens. > > One option is to do this on demand, so that as we execute tasks and resolve > their inputs, we discover the projects that need to be configured and built. > The downside here is that we can no longer assemble the full task graph > before we start executing tasks. In fact, we can't tell you what the full > task graph is until the end of the build. > > Another option is to do the resolve when assembling the task graph, so that > we resolve the task inputs enough so that we can figure out which projects > and tasks are required to build the inputs of a given task. The downside here > is that you won't be able to change these dependencies after the task graph > has been built. But this is the case now, so I think this is fine. > > Either way, we're going to have to be able to run some tasks as we're > building the task graph, for the 'BuildSrc as regular projects' case. > > > On 24/11/2012, at 7:50 AM, Adam Murdoch wrote: > >> Hi, >> >> One question that we need to answer to make progress on the publication DSL >> is how should change our dependency notation, so that we can: >> >> 1. Substitute a reference to a locally built thing with something downloaded >> from a repository [1]. For example, when I partially check out a source >> tree, I want to build the things for which there is source, and download the >> things for which there is no source, without changing the build scripts. >> >> 2. Substitute a reference to a thing in a repository with something built >> locally. For example, when I'm working on multiple source trees, I want to >> build a local version of some things that would otherwise be downloaded, >> without changing the build scripts. >> >> 3. Skip the configuration of projects which are not required to build the >> local things we need. For example, when I'm working on source tree with many >> projects, I want Gradle to configure only those projects that are relevant >> to whatever I'm doing. >> >> 4. Distinguish between the various things built by a project. For example, I >> have a project that builds a web app and a java library, and in some places >> I want to refer to the java library and in other places I want to refer to >> the web application. >> >> [1] "Downloaded from a repository" really means "fetched from somewhere >> else". It may or may not be downloaded and it may or may not come from a >> repository. >> >> Just as some background, the basic model we want to aim for is this: >> >> 1. Projects produce components. >> 2. A component is some logical thing that I can use, such as a java library >> or web application. >> 3. Dependencies are criteria that select one or more components. >> >> There's more detail here: >> https://github.com/gradle/gradle/blob/master/design-docs/dependency-model.md >> >> A few options (not all good options - just brainstorming here): >> >> 1. References to locally built things and things in a repository share the >> existing namespace we use for things in a repository. That is, every >> dependency declaration would use (group, component, version), including >> project dependencies: >> >> dependencies { >> compile 'my.org:some-repo-thing:1.2' >> compile 'my.org:some-other-project:1.4' // A project dependency >> } >> >> The obvious downside here is that version is required for a project >> dependency, but version is almost always a calculated value. Another >> downside is that projects already have a convenient identifier in their >> path, whereas this approach would require a project to define an additional >> identifier. >> >> 2. References to locally built things and things in a repository share a new >> namespace. Every dependency declaration would use a single identifier, and >> there would be a separate mapping, shared by all the projects, from >> identifier to (group, module, version) or project. >> >> The idea here is that we also deal with the 'dependencyManagement' use case >> as well, so that you define (group, module, version) in one place and use a >> simple name everywhere that you need to refer to the dependency. This is, in >> practise what larger builds do already, so it's not necessarily more >> 'complicated'. >> >> allprojects { >> modules { >> someRepoThing 'my.org:some-repo-thing:1.2' >> } >> } >> >> dependencies { >> compile someRepoThing >> runtime someLocalThing // Probably some default rule to look for a >> locally built component called 'someLocalThing' >> } >> >> The obvious downside here is that you need to extra work if the dependency >> is not reused. Plus it means every dependency declaration in every Gradle >> build script in existence would need to change. >> >> 3. A combination of the above, so that all dependency declarations refer to >> things using (group, component, version), but group and version are >> optional. If not specified, we look for a mapping from component name to >> (group, component, version) or (project, component). We'd have some place >> where you can register these mapping to things in repositories, and probably >> also have a default mapping where we look for a locally built component with >> the given name and same group as the referring project. >> >> allprojects { >> dependencies.modules { >> junit "junit:junit:4.11" >> } >> } >> >> dependencies { >> compile "com.google.guava:guava:13" >> testCompile "junit" // or perhaps ":junit:" - maps to "junit:junit:4.11" >> testCompile "junit:4.8.2" // maps to "junit:junit:4.8.2" >> runtime "some-local-thing" // default rule maps to component >> 'some-local-thing' built by another project >> } >> >> For me, this one is almost all upsides. It gives me a unified way to refer >> to things, using (if I like) a simple name, and a place to define meta-data >> about external things in a single place (such as, every dependency to >> 'junit' should use 'junit:junit:4.11' unless otherwise specified, or >> "my-lib" uses semantic versioning, or "groovy-all" conflicts with "groovy", >> etc). It also gives me a point where I can put logic in to influence whether >> a given dependency declaration is mapped to a locally built thing or a thing >> in a repository. >> >> One major downside is that every project dependency in the world would need >> to be changed, because project dependencies would go away (via a long period >> of deprecation). >> >> 4. Keep things as they are, but remove (via deprecation) the ability to >> navigate to the target project via a ProjectDependency. We would offer a way >> to ask Gradle to substitute a project dependency with an external dependency >> and vice versa. >> >> Major downside for me is that it doesn't push the move to components far >> enough. >> >> One not-so-obvious advantage of this approach is that it is very easy to >> implement configure-projects-on-demand, whereas the other approaches would >> require some kind of caching. >> >> >> Thoughts? Other options? >> >> >> -- >> Adam Murdoch >> Gradle Co-founder >> http://www.gradle.org >> VP of Engineering, Gradleware Inc. - Gradle Training, Support, Consulting >> http://www.gradleware.com >> > > > -- > Adam Murdoch > Gradle Co-founder > http://www.gradle.org > VP of Engineering, Gradleware Inc. - Gradle Training, Support, Consulting > http://www.gradleware.com >
