On 30/07/2013, at 6:52 PM, kelemen <attila.keleme...@gmail.com> wrote:
> 2013/7/30 Adam Murdoch [via Gradle] <[hidden email]>
>
> On 29/07/2013, at 4:25 PM, Alex Ruiz <[hidden email]> wrote:
>
>> Thanks, Adam. I've been looking into (and debugging) how the current API for
>> retrieving models work, to have a better understanding on how the API and
>> implementation of this new action would look like.
>>
>> Attila: This approach may also benefit your work on Netbeans. Please take a
>> look.
>>
>> This is what I have in mind, based on what I have done this weekend:
>>
>> - The action interface can be simple, and so far this is what I think about
>> its API:
>>
>> public interface ConsumerAction<T> extends Serializable {
>> T execute(Project project);
>> }
>>
>> where Project is a org.gradle.api.Project. My understanding is that by
>> having a Project we can do everything we need (through
>> Project#getService()).
>
> The action should accept something other than `org.gradle.api.Project`, for a
> few reasons:
>
> 1. `org.gradle.api.Project` is not version-independent. The entry point for
> the action should be a function of the client's tooling API version, not
> target Gradle version.
> 2. Some actions don't need a configured project. For example, an action may
> just be interested in the project hierarchy or the build environment. Or it
> may not be interested in the tasks of a project, or the dependencies of a
> project, or whatever. Instead, we need an interface that implies no
> configuration is done, except for that which is explicitly requested by the
> action.
> 3. It's not going to work well with decoupled project mode (e.g. parallel
> execution, configure on demand). This will become the default at some point.
> The plan there is that there will be no way to navigate from one project to
> another. Instead, we need an interface which provides access to the whole
> build, but in a way that allows things like parallel configuration and
> configuration on demand.
> 4. There's no opportunity to cache models. We'd have to cache the whole
> project model, which is unlikely to be possible any time soon.
>
>
> It would be very convenient to have access to `org.gradle.api.Project`
> regardless. Not being version independent is not necessarily a problem, since
> Gradle has to maintain compatibility due to the build scripts and this
> compatibility should be enough. Actually, I could provide a lot better
> experience in NB if this was possible. For example, I could easily display
> all the source root with proper classpaths including the usual "integTest"
> sources.
The process of importing Gradle into an IDE has to involve some kind of
mapping. The Gradle model is much richer that the IDEs and this model has to be
converted into the simpler model of each IDE. This mapping is lossy and as such
the mapping needs to be flexible, and the build author and the various plugins
must be able to have some influence over it. The mapping shouldn't be hardcoded
in the IDE. This is one of the reasons the tooling API exists.
So, we have:
Gradle model -> transformation -> IDE model
It's a little more detailed than that:
Gradle DSL -> transformation -> canonical model -> transformation -> IDE model
That is, the user configures some stuff through the DSL, this is then converted
to some internal model that various parts of Gradle use, including the IDE
mapping. We keep the DSL backwards compatible, but we don't for the canonical
model. We plan to, but we're not there yet. In other words, the stuff you need
from the project (i.e. the canonical model) is not backwards compatible at all.
So, rather than access `Project` directly, I'd much prefer if you help us grow
the tooling API models instead. It's easy to add stuff once you know the
pattern.
These models then form a contract, as they declare what you need from the
model. We can keep this stable and make it faster over time. They also allow us
to offer consistent behaviour across IDEs (which is important to us, possibly
not so for you).
>
> Caching can be done external to Gradle. Actually, I already cache models.
> Also, there is no way to know in general if reevaluation of a project is
> necessary, so I believe that Gradle cannot do considerably better than an
> external process.
There's an awful lot to track when checking for changes in the model. Here are
some examples:
- Changes in the build script and in one of the ancestor's build scripts.
- Changes in the settings scripts.
- Changes in the init scripts passed as command-line args, or in
~/.gradle/init.gradle or ~/.gradle/init.d
- Changes in any local or remote scripts applied by any of the above or any
init scripts bundled in the distributions.
- Changes in gradle.properties and ~/.gradle/gradle.properties.
- Changes to the Gradle version in gradle-wrapper.properties.
- Changes in the source or dependencies of buildSrc
- Changes in the classpath used at configuration time, eg local jars.
- Changes to dependencies in the remote repositories.
- Changes in the configuration files or remote resources that my custom plugin
uses to configure the build.
You get the idea. It think good change detection has to be a collaboration
between Gradle and the IDE.
>
> Anyway, I find the "invocation" API your are proposing a good general concept.
>
> So, instead, I'd introduce a new 'invocation' API. The contract would be that
> we start with an empty model and execute the action. The action can then use
> the invocation API to access the things it needs and these would be populated
> on demand.
>
> There's probably 2 basic things the API would need to do:
>
> 1. Give me the set of all the projects in the build. This would return
> something like `GradleProject` minus the tasks. It implies that the settings
> script has been executed.
> 2. For a given project, give me the model of type M.
>
> Possibly the API might also allow tasks to be scheduled and executed as well,
> but that can happen later.
>
>>
>> - The tooling API can have this method in ProjectConnection:
>>
>> <T> ConsumerActionExecutor<T> execute(ConsumerAction<T> action);
>>
>> where ConsumerActionExecutor is:
>>
>> public interface ConsumerActionExecutor<T> extends LongRunningOperation {
>> T get();
>> }
>>
>> This API sort of follows the pattern of ProjectConnection#model(Class).
>
> This is good.
>
>>
>> From here, I'm looking on how to implement what goes in between
>> ProjectConnection#execute(ConsumerAction) and ConsumerActionExecutor#get().
>
> There are some pretty deep changes here. I have some time to help out at the
> moment if you'd like me to pick up some (or most) of this.
>
> Here a potential implementation plan:
>
> 1. Add the APIs and a dummy implementation that runs the action in the
> tooling API client and returns the result.
> 2. Pass the action across the cross-version layer into the provider. Move the
> dummy implementation into the provider.
> 3. Serialize the action across to the daemon and the result back. Move the
> dummy implementation into the daemon. Now we've got a round trip into the
> real process.
> 4. Add an initial implementation of the invocation API, which simply
> configures all projects before doing anything.
> 5. Handle the case where the result references one of the models returned by
> the invocation API.
>
>
> Why is 5 a special case? Aren't they serializable already?
>
> At this point, you've got a solution to your problem, as you can now do a
> single pass over the model and collect up the bits you need and send them
> back to the client. Later steps can make this better:
>
> 1. Don't configure a project until one of its models is requested.
> 2. Don't configure the project hierarchy until it is requested.
> 3. Don't configure the tasks of a project until they are requested
>
> And generally making the invocation API richer.
>
>
>
> I have created an initial draft for the required interfaces:
> https://gist.github.com/kelemen/6111321. See if you like it and what needs to
> modified, what is missing, or if it is completely wrong. Also, before
> actually executing this `ConsumerAction`, I need to know the version of
> Gradle if I can attempt to execute a `ConsumerAction` and fallback to what I
> do currently for versions older than 1.8. For which, it would be nice if
> there was a method to compare version numbers of Gradle.
>
> View this message in context: Re: Proposal for retrieving multiple types of
> models from a project in a single pass, using the Tooling API
> Sent from the gradle-dev mailing list archive at Nabble.com.
--
Adam Murdoch
Gradle Co-founder
http://www.gradle.org
VP of Engineering, Gradleware Inc. - Gradle Training, Support, Consulting
http://www.gradleware.com
