On 2012-09-10 14:13, Alan Bateman wrote:
I think this is a great topic to discuss.
At least within Oracle then I think the majority of people do partial
builds in their local environment. When I say "partial build" then I
mean they build a subset of repositories, not all of them. So folks
working in the jdk repository tend to just build that repository with
everything else coming from whatever import JDK they are using.
Partial builds are of course very fragile and frequently people have
problems because they aren't using a matching import JDK. Also
periodically we have flag days, say changes that impact langtools+jdk
or hotspot+jdk and these usually cause a bit of disruption until the
changes get into a promoted build and into the import JDK that people
use. As to why people do partial builds then it's probably a mix of
issues including disk space, time to do the first build, and of course
it's just the way that we've always done it.
In build-infra, there is currently a "somewhat partial build" feature
that is implemented like this:
1) You check out a "master forest", containing all repos. You only need
to do this checkout once, and you are not required to pull/update it
(unless you want to keep up with changes -- like on a flag day).
2) For subsequent work, you can clone just a single repo (let's call it
"my_hotspot"), and doing some configure magic (currently a bit
complicated, but that can be rewritten to be simpler) which will point
to the "master forest" but replace the hotspot repo in the master forest
with "my_hotspot"..
3) The first time you build "my_hotspot", it will rebuild the whole
forest (but with hotspot replaced with my_hotspot). However, if you
never pull/update the master forest, this will be a one-time event.
Subsequent builds will see that nothing has changed in the master
forest, and only changes in my_hotspot will be recompiled.
The pros with this solution is:
1) help with flag days -- just pull/update the master forest and rebuild.
2) you can always build the complete product.
3) you save some disk space by only cloning the complete forest once.
However:
1) you will still need disk space for build output for the complete
forest, for every single repo.
2) you will still need to recompile the whole forest for each new single
repo you clone.
Do you think this will make the solution not good enough?
I think it is possible and not too hard to change this solution
slightly, so you would only need to compile the master forest once, and
then new single repos can use that build output as well as using the
source code from the master forest. Though proper dependency checking
might be harder, and there will be semantic issues about the meaning of
things like "make clean" that can be surprising. Is this something you
think is worth pursuing?
To completely eliminate compiling the whole product at least once seems
much harder. I can't really see a good way of using "import JDKs" the
way they've been used in the old build system. I imagine one possibility
is to prepare a tar ball, including a complete cloned forest (for a
certain point in time) and a build directory with build artifacts
corresponding to that source code and your platform(s) of choice, and
have people download that as starting point. It seems like a lot of
work, though. Do you think that is needed to get the new build system
accepted?
/Magnus
