Re: The future of partial builds

2012-09-14 Thread Kelly O'Hair
The best times I have gotten have been with the VMs in Stockholm, VMware base, 4 vcpus, lots of RAM. I don't have any newer raw hardware, just VMs. However, as I stated, it's erratic. We have some issues somewhere. I think Erik has gotten even faster times with his beefy desktop, not sure what

Re: The future of partial builds

2012-09-13 Thread Alan Bateman
On 12/09/2012 19:49, Kelly O'Hair wrote: Some stats on incremental builds. Not partial builds.. This is an older Solaris machine svc6.us.oracle.com, building the complete openjdk forest from scratch for 64bit including images took less than 14 minutes (parallel build setting was 8) and images

Re: The future of partial builds

2012-09-13 Thread Anthony Petrov
The build times for incremental builds look awesome. Now I'm really looking forward to the new build system! Thanks! Regarding partial builds: as long as the other components don't add dependencies that we didn't use to have before (when working on the jdk repo only), I guess it's all right,

Re: The future of partial builds

2012-09-12 Thread Anthony Petrov
On 9/12/2012 1:54 PM, Alan Bateman wrote: On 12/09/2012 06:46, Fredrik Öhrström wrote: : Excellent. I hope you realize how valuable it is that the build system recompiled the proper source files, then proceed to generate the the jni headers output because of native methods in those classes

Re: The future of partial builds

2012-09-12 Thread Jonathan Gibbons
On 09/11/2012 10:46 PM, Fredrik Öhrström wrote: However, this begets the question why -Xprefer:source was added in the first place. If I remember correctly, this was over a year ago. It turns out that the default behavior for implicit compilation (which is necessary for sjavac:s multi core

Re: The future of partial builds

2012-09-12 Thread Kelly O'Hair
Some stats on incremental builds. Not partial builds.. This is an older Solaris machine svc6.us.oracle.com, building the complete openjdk forest from scratch for 64bit including images took less than 14 minutes (parallel build setting was 8) and images took 2mins of this 14mins: -- Build

Re: The future of partial builds

2012-09-12 Thread Kelly O'Hair
On Sep 12, 2012, at 8:54 AM, Jonathan Gibbons wrote: On 09/11/2012 10:46 PM, Fredrik Öhrström wrote: However, this begets the question why -Xprefer:source was added in the first place. If I remember correctly, this was over a year ago. It turns out that the default behavior for implicit

Re: The future of partial builds

2012-09-12 Thread Igor Nekrestyanov
What about windows? -igor On 9/12/12 11:49 AM, Kelly O'Hair wrote: Some stats on incremental builds. Not partial builds.. This is an older Solaris machine svc6.us.oracle.com, building the complete openjdk forest from scratch for 64bit including images took less than 14 minutes (parallel

Re: The future of partial builds

2012-09-12 Thread Kelly O'Hair
Windows X64 CYGWIN Openjdk builds run between 16 and 25mins, depending on the machine used (hardware vs. VM), how many CPUs etc. But it's a bit erratic, depends on lots of things. We are still working around some CYGWIN issues. The incremental builds should be similar. I don't have any numbers

Re: The future of partial builds

2012-09-12 Thread Fredrik Öhrström
Oookay, long story, this was when we used jdk6 to build jdk7, same kind of problem could occur when using jdk7 to build jdk8: jdk7javac used new features and classes in jdk7, for example: java/util/Objects.java java/util/ConcurrentModificationException.java java/io/File.java java/nio/file/*

Re: The future of partial builds

2012-09-12 Thread Weijun Wang
Windows X64 in VM can build within 25mins? That's quite impressive. Can you share what VM/OS/CPU/mem you are using? I only have VirtualBox running on Linux X64 now, but will consider moving to some other platforms if it helps a lot. Thanks Max On 09/13/2012 04:32 AM, Kelly O'Hair wrote:

Re: The future of partial builds

2012-09-11 Thread Anthony Petrov
Magnus, You've only explained how incremental builds could work for Java classes in the new build-infra. What about incremental builds of native code? E.g. in AWT we often do the following: $ cd make/sun/awt (or make/java/awt, or make/sun/lwawt) $ make And this re-builds both AWT classes

Re: The future of partial builds

2012-09-11 Thread Chris Hegarty
On 11/09/12 14:37, Anthony Petrov wrote: Magnus, You've only explained how incremental builds could work for Java classes in the new build-infra. What about incremental builds of native code? E.g. in AWT we often do the following: $ cd make/sun/awt (or make/java/awt, or make/sun/lwawt) $

Re: The future of partial builds

2012-09-11 Thread Fredrik Öhrström
Den tisdagen den 11:e september 2012 skrev Alan Bateman: So far my experience is that touching native code and re-building is super fast, it's on par to executing specific make files in the old build (while wearing the appropriate amulet around one's neck of course). Touching java classes and

Re: The future of partial builds

2012-09-11 Thread Alan Bateman
On 11/09/2012 19:23, Fredrik Öhrström wrote: Den tisdagen den 11:e september 2012 skrev Alan Bateman: So far my experience is that touching native code and re-building is super fast, it's on par to executing specific make files in the old build (while wearing the appropriate amulet

Re: The future of partial builds

2012-09-11 Thread Fredrik Öhrström
11 sep 2012 kl. 21:59 skrev Alan Bateman: With a sjavac config I touched one source file and the incremental build took 36s so much better. In this case the one class caused 77 classes to be re-compiled, 10 native files, and 2 shared libraries to be re-linked. On the other hand, the old

The future of partial builds

2012-09-10 Thread Magnus Ihse Bursie
I'd like to start a discussion about the partial builds; what problem they solve and the best way to solve these problems in the new build-infra world. I'm currently investigating on how to handle the equivalence to partial builds in the new build system. My goal is to see to it that the new

Re: The future of partial builds

2012-09-10 Thread Chris Hegarty
On 10/09/2012 12:26, Magnus Ihse Bursie wrote: I'd like to start a discussion about the partial builds; what problem they solve and the best way to solve these problems in the new build-infra world. I'm currently investigating on how to handle the equivalence to partial builds in the new build

Re: The future of partial builds

2012-09-10 Thread Alan Bateman
I think this is a great topic to discuss. At least within Oracle then I think the majority of people do partial builds in their local environment. When I say partial build then I mean they build a subset of repositories, not all of them. So folks working in the jdk repository tend to just

Re: The future of partial builds

2012-09-10 Thread Magnus Ihse Bursie
On 2012-09-10 14:13, Alan Bateman wrote: When you say sub-directory builds then I think you mean incremental builds, or poor-man increment builds as I call it. I think the majority of people working in the jdk repository, at least in Oracle, do this because they know the area and know which

Re: The future of partial builds

2012-09-10 Thread Magnus Ihse Bursie
On 2012-09-10 14:13, Alan Bateman wrote: I think this is a great topic to discuss. At least within Oracle then I think the majority of people do partial builds in their local environment. When I say partial build then I mean they build a subset of repositories, not all of them. So folks

Re: The future of partial builds

2012-09-10 Thread Weijun Wang
It's not uncommon that I just clone the repo(s), do a quick build, run several tests and abandon it, for example, if I want to try my code changes on a different platform (from my daily work machine). Therefore I find the jdk-only build very useful. -Max On 09/10/2012 10:20 PM, Magnus Ihse

Re: The future of partial builds

2012-09-10 Thread Weijun Wang
Sorry I was not clear, by clone the repo(s), I mean either only the jdk repo, or plus the jdk/*/closed ones. I almost never clone other repos (langtools, hotspot, ...). -Max On 09/10/2012 10:48 PM, Weijun Wang wrote: It's not uncommon that I just clone the repo(s), do a quick build, run

Re: The future of partial builds

2012-09-10 Thread Jonathan Gibbons
Magnus, I'm guessing that most developers use partial builds. In the langtools/ world, we only build the langtools component, and even then, sometimes only javac. We can then use the result to run our tests. We do not want to -- and would strongly resist having to -- rebuild all of JDK

Re: The future of partial builds

2012-09-10 Thread Jonathan Gibbons
Having to compile hotspot every time one creates a new repo seems like a very significant step backwards. I can clone and build langtools in 45 seconds. $ time ( hg clone http://hg.openjdk.java.net/jdk8/tl/langtools ; cd langtools ; lt.ant build-all-classes ) destination directory:

Re: The future of partial builds

2012-09-10 Thread Kumar Srinivasan
I focus on pack200, java launcher and javac, and for the launcher I am sometimes forced to use windows as the development platform. Building incrementally the launcher, saves me a lot of time. This is how it works: 1. make launcher specific files and test with ALT_OUTPUTDIR. 2. build all the

Re: The future of partial builds

2012-09-10 Thread Phil Race
A huge step backwards. I don't want to have to clone or keep hotspot up to date and I prefer using the 'RE' builds of hotspot to any I would create. I have never found this fragile. Its worked well for over 10 years ... -phil. On 9/10/2012 8:36 AM, Jonathan Gibbons wrote: Having to compile

Re: The future of partial builds

2012-09-10 Thread Igor Nekrestyanov
Can not agree more. For repos further in the food chain (such as deploy, etc) this is even worse. Often we do not really need to build jdk repo. We doing (quick) partial builds in Hudson to run tests continuously. We also often need to build on test systems where JDK build env is not set.

Re: The future of partial builds

2012-09-10 Thread David Katleman
Is this a somewhat correct understanding? Am I missing something? Are there some other reason apart from speeding up the build to do partial builds? How many users out there are actually using partial builds? I believe, as you've seen in the replies, the question is better rephrased as How