On Dec 22, 2007 5:17 PM, Jason van Zyl <[EMAIL PROTECTED]> wrote: > > On 22 Dec 07, at 1:02 AM 22 Dec 07, Milos Kleint wrote: > > > i've played with the DefaultLifecycleExecutor and > > DefaultPluginManager a bit and tried to separate the preparation > > phase from the actual mojo executions. The preparations seem to > > attribute to astonishing 90% of the build execution. > > See http://picasaweb.google.com/mkleint/Maven/ > > photo#5146520463951211778 > > > > what have I actually done? (patch attached, I hope it gets through) > > > > in the preparation phase > > 1. iterate all the projects, figure out the mojo bindings. > > 2. for each binding, load the asociated plugin > > 3. for each project figure out what the required maximum project > > dependency resolution is and resolve it. > > > > in the execution phase actually just run the mojo's execute method. > > (ripped the project dependency resolution from there) > > > > The work i've done is probably not generally useful, but it clearly > > shows where space for improvements is. > > Why is it not generally useful? I think as part of improving the build > plan and exposing what can be configured and change they the user is > good. How the mojo bindings are put together would definitely affect > the subsequent execution. John should take a look at the patch as > well, but I will take a look but. Did you manage to reduce any > complexity or reduce any code or did you just add different processing?
The build doesn't fail fast now in some cases. It will prepare all the 20 project's lifecycles, perform dependency resolution of all the 20 projects and then the first one compiler plugin invocation fails in compilation. the only complexity reduction was trimming down 80+ invocations of project dependency resolution to 20 (thus once per project) > > > > > > However I'd like to use this in the Netbeans IDE, to "preload" > > various actions as the user edits the files, so that the build/run/ > > debug cycle gets faster (as I'm dependent on running maven goals on > > these actions) and when the user actually triggers "Run" only the > > execution phase needs to be run. > > > > Yes, definitely useful to IDE integration. Visualization of the build > plan will be one of the most useful features there will be. agree about visualization. That's something very useful and the patch is relevant to it. However my usecase was different, I want to run the preparation phase behind the scenes (triggered by a user edit of a java file for example). With some added heuristics, later when the user invokes "Run project" or "Debug project" in the main menu, I can only grab this prepared build and invoke it very fast.. That's a consequence of the fact that everything project related in netbeans is done through maven. Milos > > > Milos > > > > > > On Dec 19, 2007 10:34 PM, Jason van Zyl <[EMAIL PROTECTED]> wrote: > > > > On 19 Dec 07, at 12:28 PM 19 Dec 07, Milos Kleint wrote: > > > > > here is the actual screenshot. > > > > > > http://picasaweb.google.com/mkleint/Maven > > > > > > more inline.. > > > > > > On Dec 19, 2007 9:19 PM, Jason van Zyl <[EMAIL PROTECTED]> wrote: > > > > > >> > > >> On 19 Dec 07, at 11:10 AM 19 Dec 07, Milos Kleint wrote: > > >> > > >>> Hello, > > >>> > > >>> i've started building my project with current 2.1-SNAPSHOT trunk > > and > > >>> I noticed the build is taking more time than before. I did a bit > > of > > >>> profiling. > > >>> I've executed "mvn install" on a set of 24 projects, all build > > >>> before, right before the profiling test. > > >>> > > >>> What struck me is that half of the time seems to be spent in > > >>> DefaultPluginManager.resolveTransitiveDependencies(). See attached > > >>> picture from netbeans profiler. > > >>> It was executed 82 times. I suppose the root of the problem > > will be > > >>> somewhere much deeper and I guess there's a way to optimize the > > >>> number of calls to the method as well. > > >>> After a bit of debugging i figured that for a single project this > > >>> method is called multiple times during execution. First the > > >>> "compile" scope is resolved, later "runtime" or "test" eventually > > >>> (depends on actual plugins bound to the project) > > >>> Ideally it should be called 24 times max as far as I understand > > the > > >>> problem. The BuildPlan should know up front what mojos will be > > >>> executed and what is the maximum level or dependency resolution > > for > > >>> the given project. > > >>> > > >>> Agreed? Is that something that I should pursue further or am I on > > >>> the wrong track? > > >>> > > >> > > >> Sure, take a look but in the plugin manager I started ripping out > > all > > >> the optimizations after ripping out a bunch of other code. But > > narrow > > >> it down for one build where > > >> > > >> 1) You don't have anything downloaded i.e. a clean repository, and > > >> 2) Where you have all the dependencies downloaded > > >> > > >> For one pass it shouldn't be resolving more then once for each > > >> plugin. > > >> I wouldn't try to start caching anything but if you're seeing more > > >> then one call per plugin then something needs to be reworked. > > > > > > > > > well, from what I understand I see 1 resolution per plugin per > > > project, if > > > the plugin requires dependency resolution. > > > > Yes, this is a reactor this is to account for cases where you may have > > the same plugin used, but might have dependencies stated itself which > > can change things so you need to track this. The non-optimized version > > obviously deals with this but you will end up with a case where > > different plugin declarations have slightly different dependency > > requirements. John tried to account for everything, but we could still > > cache most of the information. > > > > > > > > my ''optimization'' involves looking up front what is the maximum > > > dependency > > > resolution scope and resolve it just once per project. In your case > > > 1 it > > > means more than necessary gets downloaded upfront probably, but > > case 2 > > > should be faster. > > > > So we can sync up here as I'm remaking the project builder and trying > > to setup a listener that can watch for things as projects are being > > built. One of the things I'm trying to collect are profiles and > > extensions so that we don't have to scan the POMs again after they are > > built. We could do the same with plugin declarations so that up-front > > you would know what you needed. So you could see that for all the > > plugin configurations there were no variations, pick off which ones > > need resolution, do that and then fire the build. > > > > So if you could determine what information you needed up front I can > > factor that in, and in the short term you can take the approach John > > has made with things like the extension scanner. But ultimately we > > should not need to read any pom.xml file more then once. Whether it > > come in the maven-artifact, the project builder or anything else. > > > > > > > > > > > Milos > > > > > > > > >> > > >> > > >>> Milos > > >>> > > >>> PS: If anyone is interested I can send the actual netbeans > > profiler > > >>> dump files for browsing (via private email). > > >>> > > >>> > > >>> > > >>> > > --------------------------------------------------------------------- > > >>> To unsubscribe, e-mail: [EMAIL PROTECTED] > > >>> For additional commands, e-mail: [EMAIL PROTECTED] > > >> > > >> Thanks, > > >> > > >> Jason > > >> > > >> ---------------------------------------------------------- > > >> Jason van Zyl > > >> Founder, Apache Maven > > >> jason at sonatype dot com > > >> ---------------------------------------------------------- > > >> > > >> believe nothing, no matter where you read it, > > >> or who has said it, > > >> not even if i have said it, > > >> unless it agrees with your own reason > > >> and your own common sense. > > >> > > >> -- Buddha > > >> > > >> > > >> > > >> > > >> > > --------------------------------------------------------------------- > > >> To unsubscribe, e-mail: [EMAIL PROTECTED] > > >> For additional commands, e-mail: [EMAIL PROTECTED] > > >> > > >> > > > > Thanks, > > > > Jason > > > > ---------------------------------------------------------- > > Jason van Zyl > > Founder, Apache Maven > > jason at sonatype dot com > > ---------------------------------------------------------- > > > > What matters is not ideas, but the people who have them. Good people > > can fix bad ideas, but good ideas can't save bad people. > > > > -- Paul Graham > > > > > > > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: [EMAIL PROTECTED] > > For additional commands, e-mail: [EMAIL PROTECTED] > > > > > > < > > lifecycleexecutor > > .patch > > >--------------------------------------------------------------------- > > To unsubscribe, e-mail: [EMAIL PROTECTED] > > For additional commands, e-mail: [EMAIL PROTECTED] > > Thanks, > > Jason > > ---------------------------------------------------------- > Jason van Zyl > Founder, Apache Maven > jason at sonatype dot com > ---------------------------------------------------------- > > We know what we are, but know not what we may be. > > -- Shakespeare > > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > >
