I think you're right about the 95% overlap. If we download the whole plugin with all their deps to calculate the execution plan we could surely construct the class realm too; that's just a few pojos per plugin. I'll investigate.
Still this whole discussion is probably tangential to the problem I set out to resolve, which seems to involve a race condition around the cache key. Kristian 2012/7/27 Jason van Zyl <[email protected]>: > I wouldn't speculate without measuring but from anecdotal experience I would > venture to say that of those 200 modules there's 95% overlap in the plugins > executed. I'm not sure it would be that inefficient given what we do today > which is to download the plugin itself to get at its metadata to build the > execution plan. But I also can't see why there would be any contention in a > short-lived process. I'm also not terribly familiar with the parallel > execution code (I looked at once when you first did it and it just worked at > that point so I didn't look much again) so I'll take a peek this weekend. > > On Jul 26, 2012, at 2:10 PM, Kristian Rosenvold wrote: > >> I'm not sure this makes sense. Given a reactor with 200 modules, it >> would mean you would be downloading all the plugins and constructing >> all the realms before even starting on the first module ? >> >> Re-using realms across modules is great, but pre-constructing >> everything would give an uneccessary performance hit ? >> >> Kristian >> >> 2012/7/26 Jason van Zyl <[email protected]>: >>> Could we not adjust such that after the execution plan is created, prepare >>> all the class realms required by the plan and once constructed left >>> immutable during the actual execution of the plan. I can't quite remember >>> but I'm not sure why we need multiple threads constructing realms with the >>> execution plan in hand. Prepare the realms according to the plan, make them >>> immutable and use them throughout the session. >>> >>> On Jul 26, 2012, at 8:44 AM, Kristian Rosenvold wrote: >>> >>>> There is a race condition in parallel builds that occurs related to >>>> this piece of code: >>>> >>>> http://maven.apache.org/ref/3.0.4/maven-core/xref/org/apache/maven/classrealm/DefaultClassRealmManager.html#75 >>>> >>>> The thing is, for some reason, there's a loop that retries the class >>>> realm generation with a random suffix if the class realm already >>>> exists. In a parallel run, there will be multiple threads requesting >>>> the same realm-id, which semantically should map to the same instance >>>> of the class realm. >>>> >>>> (Most plugins do not really mind if there's a duplicate class realm >>>> every now and then, but some take it very seriously ;) >>>> >>>> I'm tempted to change the semantics of the "newRealm" method to >>>> "getOrCreateRealm", since that seems to be the correct semantics no >>>> matter what. I've tried tracking the origin of the while loop, and it >>>> seems to be very old. Anyone have any idea of what purpose it served ? >>>> >>>> Kristian >>>> >>>> --------------------------------------------------------------------- >>>> To unsubscribe, e-mail: [email protected] >>>> For additional commands, e-mail: [email protected] >>>> >>> >>> Thanks, >>> >>> Jason >>> >>> ---------------------------------------------------------- >>> Jason van Zyl >>> Founder & CTO, Sonatype >>> Founder, Apache Maven >>> http://twitter.com/jvanzyl >>> --------------------------------------------------------- >>> >>> Simplex sigillum veri. (Simplicity is the seal of truth.) >>> >>> >>> >>> >>> >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: [email protected] >> For additional commands, e-mail: [email protected] >> > > Thanks, > > Jason > > ---------------------------------------------------------- > Jason van Zyl > Founder & CTO, Sonatype > Founder, Apache Maven > http://twitter.com/jvanzyl > --------------------------------------------------------- > > believe nothing, no matter where you read it, > or who has said it, > not even if i have said it, > unless it agrees with your own reason > and your own common sense. > > -- Buddha > > > > > --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
