Hi Stephen,
I’ve narrowed down the issue to the Plexus facade - specifically the scheduler
code which activates component cycles/groups. The problem it tries to solve is
when you have a component cycle like A-B-C-D-A where one or more of these
components (for example C) also has a lifecycle phase like Initializable. You
can’t call Initializable immediately after building C and injecting its
requirements because one of those requirements might be a temporary proxy (used
to break the dependency cycle) - and the real component the proxy will
eventually point to might not have been created yet. Instead you have to wait
for the group as a whole to finish construction before calling lifecycle phases.
Your sample project seems to be triggering this activation code (I’ve yet to
determine if there really is a cycle which just happens to involve
DefaultDependencyGraphBuilder as a tangential requirement, or if this was a
false-positive). The facade activation code is then holding back
Contextualizable until the component group is ready. Normally this wouldn’t be
an issue but in this case you also have a mix of scopes in the group. The end
result is that while Guice considers the component to be ready, the Plexus
facade isn’t done with it yet.
I’m testing a couple of solutions and I’ll let you know when there’s a patched
jar you can try.
--
Cheers, Stuart
On Tuesday, 2 February 2016 at 16:34, Stuart McCulloch wrote:
> PS. both Guice and the facade have multi-threaded concurrency tests, so once
> I confirm this is what’s going on I’ll also look into why this wasn’t picked
> up by the existing tests
>
> On Tuesday, 2 February 2016 at 16:13, Stuart McCulloch wrote:
>
> > Hi Stephen,
> >
> > Thanks for the update.
> >
> > The component shouldn’t be escaping to consumers before contextualize has
> > finished, that sounds like a bug. Note that while it still looks like
> > Plexus from an API perspective, ever since Maven 3 it’s actually Guice plus
> > a thin facade to support the same API and behaviour as the old Plexus
> > container. For example the contextualize step (along with other Plexus
> > phases) is implemented by registering a custom ProvisionListener with
> > Guice, which gets called as part of injection and is meant to finish before
> > the object can be seen from the perspective of wherever it’s being injected.
> >
> > There have been some changes to Guice master in a related area, so I’ll try
> > the latest development snapshot just in case that helps - I’ll also confirm
> > what’s happening in the facade.
> >
> > --
> > Cheers, Stuart
> >
> > On Tuesday, 2 February 2016 at 15:33, Stephen Evanchik wrote:
> >
> > > Hi Stuart,
> > >
> > > The synchronized let me see what was happening as far as ordering was
> > > concerned. It appears that the contextualize() call races with calls to
> > > buildDependencyGraph() and that that Container is not performing lifecycle
> > > operations on the DefaultDependencyGraphBuilder object before consumers of
> > > that object invoke methods.
> > >
> > > I think the maven-bundle-plugin needs to have @threadSafe = false since
> > > it's not thread safe due its dependencies not being thread safe. I don't
> > > know enough about Maven and Plexus to force some sort of explicit ordering
> > > in the bean's lifecycle.
> > >
> > > However, this patch does seem to fix the underlying issue though it could
> > > be subject to a dead lock if all of the build threads wait on the latch.
> > > The real solution is for the container to properly follow concurrent
> > > lifecycle rules.
> > >
> > > ===================================================================
> > > ---
> > > src/main/java/org/apache/maven/shared/dependency/graph/internal/DefaultDependencyGraphBuilder.java
> > > (revision 1726911)
> > > +++
> > > src/main/java/org/apache/maven/shared/dependency/graph/internal/DefaultDependencyGraphBuilder.java
> > > (working copy)
> > > @@ -1,5 +1,9 @@
> > > package org.apache.maven.shared.dependency.graph.internal;
> > >
> > > +import java.util.Collection;
> > > +import java.util.concurrent.CountDownLatch;
> > > +import java.util.concurrent.TimeUnit;
> > > +
> > > /*
> > > * Licensed to the Apache Software Foundation (ASF) under one
> > > * or more contributor license agreements. See the NOTICE file
> > > @@ -33,9 +37,6 @@
> > > import org.codehaus.plexus.logging.AbstractLogEnabled;
> > > import
> > > org.codehaus.plexus.personality.plexus.lifecycle.phase.Contextualizable;
> > >
> > > -import java.util.Collection;
> > > -import java.util.Collections;
> > > -
> > > /**
> > > * Default dependency graph builder that detects current Maven version to
> > > delegate to either Maven 2 or Maven 3 specific
> > > * code.
> > > @@ -52,6 +53,8 @@
> > > {
> > > protected PlexusContainer container;
> > >
> > > + private final CountDownLatch l = new CountDownLatch(1);
> > > +
> > > /**
> > > * Builds a dependency graph.
> > > *
> > > @@ -81,6 +84,8 @@
> > > {
> > > try
> > > {
> > > + l.await(30, TimeUnit.SECONDS);
> > > +
> > > String hint = isMaven31() ? "maven31" : isMaven2x() ? "maven2"
> > > : "maven3";
> > >
> > > DependencyGraphBuilder effectiveGraphBuilder =
> > > @@ -90,6 +95,9 @@
> > >
> > > return effectiveGraphBuilder.buildDependencyGraph( project,
> > > filter, reactorProjects );
> > > }
> > > + catch ( InterruptedException e ) {
> > > + throw new DependencyGraphBuilderException( e.getMessage(), e );
> > > + }
> > > catch ( ComponentLookupException e )
> > > {
> > > throw new DependencyGraphBuilderException( e.getMessage(), e );
> > > @@ -136,5 +144,7 @@
> > > throws ContextException
> > > {
> > > container = (PlexusContainer) context.get(
> > > PlexusConstants.PLEXUS_KEY );
> > > +
> > > + l.countDown();
> > > }
> > > }
> > >
> > >
> > >
> > > Stephen
> > >
> > >
> > > On Tue, Feb 2, 2016 at 10:14 AM, Stephen Evanchik <evanc...@gmail.com
> > > (mailto:evanc...@gmail.com)>
> > > wrote:
> > >
> > > > Hi Stuart,
> > > >
> > > > I think I found the problem. In maven-dependency-tree:{2.1,2.2} the
> > > > Contextualizable.contextualize() method is invoked concurrently with the
> > > > two buildDependencyGraph() methods. This is why the container field is
> > > > null. I simply synchronized the methods against each other.
> > > >
> > > > Stephen
> > > >
> > > > On Mon, Feb 1, 2016 at 10:35 AM, Stephen Evanchik <evanc...@gmail.com
> > > > (mailto:evanc...@gmail.com)>
> > > > wrote:
> > > >
> > > > > Hi Stuart,
> > > > >
> > > > > I was looking at some parallel builds we ran over the weekend and
> > > > > notice
> > > > > some failures. I started to track them down and noticed that there
> > > > > was a
> > > > > very strange error message which seems to indicate that
> > > > > maven-bundle-plugin
> > > > > is not thread safe or has a fairly serious bug:
> > > > >
> > > > > [ERROR] Failed to execute goal
> > > > > org.apache.felix:maven-bundle-plugin:3.0.1:manifest (bundle-manifest)
> > > > > on
> > > > > project PROJECT_A: Execution bundle-manifest of goal
> > > > > org.apache.felix:maven-bundle-plugin:3.0.1:manifest failed: A Jar can
> > > > > only
> > > > > accept a valid file or directory:
> > > > > /full/build/path/PROJECT_B/target/classes
> > > > >
> > > > > There is a compile dependency between the projects (PROJECT_A depends
> > > > > on
> > > > > PROJECT_B) but it looks like PROJECT_B was not yet built.
> > > > >
> > > > > Stephen
> > > > >
> > > > >
> > > > >
> > > > > On Thu, Jan 28, 2016 at 8:37 PM, Stephen Evanchik <evanc...@gmail.com
> > > > > (mailto:evanc...@gmail.com)>
> > > > > wrote:
> > > > >
> > > > > > Hi Stuart,
> > > > > >
> > > > > > A final update for tonight: I was able to reproduce the problem with
> > > > > > maven-bundle-plugin:3.0.0 but not with maven-bundle-plugin:2.5.4
> > > > > > even
> > > > > > though it looks like the DefaultDependencyGraphBuilder is in all
> > > > > > three
> > > > > > versions.
> > > > > >
> > > > > > Stephen
> > > > > >
> > > > > > On Thu, Jan 28, 2016 at 6:18 PM, Stephen Evanchik
> > > > > > <evanc...@gmail.com (mailto:evanc...@gmail.com)>
> > > > > > wrote:
> > > > > >
> > > > > > > Hi Stuart,
> > > > > > >
> > > > > > > Thanks for confirming the behavior. I too noticed the redundant
> > > > > > > packaging and execution stanzas. I suspect the original developer
> > > > > > > that
> > > > > > > added this project didn't understand how the bundle extension
> > > > > > > worked.
> > > > > > >
> > > > > > > When I noticed the redundancy I updated all of the projects to
> > > > > > > use the
> > > > > > > <execution>/<package> jar pattern hoping that the problem would
> > > > > > > go away.
> > > > > > > Unfortunately, I saw an improvement but still have build failures.
> > > > > > >
> > > > > > > I'm not sure if this is a bug in Maven, maven-dependency-tree or,
> > > > > > > maven-bundle-plugin
> > > > > > >
> > > > > > > Stephen
> > > > > > >
> > > > > > >
> > > > > > > On Thu, Jan 28, 2016 at 5:43 PM, Stuart McCulloch
> > > > > > > <mccu...@gmail.com (mailto:mccu...@gmail.com)>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Hi Stephen,
> > > > > > > >
> > > > > > > > I've been able to recreate this with Maven 3.3.9 (by cloning
> > > > > > > > your
> > > > > > > > example
> > > > > > > > project several times in a multi-module project) - will
> > > > > > > > investigate
> > > > > > > > further
> > > > > > > > over the weekend.
> > > > > > > >
> > > > > > > > I did notice you're using the 'bundle' packaging, but you also
> > > > > > > > have an
> > > > > > > > explicit execution of the 'bundle' goal which should already be
> > > > > > > > covered by
> > > > > > > > the 'bundle' packaging's lifecycle - is there a reason for this
> > > > > > > > extra
> > > > > > > > execution?
> > > > > > > > On 28 Jan 2016 18:00, "Stephen Evanchik" <evanc...@gmail.com
> > > > > > > > (mailto:evanc...@gmail.com)> wrote:
> > > > > > > >
> > > > > > > > > Hi Stuart,
> > > > > > > > >
> > > > > > > > > I am using Maven 3.3.3 please disregard my earlier email. I
> > > > > > > > > did see
> > > > > > > > this
> > > > > > > > > problem in 3.3.1 so there doesn't seem to be a difference. I
> > > > > > > > > am
> > > > > > > >
> > > > > > > > going to
> > > > > > > > > try with 3.3.9 but I'm not going to hold out hope that it
> > > > > > > > > will make a
> > > > > > > > > difference.
> > > > > > > > >
> > > > > > > > > I think the problem is that DefaultDependencyGraphBuilder
> > > > > > > > > does not
> > > > > > > > get
> > > > > > > > > initialized properly. This only occurs in parallel builds and
> > > > > > > > > is
> > > > > > > >
> > > > > > > > strongly
> > > > > > > > > associated with relative timing. For example, if I issue a
> > > > > > > > > mvn -T2.0C
> > > > > > > > > install I will see the failure >80% of the time but if I use
> > > > > > > > > mvn -X
> > > > > > > > >
> > > > > > > >
> > > > > > > > -T2.0C
> > > > > > > > > I cannot reproduce the failure at all.
> > > > > > > > >
> > > > > > > > > I'm not sure embedding a pom is the right approach but here's
> > > > > > > > > a
> > > > > > > > pastbin
> > > > > > > > > link to a sample project that fails:
> > > > > > > > >
> > > > > > > > > https://paste.apache.org/dpeQ
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > Stephen
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > On Wed, Jan 27, 2016 at 4:49 AM, Stuart McCulloch
> > > > > > > > > <mccu...@gmail.com (mailto:mccu...@gmail.com)
> > > > > > > > >
> > > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > On Wednesday, 27 January 2016 at 06:03, Stephen Evanchik
> > > > > > > > > > wrote:
> > > > > > > > > > > Hi everyone,
> > > > > > > > > > >
> > > > > > > > > > > I'm having trouble tracking down an intermittent but
> > > > > > > > > > > frequent
> > > > > > > > build
> > > > > > > > > > failure
> > > > > > > > > > > using the maven-bundle-plugin to wrap non-OSGi projects.
> > > > > > > > > > > I'm
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > > using
> > > > > > > > > Maven
> > > > > > > > > > > 3.3.1 and see the following NPE:
> > > > > > > > > > >
> > > > > > > > > > > Caused by: java.lang.NullPointerException
> > > > > > > > > > > at
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > > org.apache.maven.shared.dependency.graph.internal.DefaultDependencyGraphBuilder.buildDependencyGraph(DefaultDependencyGraphBuilder.java:60)
> > > > > > > > > > > at
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > > org.apache.felix.bundleplugin.BundlePlugin.buildDependencyGraph(BundlePlugin.java:334)
> > > > > > > > > > > at
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > > org.apache.felix.bundleplugin.BundlePlugin.execute(BundlePlugin.java:359)
> > > > > > > > > > > at
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > > org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:134)
> > > > > > > > > > > ... 11 more
> > > > > > > > > > >
> > > > > > > > > > > whenever I invoke a parallel build (-T2.0C for example).
> > > > > > > > > > >
> > > > > > > > > > > There are many projects that will fail with this
> > > > > > > > > > > exception. I can
> > > > > > > > > > provide a
> > > > > > > > > > > fairly simple one if that makes sense.
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > a reproducible test project is always helpful - have you
> > > > > > > > > > tried a
> > > > > > > > more
> > > > > > > > > > recent version of Maven like 3.3.3 or 3.3.9 to see if that
> > > > > > > > > > helps?
> > > > > > > > > > > It looks like the 3.0.1 version is using
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > > maven-dependency-tree-2.1
> > > > > > > > > whose
> > > > > > > > > > > buildDependency method looks like:
> > > > > > > > > > >
> > > > > > > > > > > public DependencyNode buildDependencyGraph( MavenProject
> > > > > > > > > > > project,
> > > > > > > > > > > ArtifactFilter filter )
> > > > > > > > > > > throws DependencyGraphBuilderException
> > > > > > > > > > > {
> > > > > > > > > > > try
> > > > > > > > > > > {
> > > > > > > > > > > String hint = isMaven31() ? "maven31" : isMaven2x() ?
> > > > > > > > > > > "maven2"
> > > > > > > > > > > : "maven3";
> > > > > > > > > > > getLogger().debug( "building " + hint + " dependency
> > > > > > > > > > > graph for
> > > > > > > > > > > " + project.getId() );
> > > > > > > > > > >
> > > > > > > > > > > DependencyGraphBuilder effectiveGraphBuilder =
> > > > > > > > > > > (DependencyGraphBuilder) container.lookup(
> > > > > > > > > > > DependencyGraphBuilder.class.getCanonicalName(), hint );
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > where the NPE is on:
> > > > > > > > > > >
> > > > > > > > > > > DependencyGraphBuilder effectiveGraphBuilder =
> > > > > > > > > > > (DependencyGraphBuilder) container.lookup(
> > > > > > > > > > > DependencyGraphBuilder.class.getCanonicalName(), hint );
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > I'm not sure why this could NPE as it seems like the
> > > > > > > > > > > Contextualizable.contextualize() is called successfully
> > > > > > > > > > > for other
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > projects
> > > > > > > > > > > in the build.
> > > > > > > > > > >
> > > > > > > > > > > Any ideas on how to track this down? I can't enable
> > > > > > > > > > > debugging
> > > > > > > > (mvn -X)
> > > > > > > > > > > because that affects the timing just enough to avoid the
> > > > > > > > > > > issue.
> > > > > > > > > > >
> > > > > > > > > > > Thanks,
> > > > > > > > > > > Stephen
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > --
> > > > > > > > > Stephen Evanchik
> > > > > > > > > http://stephen.evanchik.com
> > > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > > Stephen Evanchik
> > > > > > > http://stephen.evanchik.com
> > > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Stephen Evanchik
> > > > > > http://stephen.evanchik.com
> > > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Stephen Evanchik
> > > > > http://stephen.evanchik.com
> > > > >
> > > >
> > > >
> > > >
> > > >
> > > > --
> > > > Stephen Evanchik
> > > > http://stephen.evanchik.com
> > > >
> > >
> > >
> > >
> > >
> > > --
> > > Stephen Evanchik
> > > http://stephen.evanchik.com
> > >
> > >
> > >
> >
> >
>
