Hi Curtis,
First I want to say this plugin works great, I added it to the build and it
caught a number of issues before we went to release.
I do have a few questions. First I'm not sure I have it fully configured
correctly, my usage is below. With this usage it found all the cases where
the build was consuming a snapshot not being generated in the reactor which
is exactly what I was looking for.
There are a few other options I need as well and was hoping you could
provide some guidance on how to support them.
1. I didn't set requireReproducibleBuilds as I wasn't sure how or if that
is done internally by the verify-no-snapshots goal. (Worked really well
w/o setting it.)
2. How to skip per pom? I didn't see any skip configuration option so as a
workaround I set the groupId in configuration to be 'skip' and that works.
3. How to disable globally via command line argument? I tried
-Denforcer.skip=true but it doesn't seem to work...still testing that again.
4. How to exclude a few groupIds from enforcement? We have a couple cases
where we do consume an approved snapshot so I need a way to configure
allowed snapshot groupIds.
5. How to ignore certain points of failure such as a artifact pom (not a
snapshot one) that has two organizational sections currently causes a build
failure, I'd like to just warn on those types of issues.
Again, works super well but I just need to tweak its configuration a bit.
<plugin>
<artifactId>maven-enforcer-plugin</artifactId>
<version>1.3.1</version>
<dependencies>
<dependency>
<groupId>org.codehaus.mojo</groupId>
<artifactId>extra-enforcer-rules</artifactId>
<version>1.0-beta-3</version>
</dependency>
<dependency>
<groupId>org.scijava</groupId>
<artifactId>scijava-maven-plugin</artifactId>
<version>${scijava-maven-plugin.version}</version>
</dependency>
</dependencies>
</plugin>
<plugin>
<groupId>org.scijava</groupId>
<artifactId>scijava-maven-plugin</artifactId>
<executions>
<execution>
<id>verify-no-nonreactor-snapshots</id>
<phase>validate</phase>
<goals>
<goal>verify-no-snapshots</goal>
</goals>
<configuration>
</configuration>
</execution>
</executions>
</plugin>
On Wed, Jul 22, 2015 at 10:12 PM, Curtis Rueden <[email protected]> wrote:
> Hi Dave,
>
> > if you have a plugin that solves the same problem that is fine.
> > If you could elaborate on your solution that would be great.
>
> The plugin I mentioned, scijava-maven-plugin [1], has a
> "verify-no-snapshots" goal—and corresponding enforcer rule
> "requireReproducibleBuilds"—which is more thorough than the enforcer's
> built-in "no snapshots" rule: it recursively scans dependencies for
> snapshot dependencies and parents, and fails the build if any are found.
> This is important because some "releases" on Maven Central are actually
> tainted, containing snapshot dependencies somewhere upstream.
>
> My group's projects use one Git repo = one single-module Maven project =
> one JAR file. Projects use release couplings between components so that
> builds are reproducible [2]. We avoid multi-module builds because they are
> more complex and make it harder to release components individually
> [3]—although it is still possible [4]. The requireReproducibleBuilds rule
> understands multi-module builds and will not fail the build unless there is
> a snapshot dependency outside the reactor.
>
> The requireReproducibleBuilds rule would go a long way toward solving your
> use case, but might not catch every scenario such as old GAVs referenced
> from an unpack goal config. However, the proper way to solve that is to
> declare version properties in the parent for all your GAs, then use those
> properties everywhere that version is needed. E.g.:
>
> -
>
> https://github.com/scijava/pom-scijava/blob/pom-scijava-7.5.2/pom.xml#L133-L134
> -
>
> https://github.com/scijava/pom-scijava/blob/pom-scijava-7.5.2/pom.xml#L815-L819
> -
>
> https://github.com/scijava/pom-scijava/blob/pom-scijava-7.5.2/pom.xml#L1094-L1099
>
> Then your chances of out-of-sync versions will be drastically reduced.
>
> Regards,
> Curtis
>
> [1] https://github.com/scijava/scijava-maven-plugin
>
> [2] http://imagej.net/Reproducible_builds
>
> [3] http://imagej.net/Philosophy#Release_early.2C_release_often
>
> [4] We do have one exception (https://github.com/trakem2/TrakEM2) which is
> a multi-module build, but with separately versioned components. Cutting a
> release of (some subset of) those modules is unintuitive:
> - We comment out the irrelevant modules in the aggregator POM (which
> doubles as the parent POM), leaving only the modules to be released; then
> - bump the versions of those modules forward to their new release versions;
> and
> - bump the version properties of the not-being-released modules _backward_
> to their last release versions.
> Then the (reduced in size) reactor has no snapshot dependencies in the
> released modules. But we cannot release in this fashion using the
> maven-release-plugin, unfortunately. So there are lots of downsides. The
> upside is that you maintain a reproducible build on master always, with
> tightly coupled modules normally, and proper release couplings for each set
> of released modules.
>
> On Wed, Jul 22, 2015 at 9:49 AM, David Hoffer <[email protected]> wrote:
>
> > Hi Curtis,
> >
> > Yes that is the issue I'm trying to solve, except that we are less formal
> > than your description implies. We don't have 'approved' and
> 'non-approved'
> > snapshots, rather we have valid ones and ones made obsolete by the
> > refactor.
> >
> > Leaning on Nexus doesn't seem like a bad solution as after the refactor,
> as
> > it does no one any good (including Nexus) to continue to retain the old
> > artifacts, it would just clean things up if they could be deleted. That
> > being said...if you have a plugin that solves the same problem that is
> > fine. If you could elaborate on your solution that would be great. I'd
> > actually prefer to 'fix' this in the pom rather than Nexus as our Nexus
> is
> > managed by IT and it's difficult for devs to get more than read
> privileges.
> >
> > -Dave
> >
> > On Wed, Jul 22, 2015 at 8:24 AM, Curtis Rueden <[email protected]>
> wrote:
> >
> > > Hi Dave,
> > >
> > > This problem strikes me as just a particular incarnation of "make sure
> > only
> > > approved deps are used" where old snapshot versions of 1st party
> modules
> > > are no longer "approved" after a refactoring.
> > >
> > > As such, I would suggest looking for tools intended to support such
> > > dependency analysis more generally, rather than leaning on Nexus in the
> > way
> > > you are trying to do.
> > >
> > > I personally also prefer no snapshot deps ever outside the reactor, as
> it
> > > will result in irreproducible builds. My group actually wrote an
> enforcer
> > > rule to detect exactly that situation; it is part of the
> > > scijava-maven-plugin. Happy to elaborate if you are interested.
> > >
> > > -Curtis
> > > On Jul 22, 2015 8:47 AM, "Ron Wheeler" <[email protected]
> >
> > > wrote:
> > >
> > > > Glad you have found a solution even if I can not understand the
> source
> > of
> > > > the problem.
> > > > It does not seem to a general problem which is why you have to
> develop
> > a
> > > > custom solution.
> > > > This is a pretty big mailing list and attracts a lot of experienced
> > > > developers and if there was a general solution, someone would have
> > chimed
> > > > in by now.
> > > >
> > > > Good luck.
> > > > Ron
> > > >
> > > >
> > > > On 22/07/2015 1:11 AM, David Hoffer wrote:
> > > >
> > > >> Apparently our communication has broken down, it seems your not
> > > >> understanding the issue/question.
> > > >>
> > > >
> > > >
> > > >> I did find that Nexus does have an API we can use for this...I sure
> > wish
> > > >> there was a more 'packaged' solution but I've discussed it with our
> IT
> > > >> department and between us I think we can solve this issue using that
> > > >> approach. If anyone knows of a better solution please let me know.
> > > >>
> > > >> -Dave
> > > >>
> > > >> On Tue, Jul 21, 2015 at 10:32 PM, Ron Wheeler <
> > > >> [email protected]> wrote:
> > > >>
> > > >> If you have SNAPSHOTs specified, you will get SNAPSHOTs in the
> build.
> > > >>> When you remove the SNAPSHOT from the parent, there should not be
> any
> > > way
> > > >>> for a SNAPSHOT to be included.
> > > >>>
> > > >>>
> > > >>> I am not sure where the SNAPSHOTS are being brought in to a x.x.x
> > > release
> > > >>> unless you are allowing modules to specify the versions of their
> > > >>> dependencies.
> > > >>> Stop that.
> > > >>> Modules should have no versions on an dependency unless it is a
> > > reference
> > > >>> to a property in the parent.
> > > >>> Modules should have no references to a version on a dependency that
> > is
> > > >>> one
> > > >>> of the modules that your wrote -
> > <version>${project.version}</version>
> > > >>>
> > > >>> Ron
> > > >>>
> > > >>>
> > > >>> On 21/07/2015 11:40 PM, David Hoffer wrote:
> > > >>>
> > > >>> I didn't say x.x.x is the only version in the parent. I said it
> is
> > a
> > > >>>> SNAPSHOT. The version varies (of course) but in my prior example
> I
> > > said
> > > >>>> it
> > > >>>> was 1.0-SNAPSHOT.
> > > >>>>
> > > >>>> -Dave
> > > >>>>
> > > >>>> On Tue, Jul 21, 2015 at 9:36 PM, Ron Wheeler <
> > > >>>> [email protected]
> > > >>>>
> > > >>>> wrote:
> > > >>>>> Where are the possible SNAPSHOT versions creeping into the build
> if
> > > >>>>> x.x.x
> > > >>>>> is the only versions in your parent and the dependencies do not
> > have
> > > >>>>> any
> > > >>>>> versions (as I suggested).
> > > >>>>>
> > > >>>>> Ron
> > > >>>>>
> > > >>>>>
> > > >>>>> On 21/07/2015 10:54 PM, David Hoffer wrote:
> > > >>>>>
> > > >>>>> Yes we use one version for all modules...comes from top level.
> > > What
> > > >>>>> I
> > > >>>>>
> > > >>>>>> mean
> > > >>>>>> is this is a non-release build so by maven definition is a
> > snapshot.
> > > >>>>>> E.g.
> > > >>>>>> x.x.x is built only once at release, x.x.x-SNAPSHOT is built on
> > > every
> > > >>>>>> CI
> > > >>>>>> build.
> > > >>>>>>
> > > >>>>>> -Dave
> > > >>>>>>
> > > >>>>>> On Tue, Jul 21, 2015 at 8:38 PM, Ron Wheeler <
> > > >>>>>> [email protected]
> > > >>>>>>
> > > >>>>>> wrote:
> > > >>>>>>
> > > >>>>>>> On 21/07/2015 5:53 PM, David Hoffer wrote:
> > > >>>>>>>
> > > >>>>>>> I'm not sure I understand your reply. We use dependency
> > > >>>>>>> management
> > > >>>>>>> to
> > > >>>>>>>
> > > >>>>>>> specify versions (for both external & project dependencies),
> > > however
> > > >>>>>>>> that's
> > > >>>>>>>> not the issue, we have no problem specifying the version to
> use
> > > for
> > > >>>>>>>> both
> > > >>>>>>>> of
> > > >>>>>>>> those. What is only in view here are the multi-module project
> > > >>>>>>>> dependencies
> > > >>>>>>>> and by definition they are all SNAPSHOTS as we have not
> released
> > > >>>>>>>> yet.
> > > >>>>>>>>
> > > >>>>>>>> What do you mean "by definition"?
> > > >>>>>>>>
> > > >>>>>>>> If the modules use the parent version as their version, they
> > will
> > > >>>>>>> be
> > > >>>>>>> SNAPSHOTS or releases depending on the parent pom having a
> > version
> > > of
> > > >>>>>>> x.x.x-SNAPSHOT or x.x.x.
> > > >>>>>>> i.e. the module version is missing so that the parent's version
> > is
> > > >>>>>>> the
> > > >>>>>>> version of the module.
> > > >>>>>>> Any dependency in another module that is part of the project is
> > set
> > > >>>>>>> to
> > > >>>>>>> <version>${project.version}</version>
> > > >>>>>>> <dependency>
> > > >>>>>>> <groupId>com.example</groupId>
> > > >>>>>>> <artifactId>anything-core</artifactId>
> > > >>>>>>> <version>${project.version}</version>
> > > >>>>>>> <scope>provided</scope>
> > > >>>>>>> </dependency>
> > > >>>>>>>
> > > >>>>>>> Ron
> > > >>>>>>>
> > > >>>>>>>
> > > >>>>>>>
> > > >>>>>>>
> > > >>>>>>> Let me give an example that might help. The multi-module
> > > project
> > > >>>>>>> is
> > > >>>>>>>
> > > >>>>>>> large
> > > >>>>>>>> and is growing...you start out with these modules (all the
> > > versions
> > > >>>>>>>> are
> > > >>>>>>>> 1.0-SNAPSHOT).
> > > >>>>>>>>
> > > >>>>>>>> groupId=com.mycompany.myproject
> > > >>>>>>>> artifactId=artifactA, artifactB, artifactC, artifact1,
> > artifact2,
> > > >>>>>>>> artifact3
> > > >>>>>>>>
> > > >>>>>>>> This has been building with your CI system for 1 month when
> you
> > > >>>>>>>> realize
> > > >>>>>>>> you
> > > >>>>>>>> really want these modules.
> > > >>>>>>>>
> > > >>>>>>>> groupId=com.mycompany.myproject
> > > >>>>>>>> artifactId=app-parent
> > > >>>>>>>>
> > > >>>>>>>> groupId=com.mycompany.myproject.service
> > > >>>>>>>> artifactId=artifactA, artifactB, artifactC
> > > >>>>>>>>
> > > >>>>>>>> groupId=com.mycompany.myproject.transform
> > > >>>>>>>> artifactId=artifact1, artifact2, artifact3
> > > >>>>>>>>
> > > >>>>>>>> This too builds fine, however in reality somewhere in this new
> > > build
> > > >>>>>>>> is
> > > >>>>>>>> a
> > > >>>>>>>> reference to
> > > >>>>>>>> com.mycompany.myproject:artifactA:1.0-SNAPSHOT...perhaps
> > > >>>>>>>> for
> > > >>>>>>>> an unpack goal. The build is fine as Nexus will always have
> > this
> > > >>>>>>>> artifact
> > > >>>>>>>> although it was removed from the build during the refactor.
> > > >>>>>>>>
> > > >>>>>>>> We want to purge all com.mycompany.myproject.* snapshots from
> > > Nexus
> > > >>>>>>>> so
> > > >>>>>>>> the
> > > >>>>>>>> CI build will fail until the build is correct.
> > > >>>>>>>>
> > > >>>>>>>> -Dave
> > > >>>>>>>>
> > > >>>>>>>>
> > > >>>>>>>>
> > > >>>>>>>>
> > > >>>>>>>>
> > > >>>>>>>> On Tue, Jul 21, 2015 at 3:20 PM, Ron Wheeler <
> > > >>>>>>>> [email protected]
> > > >>>>>>>>
> > > >>>>>>>> wrote:
> > > >>>>>>>>
> > > >>>>>>>> Using the parent pom to specify the versions of dependencies
> > > solves
> > > >>>>>>>>> this
> > > >>>>>>>>> problem for most people.
> > > >>>>>>>>>
> > > >>>>>>>>> If there are no SNAPSHOTS in the parent's properties and the
> > > parent
> > > >>>>>>>>> poms
> > > >>>>>>>>> version is not a SNAPSHOT, then your project is not being
> built
> > > >>>>>>>>> with
> > > >>>>>>>>> SNAPSHOTS.
> > > >>>>>>>>>
> > > >>>>>>>>> We never worry about the SNAPSHOTs in the repo.
> > > >>>>>>>>>
> > > >>>>>>>>> Ron
> > > >>>>>>>>>
> > > >>>>>>>>>
> > > >>>>>>>>> On 21/07/2015 2:42 PM, David Hoffer wrote:
> > > >>>>>>>>>
> > > >>>>>>>>> Yeah it appears our IT group is right...Nexus doesn't
> have
> > a
> > > >>>>>>>>> UI/feature
> > > >>>>>>>>>
> > > >>>>>>>>> to
> > > >>>>>>>>>
> > > >>>>>>>>>> do what we want. What other options are there?
> > > >>>>>>>>>>
> > > >>>>>>>>>> This would seem a common need, major project does a refactor
> > of
> > > >>>>>>>>>> Maven
> > > >>>>>>>>>> GA
> > > >>>>>>>>>> and want to delete all SNAPSHOTS used by the project to
> verify
> > > the
> > > >>>>>>>>>> refactor
> > > >>>>>>>>>> is 100% complete. We have had too many cases where the
> build
> > is
> > > >>>>>>>>>> still
> > > >>>>>>>>>> pointing to an old artifact that isn't part of the build
> > anymore
> > > >>>>>>>>>> yet
> > > >>>>>>>>>> the
> > > >>>>>>>>>> build is happy because old artifacts are still in Nexus.
> > > >>>>>>>>>>
> > > >>>>>>>>>> -Dave
> > > >>>>>>>>>>
> > > >>>>>>>>>> On Tue, Jul 21, 2015 at 12:36 PM, Karl Heinz Marbaise <
> > > >>>>>>>>>> [email protected]>
> > > >>>>>>>>>> wrote:
> > > >>>>>>>>>>
> > > >>>>>>>>>> Hi David,
> > > >>>>>>>>>>
> > > >>>>>>>>>> On 7/21/15 6:03 PM, David Hoffer wrote:
> > > >>>>>>>>>>
> > > >>>>>>>>>> We use Nexus as our corporate Maven repository and
> would
> > > >>>>>>>>>>> like
> > > >>>>>>>>>>> to
> > > >>>>>>>>>>>
> > > >>>>>>>>>>> periodically delete certain SNAPSHOT artifacts. We need
> > to
> > > be
> > > >>>>>>>>>>> able
> > > >>>>>>>>>>>
> > > >>>>>>>>>>> to
> > > >>>>>>>>>>>> filter/select by groupId and by version...so delete all
> > where
> > > >>>>>>>>>>>> groupId=com.mycomp.mygroupid.* and version=X.SNAPSHOT.
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>> You can only delete all kind of SNAPSHOT's in Nexus
> > based
> > > >>>>>>>>>>>> on a
> > > >>>>>>>>>>>> time
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>> frame
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>> for example delete all SNAPSHOT's which are older than 30
> > > days
> > > >>>>>>>>>>> etc..
> > > >>>>>>>>>>>
> > > >>>>>>>>>>>
> > > >>>>>>>>>>>
> > > >>>>>>>>>>> Our use case is that when we refactor part of the
> build
> > to
> > > >>>>>>>>>>> use
> > > >>>>>>>>>>> new
> > > >>>>>>>>>>>
> > > >>>>>>>>>>> groupIds
> > > >>>>>>>>>>>
> > > >>>>>>>>>>> the old ones are not valid anymore however sometimes there
> > is
> > > a
> > > >>>>>>>>>>>> lingering
> > > >>>>>>>>>>>> reference to the old groupId, if we can delete all the old
> > > >>>>>>>>>>>> SNAPSHOTS
> > > >>>>>>>>>>>> we
> > > >>>>>>>>>>>> could find those errors now instead of when we release.
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>> Any ideas on how to do this are much appreciated.
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>> -Dave
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>> Kind regards
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>> Karl Heinz Marbaise
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>>
> > > >>>
> > > >
> > > > --
> > > > Ron Wheeler
> > > > President
> > > > Artifact Software Inc
> > > > email: [email protected]
> > > > skype: ronaldmwheeler
> > > > phone: 866-970-2435, ext 102
> > > >
> > > >
> > > > ---------------------------------------------------------------------
> > > > To unsubscribe, e-mail: [email protected]
> > > > For additional commands, e-mail: [email protected]
> > > >
> > > >
> > >
> >
>
