> What is confusing is that the Apache parent pom defines the > maven-release-plugin > only in build.pluginManagement where it calls the apache-release profile. > Similarly, the Druid pom only defines the release plugin in > build.pluginManagement. > It has been my understanding that "pluginManagement" was only for > inheritance. > and that for a plugin to be actually executed it had to be referenced in > the build.plugins > section or in a profile.build.plugins.
This is getting outside of my Maven knowledge, but I would guess that the release plugin is special and doesn't need to be listed as an explicit plugin, since it has a special command to activate it (mvn release:prepare or release:perform). It doesn't run on "normal" Maven commands. > I don't see a profile called "dist". > Where do I find that? Both list and source-release-assembly-druid are in the "distribution" pom: https://github.com/apache/incubator-druid/blob/master/distribution/pom.xml. (Druid is a multi-module project, and the "distribution" module is the one that builds the tarballs. The main pom 'sets the stage' but doesn't actually do anything.) On Fri, Aug 2, 2019 at 2:18 PM leerho <[email protected]> wrote: > > > > It happens as part of the maven-release-plugin. > > What is confusing is that the Apache parent pom defines the > maven-release-plugin > only in build.pluginManagement where it calls the apache-release profile. > Similarly, the Druid pom only defines the release plugin in > build.pluginManagement. > It has been my understanding that "pluginManagement" was only for > inheritance. > and that for a plugin to be actually executed it had to be referenced in > the build.plugins > section or in a profile.build.plugins. > > > > > - Check out the tag, then build release candidate with: > > mvn clean install > > -Papache-release,dist,rat > > -DskipTests > > -Dgpg.keyname=<your GPG key fingerprint> > > I don't see a profile called "dist". > > We've defined our own, check out "source-release-assembly-druid". > > > Where do I find that? > > > > > > > On Fri, Aug 2, 2019 at 12:37 PM Gian Merlino <[email protected]> wrote: > > > > 1) When you deploy the jar files to Nexus, I note that they are signed > by > > > GPG, yet, you do not have a GPG plugin defined in your pom. If you are > > > depending on the Apache parent pom "apache-release" profile, then where > > is > > > it being called? From a script? > > > > It happens as part of the maven-release-plugin. The commands we run to > do a > > release candidate are something like: > > > > - Create a tag with: mvn -DreleaseVersion=0.14.1-incubating > > -DdevelopmentVersion=0.14.2-incubating-SNAPSHOT > > -Dtag=druid-0.14.1-incubating-rc1 -DpushChanges=false clean release:clean > > release:prepare > > - Check out the tag, then build release candidate with: mvn clean install > > -Papache-release,dist,rat -DskipTests -Dgpg.keyname=<your GPG key > > fingerprint> > > > > Later on, to upload the final artifacts to Nexus: > > > > - Using a saved release.properties file from (1), run: mvn > release:perform > > -Darguments=-Dgpg.keyname=<my-key> > > > > (If you lose the release.properties file, you should be able to recover > > anyway by specifying a few more arguments on the release:perform line, > but > > I don't recall what they are off the top of my head.) > > > > > 2) You have a profile called "apache-release" that disables the > > > source-release-assembly. For the release to DIST, I presume you are > > > building the tarball yourself, where? The related question is what > > script > > > is using the above source-assembly.xml file to include the git.version > + > > > other stuff.? > > > > We've defined our own, check out "source-release-assembly-druid". We > > disable the source-release-assembly inherited from the parent pom so we > can > > use ours instead. It gets built in the "mvn clean install" step above. > > > > > 3) I have now studied the "git-commit-id-plugin". Nice and well > > > documented. Does this plugin introduce variables into the build > process > > > like ${git.commit.id}, ${git.build.version}, etc? They are not > defined > > > elsewhere. The jar-plugin needs the to insert into the manifest. > > > > I don't know the details, but that sounds like a very reasonable guess. > > > > On Fri, Aug 2, 2019 at 11:46 AM leerho <[email protected]> wrote: > > > > > Two questions WRT the Druid root pom and deployment process > > > > > > 1) When you deploy the jar files to Nexus, I note that they are signed > by > > > GPG, yet, you do not have a GPG plugin defined in your pom. If you are > > > depending on the Apache parent pom "apache-release" profile, then where > > is > > > it being called? From a script? > > > > > > 2) You have a profile called "apache-release" that disables the > > > source-release-assembly. For the release to DIST, I presume you are > > > building the tarball yourself, where? The related question is what > > script > > > is using the above source-assembly.xml file to include the git.version > + > > > other stuff.? > > > > > > 3) I have now studied the "git-commit-id-plugin". Nice and well > > > documented. Does this plugin introduce variables into the build > process > > > like ${git.commit.id}, ${git.build.version}, etc? They are not > defined > > > elsewhere. The jar-plugin needs the to insert into the manifest. > > > > > > > > > > > > On Thu, Aug 1, 2019 at 11:42 AM Gian Merlino <[email protected]> wrote: > > > > > > > > OK, I see it now in the tarball. Where is the script or code that > > > loads > > > > > the data into git.version ? > > > > > > > > It's done by git-commit-id-plugin in our main POM: > > > > https://github.com/apache/incubator-druid/blob/master/pom.xml. Then > > it's > > > > included into the source assembly via: > > > > > > > > > > > > > > https://github.com/apache/incubator-druid/blob/master/distribution/src/assembly/source-assembly.xml > > > > > > > > > Yes, I understand. Once we graduate (PPMC -> PMC), isn't it just > the > > > PMC > > > > > that votes on a release? > > > > > > > > Anyone can vote, but only PMC votes are binding. Even top level > > projects > > > > still should have a 72 hour voting period for each release candidate, > > > > though, which can make release cycles take longer than you might be > > used > > > > to. > > > > > > > > On Thu, Aug 1, 2019 at 11:06 AM leerho <[email protected]> wrote: > > > > > > > > > > > > > > > > We do have it in the tarball root for the real release > > > > > > > > > > OK, I see it now in the tarball. Where is the script or code that > > > loads > > > > > the data into git.version ? > > > > > > > > > > If they stay separate... > > > > > > > > > > > > > > > Yes, I understand. Once we graduate (PPMC -> PMC), isn't it just > the > > > PMC > > > > > that votes on a release? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Wed, Jul 31, 2019 at 7:46 PM Gian Merlino <[email protected]> > > wrote: > > > > > > > > > > > > I gather you decided not to have a git.version file at the root > > and > > > > you > > > > > > > added that info into the jar manifests, so where do you have > that > > > > > > > information for the tar.gz assemblies? > > > > > > > > > > > > We do have it in the tarball root for the real release -- the > > source > > > > > > tarball (ASF policy considers source releases as the sole > official > > > > > > release). For the binary convenience artifacts, we don't include > it > > > at > > > > > the > > > > > > root, but the jars have the info if people go digging. > > > > > > > > > > > > > Even the Java-only components have vastly different > dependencies > > > and > > > > > > > version cycles and don't necessarily need to be released at the > > > same > > > > > time > > > > > > > the core-java component is released. So far, we don't have > > > releases > > > > > that > > > > > > > span multiple components. For example, the core-java library > may > > > be > > > > > > > several versions ahead of the hive or pig components. > > > > > > > > > > > > Sure, that makes sense. I would still consider aligning their > > release > > > > > > processes though. If they stay separate, each component release > > would > > > > > need > > > > > > its own release manager, its own voting cycle, its own publish > and > > > > > > announcement process, etc, and that can be a lot of > administrative > > > > > > overhead. > > > > > > > > > > > > On Wed, Jul 31, 2019 at 7:07 PM leerho <[email protected]> wrote: > > > > > > > > > > > > > Gian, > > > > > > > Thanks! > > > > > > > > > > > > > > > > > > > > > > https://github.com/apache/incubator-druid/pull/6482. > > > > > > > > > > > > > > > > > > > > > The discussion about the traceability of the GA back to the > final > > > RC > > > > is > > > > > > > interesting. > > > > > > > I gather you decided not to have a git.version file at the root > > and > > > > you > > > > > > > added that info into the jar manifests, so where do you have > that > > > > > > > information for the tar.gz assemblies? > > > > > > > > > > > > > > Also, it is more work upfront, but you might find that in the > > long > > > > run, > > > > > > you > > > > > > > > enjoy switching the project to a single repo with a > > multi-module > > > > > Maven > > > > > > > > build. We do this in Druid and it works quite well. It makes > it > > > way > > > > > > > easier > > > > > > > > to do patches and releases that span multiple modules. > > > > > > > > > > > > > > > > > > > > > The DataSketches library is not like a single product with > > > components > > > > > > that > > > > > > > are released together. Some of our "components" have different > > > > > > life-cycles > > > > > > > and versioning. Also, we have C++ / Python repositories that > > don't > > > > fit > > > > > > > well with Maven, which is all about Java. > > > > > > > > > > > > > > Even the Java-only components have vastly different > dependencies > > > and > > > > > > > version cycles and don't necessarily need to be released at the > > > same > > > > > time > > > > > > > the core-java component is released. So far, we don't have > > > releases > > > > > that > > > > > > > span multiple components. For example, the core-java library > may > > > be > > > > > > > several versions ahead of the hive or pig components. > > > > > > > > > > > > > > Definitely doing as much as possible in the build system > (whether > > > > Maven > > > > > > or > > > > > > > > Gradle) is a good thing, IMO, since they tend to reward > living > > in > > > > > their > > > > > > > > worlds. Also the more verifications you can push into the > build > > > > tool, > > > > > > the > > > > > > > > better. > > > > > > > > > > > > > > > > > > > > > I agree. > > > > > > > > > > > > > > GPG ? > > > > > > > > > > > > > > > > > > > > > You are correct. The .asc signature is generated with GPG. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Tue, Jul 30, 2019 at 5:22 PM Gian Merlino <[email protected]> > > > > wrote: > > > > > > > > > > > > > > > Hey Lee, > > > > > > > > > > > > > > > > You might find this PR useful, it is where we Apache-fied the > > > Druid > > > > > > POM: > > > > > > > > https://github.com/apache/incubator-druid/pull/6482. It > helps > > > > > > highlight > > > > > > > > the > > > > > > > > specific areas that are relevant to Apache-fying a build. > > > > > > > > > > > > > > > > Also, it is more work upfront, but you might find that in the > > > long > > > > > run, > > > > > > > you > > > > > > > > enjoy switching the project to a single repo with a > > multi-module > > > > > Maven > > > > > > > > build. We do this in Druid and it works quite well. It makes > it > > > way > > > > > > > easier > > > > > > > > to do patches and releases that span multiple modules. > > > > > > > > > > > > > > > > > So it seems that eventually, either we need to go whole hog > > > > having > > > > > > > Maven > > > > > > > > do > > > > > > > > > everything, which seems overly complicated, or consider > > moving > > > to > > > > > > > Gradle > > > > > > > > > (which I am not familiar with either, but it seems lots of > > > teams > > > > > are > > > > > > > > moving > > > > > > > > > that way). > > > > > > > > > > > > > > > > Definitely doing as much as possible in the build system > > (whether > > > > > Maven > > > > > > > or > > > > > > > > Gradle) is a good thing, IMO, since they tend to reward > living > > in > > > > > their > > > > > > > > worlds. Also the more verifications you can push into the > build > > > > tool, > > > > > > the > > > > > > > > better. > > > > > > > > > > > > > > > > In Druid we have also been considering moving to Gradle: > > > > > > > > https://github.com/apache/incubator-druid/issues/7866. > However > > > we > > > > > are > > > > > > > > still > > > > > > > > using Maven for now and may be for some time to come, since > it > > > > seems > > > > > > the > > > > > > > > benefits of switching to Gradle are definitely noticeable, > but > > > not > > > > > > huge. > > > > > > > > > > > > > > > > > - It doesn't appear that the Nexus artifacts require a GPG > > > > > signature > > > > > > > only > > > > > > > > > the DIST assembly requires GPG, which the script is > handling. > > > So > > > > > why > > > > > > > do > > > > > > > > we > > > > > > > > > need the gpg-plugin? > > > > > > > > > > > > > > > > IIRC it's because Sonatype's Central Repository (where the > > > > > Maven-ready > > > > > > > > artifacts ultimately end up) requires GPG signatures. > > > > > > > > > > > > > > > > > - It is not clear to me when a profile should be used > (e.g., > > > for > > > > > > > deploy) > > > > > > > > vs > > > > > > > > > using the deploy process as part of the pom main build > > > lifecycle. > > > > > > > > > > > > > > > > I've found profiles to be most useful when you want something > > > like > > > > > "mvn > > > > > > > > package" to do different things based on a profile. e.g. > > having a > > > > > > profile > > > > > > > > to apply strict checks that takes extra time, having a > profile > > to > > > > > build > > > > > > > > extra assemblies, etc. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
