> > It happens as part of the maven-release-plugin. What is confusing is that the Apache parent pom defines the maven-release-plugin only in build.pluginManagement where it calls the apache-release profile. Similarly, the Druid pom only defines the release plugin in build.pluginManagement. It has been my understanding that "pluginManagement" was only for inheritance. and that for a plugin to be actually executed it had to be referenced in the build.plugins section or in a profile.build.plugins.
> - Check out the tag, then build release candidate with: > mvn clean install > -Papache-release,dist,rat > -DskipTests > -Dgpg.keyname=<your GPG key fingerprint> I don't see a profile called "dist". We've defined our own, check out "source-release-assembly-druid". Where do I find that? On Fri, Aug 2, 2019 at 12:37 PM Gian Merlino <[email protected]> wrote: > > 1) When you deploy the jar files to Nexus, I note that they are signed by > > GPG, yet, you do not have a GPG plugin defined in your pom. If you are > > depending on the Apache parent pom "apache-release" profile, then where > is > > it being called? From a script? > > It happens as part of the maven-release-plugin. The commands we run to do a > release candidate are something like: > > - Create a tag with: mvn -DreleaseVersion=0.14.1-incubating > -DdevelopmentVersion=0.14.2-incubating-SNAPSHOT > -Dtag=druid-0.14.1-incubating-rc1 -DpushChanges=false clean release:clean > release:prepare > - Check out the tag, then build release candidate with: mvn clean install > -Papache-release,dist,rat -DskipTests -Dgpg.keyname=<your GPG key > fingerprint> > > Later on, to upload the final artifacts to Nexus: > > - Using a saved release.properties file from (1), run: mvn release:perform > -Darguments=-Dgpg.keyname=<my-key> > > (If you lose the release.properties file, you should be able to recover > anyway by specifying a few more arguments on the release:perform line, but > I don't recall what they are off the top of my head.) > > > 2) You have a profile called "apache-release" that disables the > > source-release-assembly. For the release to DIST, I presume you are > > building the tarball yourself, where? The related question is what > script > > is using the above source-assembly.xml file to include the git.version + > > other stuff.? > > We've defined our own, check out "source-release-assembly-druid". We > disable the source-release-assembly inherited from the parent pom so we can > use ours instead. It gets built in the "mvn clean install" step above. > > > 3) I have now studied the "git-commit-id-plugin". Nice and well > > documented. Does this plugin introduce variables into the build process > > like ${git.commit.id}, ${git.build.version}, etc? They are not defined > > elsewhere. The jar-plugin needs the to insert into the manifest. > > I don't know the details, but that sounds like a very reasonable guess. > > On Fri, Aug 2, 2019 at 11:46 AM leerho <[email protected]> wrote: > > > Two questions WRT the Druid root pom and deployment process > > > > 1) When you deploy the jar files to Nexus, I note that they are signed by > > GPG, yet, you do not have a GPG plugin defined in your pom. If you are > > depending on the Apache parent pom "apache-release" profile, then where > is > > it being called? From a script? > > > > 2) You have a profile called "apache-release" that disables the > > source-release-assembly. For the release to DIST, I presume you are > > building the tarball yourself, where? The related question is what > script > > is using the above source-assembly.xml file to include the git.version + > > other stuff.? > > > > 3) I have now studied the "git-commit-id-plugin". Nice and well > > documented. Does this plugin introduce variables into the build process > > like ${git.commit.id}, ${git.build.version}, etc? They are not defined > > elsewhere. The jar-plugin needs the to insert into the manifest. > > > > > > > > On Thu, Aug 1, 2019 at 11:42 AM Gian Merlino <[email protected]> wrote: > > > > > > OK, I see it now in the tarball. Where is the script or code that > > loads > > > > the data into git.version ? > > > > > > It's done by git-commit-id-plugin in our main POM: > > > https://github.com/apache/incubator-druid/blob/master/pom.xml. Then > it's > > > included into the source assembly via: > > > > > > > > > https://github.com/apache/incubator-druid/blob/master/distribution/src/assembly/source-assembly.xml > > > > > > > Yes, I understand. Once we graduate (PPMC -> PMC), isn't it just the > > PMC > > > > that votes on a release? > > > > > > Anyone can vote, but only PMC votes are binding. Even top level > projects > > > still should have a 72 hour voting period for each release candidate, > > > though, which can make release cycles take longer than you might be > used > > > to. > > > > > > On Thu, Aug 1, 2019 at 11:06 AM leerho <[email protected]> wrote: > > > > > > > > > > > > > We do have it in the tarball root for the real release > > > > > > > > OK, I see it now in the tarball. Where is the script or code that > > loads > > > > the data into git.version ? > > > > > > > > If they stay separate... > > > > > > > > > > > > Yes, I understand. Once we graduate (PPMC -> PMC), isn't it just the > > PMC > > > > that votes on a release? > > > > > > > > > > > > > > > > > > > > > > > > On Wed, Jul 31, 2019 at 7:46 PM Gian Merlino <[email protected]> > wrote: > > > > > > > > > > I gather you decided not to have a git.version file at the root > and > > > you > > > > > > added that info into the jar manifests, so where do you have that > > > > > > information for the tar.gz assemblies? > > > > > > > > > > We do have it in the tarball root for the real release -- the > source > > > > > tarball (ASF policy considers source releases as the sole official > > > > > release). For the binary convenience artifacts, we don't include it > > at > > > > the > > > > > root, but the jars have the info if people go digging. > > > > > > > > > > > Even the Java-only components have vastly different dependencies > > and > > > > > > version cycles and don't necessarily need to be released at the > > same > > > > time > > > > > > the core-java component is released. So far, we don't have > > releases > > > > that > > > > > > span multiple components. For example, the core-java library may > > be > > > > > > several versions ahead of the hive or pig components. > > > > > > > > > > Sure, that makes sense. I would still consider aligning their > release > > > > > processes though. If they stay separate, each component release > would > > > > need > > > > > its own release manager, its own voting cycle, its own publish and > > > > > announcement process, etc, and that can be a lot of administrative > > > > > overhead. > > > > > > > > > > On Wed, Jul 31, 2019 at 7:07 PM leerho <[email protected]> wrote: > > > > > > > > > > > Gian, > > > > > > Thanks! > > > > > > > > > > > > > > > > > > > https://github.com/apache/incubator-druid/pull/6482. > > > > > > > > > > > > > > > > > > The discussion about the traceability of the GA back to the final > > RC > > > is > > > > > > interesting. > > > > > > I gather you decided not to have a git.version file at the root > and > > > you > > > > > > added that info into the jar manifests, so where do you have that > > > > > > information for the tar.gz assemblies? > > > > > > > > > > > > Also, it is more work upfront, but you might find that in the > long > > > run, > > > > > you > > > > > > > enjoy switching the project to a single repo with a > multi-module > > > > Maven > > > > > > > build. We do this in Druid and it works quite well. It makes it > > way > > > > > > easier > > > > > > > to do patches and releases that span multiple modules. > > > > > > > > > > > > > > > > > > The DataSketches library is not like a single product with > > components > > > > > that > > > > > > are released together. Some of our "components" have different > > > > > life-cycles > > > > > > and versioning. Also, we have C++ / Python repositories that > don't > > > fit > > > > > > well with Maven, which is all about Java. > > > > > > > > > > > > Even the Java-only components have vastly different dependencies > > and > > > > > > version cycles and don't necessarily need to be released at the > > same > > > > time > > > > > > the core-java component is released. So far, we don't have > > releases > > > > that > > > > > > span multiple components. For example, the core-java library may > > be > > > > > > several versions ahead of the hive or pig components. > > > > > > > > > > > > Definitely doing as much as possible in the build system (whether > > > Maven > > > > > or > > > > > > > Gradle) is a good thing, IMO, since they tend to reward living > in > > > > their > > > > > > > worlds. Also the more verifications you can push into the build > > > tool, > > > > > the > > > > > > > better. > > > > > > > > > > > > > > > > > > I agree. > > > > > > > > > > > > GPG ? > > > > > > > > > > > > > > > > > > You are correct. The .asc signature is generated with GPG. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Tue, Jul 30, 2019 at 5:22 PM Gian Merlino <[email protected]> > > > wrote: > > > > > > > > > > > > > Hey Lee, > > > > > > > > > > > > > > You might find this PR useful, it is where we Apache-fied the > > Druid > > > > > POM: > > > > > > > https://github.com/apache/incubator-druid/pull/6482. It helps > > > > > highlight > > > > > > > the > > > > > > > specific areas that are relevant to Apache-fying a build. > > > > > > > > > > > > > > Also, it is more work upfront, but you might find that in the > > long > > > > run, > > > > > > you > > > > > > > enjoy switching the project to a single repo with a > multi-module > > > > Maven > > > > > > > build. We do this in Druid and it works quite well. It makes it > > way > > > > > > easier > > > > > > > to do patches and releases that span multiple modules. > > > > > > > > > > > > > > > So it seems that eventually, either we need to go whole hog > > > having > > > > > > Maven > > > > > > > do > > > > > > > > everything, which seems overly complicated, or consider > moving > > to > > > > > > Gradle > > > > > > > > (which I am not familiar with either, but it seems lots of > > teams > > > > are > > > > > > > moving > > > > > > > > that way). > > > > > > > > > > > > > > Definitely doing as much as possible in the build system > (whether > > > > Maven > > > > > > or > > > > > > > Gradle) is a good thing, IMO, since they tend to reward living > in > > > > their > > > > > > > worlds. Also the more verifications you can push into the build > > > tool, > > > > > the > > > > > > > better. > > > > > > > > > > > > > > In Druid we have also been considering moving to Gradle: > > > > > > > https://github.com/apache/incubator-druid/issues/7866. However > > we > > > > are > > > > > > > still > > > > > > > using Maven for now and may be for some time to come, since it > > > seems > > > > > the > > > > > > > benefits of switching to Gradle are definitely noticeable, but > > not > > > > > huge. > > > > > > > > > > > > > > > - It doesn't appear that the Nexus artifacts require a GPG > > > > signature > > > > > > only > > > > > > > > the DIST assembly requires GPG, which the script is handling. > > So > > > > why > > > > > > do > > > > > > > we > > > > > > > > need the gpg-plugin? > > > > > > > > > > > > > > IIRC it's because Sonatype's Central Repository (where the > > > > Maven-ready > > > > > > > artifacts ultimately end up) requires GPG signatures. > > > > > > > > > > > > > > > - It is not clear to me when a profile should be used (e.g., > > for > > > > > > deploy) > > > > > > > vs > > > > > > > > using the deploy process as part of the pom main build > > lifecycle. > > > > > > > > > > > > > > I've found profiles to be most useful when you want something > > like > > > > "mvn > > > > > > > package" to do different things based on a profile. e.g. > having a > > > > > profile > > > > > > > to apply strict checks that takes extra time, having a profile > to > > > > build > > > > > > > extra assemblies, etc. > > > > > > > > > > > > > > > > > > > > > > > > > > > >
