Two questions WRT the Druid root pom and deployment process
1) When you deploy the jar files to Nexus, I note that they are signed by
GPG, yet, you do not have a GPG plugin defined in your pom. If you are
depending on the Apache parent pom "apache-release" profile, then where is
it being called? From a script?
2) You have a profile called "apache-release" that disables the
source-release-assembly. For the release to DIST, I presume you are
building the tarball yourself, where? The related question is what script
is using the above source-assembly.xml file to include the git.version +
other stuff.?
3) I have now studied the "git-commit-id-plugin". Nice and well
documented. Does this plugin introduce variables into the build process
like ${git.commit.id}, ${git.build.version}, etc? They are not defined
elsewhere. The jar-plugin needs the to insert into the manifest.
On Thu, Aug 1, 2019 at 11:42 AM Gian Merlino <[email protected]> wrote:
> > OK, I see it now in the tarball. Where is the script or code that loads
> > the data into git.version ?
>
> It's done by git-commit-id-plugin in our main POM:
> https://github.com/apache/incubator-druid/blob/master/pom.xml. Then it's
> included into the source assembly via:
>
> https://github.com/apache/incubator-druid/blob/master/distribution/src/assembly/source-assembly.xml
>
> > Yes, I understand. Once we graduate (PPMC -> PMC), isn't it just the PMC
> > that votes on a release?
>
> Anyone can vote, but only PMC votes are binding. Even top level projects
> still should have a 72 hour voting period for each release candidate,
> though, which can make release cycles take longer than you might be used
> to.
>
> On Thu, Aug 1, 2019 at 11:06 AM leerho <[email protected]> wrote:
>
> > >
> > > We do have it in the tarball root for the real release
> >
> > OK, I see it now in the tarball. Where is the script or code that loads
> > the data into git.version ?
> >
> > If they stay separate...
> >
> >
> > Yes, I understand. Once we graduate (PPMC -> PMC), isn't it just the PMC
> > that votes on a release?
> >
> >
> >
> >
> >
> > On Wed, Jul 31, 2019 at 7:46 PM Gian Merlino <[email protected]> wrote:
> >
> > > > I gather you decided not to have a git.version file at the root and
> you
> > > > added that info into the jar manifests, so where do you have that
> > > > information for the tar.gz assemblies?
> > >
> > > We do have it in the tarball root for the real release -- the source
> > > tarball (ASF policy considers source releases as the sole official
> > > release). For the binary convenience artifacts, we don't include it at
> > the
> > > root, but the jars have the info if people go digging.
> > >
> > > > Even the Java-only components have vastly different dependencies and
> > > > version cycles and don't necessarily need to be released at the same
> > time
> > > > the core-java component is released. So far, we don't have releases
> > that
> > > > span multiple components. For example, the core-java library may be
> > > > several versions ahead of the hive or pig components.
> > >
> > > Sure, that makes sense. I would still consider aligning their release
> > > processes though. If they stay separate, each component release would
> > need
> > > its own release manager, its own voting cycle, its own publish and
> > > announcement process, etc, and that can be a lot of administrative
> > > overhead.
> > >
> > > On Wed, Jul 31, 2019 at 7:07 PM leerho <[email protected]> wrote:
> > >
> > > > Gian,
> > > > Thanks!
> > > >
> > > >
> > > > > https://github.com/apache/incubator-druid/pull/6482.
> > > >
> > > >
> > > > The discussion about the traceability of the GA back to the final RC
> is
> > > > interesting.
> > > > I gather you decided not to have a git.version file at the root and
> you
> > > > added that info into the jar manifests, so where do you have that
> > > > information for the tar.gz assemblies?
> > > >
> > > > Also, it is more work upfront, but you might find that in the long
> run,
> > > you
> > > > > enjoy switching the project to a single repo with a multi-module
> > Maven
> > > > > build. We do this in Druid and it works quite well. It makes it way
> > > > easier
> > > > > to do patches and releases that span multiple modules.
> > > >
> > > >
> > > > The DataSketches library is not like a single product with components
> > > that
> > > > are released together. Some of our "components" have different
> > > life-cycles
> > > > and versioning. Also, we have C++ / Python repositories that don't
> fit
> > > > well with Maven, which is all about Java.
> > > >
> > > > Even the Java-only components have vastly different dependencies and
> > > > version cycles and don't necessarily need to be released at the same
> > time
> > > > the core-java component is released. So far, we don't have releases
> > that
> > > > span multiple components. For example, the core-java library may be
> > > > several versions ahead of the hive or pig components.
> > > >
> > > > Definitely doing as much as possible in the build system (whether
> Maven
> > > or
> > > > > Gradle) is a good thing, IMO, since they tend to reward living in
> > their
> > > > > worlds. Also the more verifications you can push into the build
> tool,
> > > the
> > > > > better.
> > > >
> > > >
> > > > I agree.
> > > >
> > > > GPG ?
> > > >
> > > >
> > > > You are correct. The .asc signature is generated with GPG.
> > > >
> > > >
> > > >
> > > >
> > > > On Tue, Jul 30, 2019 at 5:22 PM Gian Merlino <[email protected]>
> wrote:
> > > >
> > > > > Hey Lee,
> > > > >
> > > > > You might find this PR useful, it is where we Apache-fied the Druid
> > > POM:
> > > > > https://github.com/apache/incubator-druid/pull/6482. It helps
> > > highlight
> > > > > the
> > > > > specific areas that are relevant to Apache-fying a build.
> > > > >
> > > > > Also, it is more work upfront, but you might find that in the long
> > run,
> > > > you
> > > > > enjoy switching the project to a single repo with a multi-module
> > Maven
> > > > > build. We do this in Druid and it works quite well. It makes it way
> > > > easier
> > > > > to do patches and releases that span multiple modules.
> > > > >
> > > > > > So it seems that eventually, either we need to go whole hog
> having
> > > > Maven
> > > > > do
> > > > > > everything, which seems overly complicated, or consider moving to
> > > > Gradle
> > > > > > (which I am not familiar with either, but it seems lots of teams
> > are
> > > > > moving
> > > > > > that way).
> > > > >
> > > > > Definitely doing as much as possible in the build system (whether
> > Maven
> > > > or
> > > > > Gradle) is a good thing, IMO, since they tend to reward living in
> > their
> > > > > worlds. Also the more verifications you can push into the build
> tool,
> > > the
> > > > > better.
> > > > >
> > > > > In Druid we have also been considering moving to Gradle:
> > > > > https://github.com/apache/incubator-druid/issues/7866. However we
> > are
> > > > > still
> > > > > using Maven for now and may be for some time to come, since it
> seems
> > > the
> > > > > benefits of switching to Gradle are definitely noticeable, but not
> > > huge.
> > > > >
> > > > > > - It doesn't appear that the Nexus artifacts require a GPG
> > signature
> > > > only
> > > > > > the DIST assembly requires GPG, which the script is handling. So
> > why
> > > > do
> > > > > we
> > > > > > need the gpg-plugin?
> > > > >
> > > > > IIRC it's because Sonatype's Central Repository (where the
> > Maven-ready
> > > > > artifacts ultimately end up) requires GPG signatures.
> > > > >
> > > > > > - It is not clear to me when a profile should be used (e.g., for
> > > > deploy)
> > > > > vs
> > > > > > using the deploy process as part of the pom main build lifecycle.
> > > > >
> > > > > I've found profiles to be most useful when you want something like
> > "mvn
> > > > > package" to do different things based on a profile. e.g. having a
> > > profile
> > > > > to apply strict checks that takes extra time, having a profile to
> > build
> > > > > extra assemblies, etc.
> > > > >
> > > >
> > >
> >
>
