Here is my take on this discussion. Please stick with me for a bit.
I think that there is a communication probleem; crossed wires in
terminology. Matthew talks about "general public" and most of us grey
heads (trying not to be gender specific here) in the ASF hear "release".
What I think Matthew is talking about is a wider testing audience. People
who understand that the code is not "released". If that is
indeed the case, and I am certain Matthew will tell me if I am wrong, then
the following should make sense.
- Pekko can build and place into the SNAPSHOT repository, nightly
builds. These builds should be named <pekko-module>-x.y.z-SNAPSHOT. The
repository will take care of adding dates to them to ensure the
latest SNAPSHOT is returned. (The repository should also automatically
retain only 5 or so snapshots for pekko-module-x.y.z but that is a
different issue)
- Pekko can build and place into the SNAPSHOT repository, a longer
period (weekly,monthly) or manually triggered build that has a name like
<pekko-module>-x.y.z-M-SNAPSHOT. Where "M" is the literal 'M'. This will
have the same retention properties as <pekko-module>-x.y.z-SNAPSHOT.
- The "wider testing audience" needs to be pointed to the dev mailing
list. There are several reasons for this, most notably: They have access
to the discussion about what is changing, they can be notified when an "M"
release is created, they can contribute ideas, suggestions and concepts
back to the community, they become part of the community.
- The "wider testing audience" can depend upon
<pekko-module>-x.y.z.M-SNAPSHOT during their builds, provided they add the
ASF snapshot repository.
- There is no "release" here, all work is in the "dev" side of the house.
- The pekko dev team can produce Milestone ("M") builds when they
desire, but may need to do manual cleanup of the repository when the actual
release is made.
- The pekko dev team can produce Regression ("R") builds that perform
extensive regression testing if so desired.
- The pekko dev team can produce Foo ("F") builds to ensure foo
correctness (whatever that is) if so desired.
- The pekko dev team will need to clean up all the lettered versions
when the release is made or Infra will become very annoyed.
- The pekko dev team should try to limit the number of lettered versions
so that infra does not become annoyed.
The short bit here is that as long as the work is development work
(building, testing, and verification) and not for the "general public"
(grey head definition) then it can safely be done and "released" (non-grey
head definition) within the dev environment (source, build, repository, and
community tools).
My recommendations are:
- Bring the "wider testing audience" into the dev conversations.
- Announce milestone builds and internal releases in the dev list.
- Use the snapshot repositories to their fullest.
- expand the pool of potential team members.
That last point brings me to the other topic of PPMC members, and I
apologize for not seeing and paying attention to the discussion on the
private list and will endeavour to do better in future.
Claude
On Sun, Aug 6, 2023 at 4:01 PM Matthew de Detrich
<matthew.dedetr...@aiven.io.invalid> wrote:
> > I agree with all of Justin McLean's points.
>
> > I care about the ASF rules
>
> I am still waiting for an actual reference to a rule on this and Justin
> appears to have misunderstood what was being suggested (but I
> will wait for his answer)
>
> > and norms.
>
> I have seen more than non negligible amount of so-called "norms"
> which turn out to be either false or some convention which projects
> copy from other projects just because. Pekko is also not a typical
> project in this specific regard, which by definition means its not
> normal so I in this case I don't about norms unless there is a rule
> behind it.
>
>
> On Sun, Aug 6, 2023 at 3:48 PM PJ Fanning <fannin...@apache.org> wrote:
>
> > I agree with all of Justin McLean's points.
> >
> > I absolutely don't care what non-ASF projects do. I care about the ASF
> > rules and norms.
> >
> > On Sun, 6 Aug 2023 at 14:44, Matthew de Detrich
> > <matthew.dedetr...@aiven.io.invalid> wrote:
> > >
> > > > I remain strongly opposed to publishing to maven central without a
> > > full release process with 2 phase voting.
> > >
> > > Can you elaborate on this opposition? There is plenty of software that
> is
> > > distributed on maven using commonly established milestone suffix
> > > (i.e. -M0, -M1 etc etc) and I would argue it's far from controversial
> to
> > > state that it's extremely clear for users that such an artifact is a
> > > milestone, this is already established.
> > >
> > > I just want to know where you are coming from, because if it's just due
> > > to it being a rule/Apache process this is what I want to clarify (if
> > there
> > > is a clear specific rule on this that this is not allowed; which has
> yet
> > > to be provided yet then yes I will drop it).
> > >
> > > On Sun, Aug 6, 2023 at 3:30 PM PJ Fanning <fannin...@gmail.com> wrote:
> > >
> > > > I remain strongly opposed to publishing to maven central without full
> > > > release process with 2 phase voting.
> > > >
> > > > I can live with putting milestone jars elsewhere like
> > > > nightlies.apache.org. We used to have a CI job that published jars
> to
> > > > nightlies.apache.org. This can easily be dusted down and repurposed.
> > > > My vague recollection was that the jars were published in a Maven
> repo
> > > > compatible data structure so that they were easily consumable in
> > > > mvn/gradle/sbt builds.
> > > >
> > > > On Sun, 6 Aug 2023 at 14:18, Matthew de Detrich
> > > > <matthew.dedetr...@aiven.io.invalid> wrote:
> > > > >
> > > > > > And we can release voted on milestones.
> > > > >
> > > > > We can but for reasons stated earlier this to me seems like a
> bandaid
> > > > > and as you pointed out in [1] we are having issues currently with
> > getting
> > > > > enough votes for releases. Hence my biggest complaint here is that
> > > > > requiring votes sends the wrong signals, after all one of the main
> > points
> > > > > of my suggested use of milestones is to test published milestones
> in
> > > > > downstream modules to confirm that they are suitable for general
> > public
> > > > > consumption which is the point that Justin raised earlier.
> > > > >
> > > > > > In the meantime, we have:
> > > > >
> > > > > >
> > > > >
> > > >
> >
> https://cwiki.apache.org/confluence/display/PEKKO/Testing+with+Pekko+Snapshot+Jars
> > > > >
> > > > > > Which is not ideal but it does provide some help.
> > > > >
> > > > > While this is (for now) fine at least for one case (i.e.
> > users/developers
> > > > > testing quick changes which is intended use for snapshots) it
> doesn't
> > > > > practically alleviate the other case regarding testing downstream
> > pekko
> > > > > modules with more recent upstream pekko changes.
> > > > >
> > > > > Using snapshots for this means we have to continuously add/remove
> > > > > the Apache Nexus snapshot repository in the downstream modules
> build
> > > > > files.
> > > > >
> > > > > This point in general actually becoming increasingly more important
> > now
> > > > > considering we are getting significant changes merged upstream in
> the
> > > > > Pekko 1.1.x series and our current approach of only bringing in the
> > > > > changes when a release candidate is being voted on means that
> > > > > we will be bringing in ~6-12 months of changes all at once at an
> > > > > inopportune time (i.e. a release is being made)
> > > > >
> > > > >
> > > > > [1]
> https://lists.apache.org/thread/b57nmyrg5xhgdv3gnbq2o9s6wsqf31q6
> > > > >
> > > > > On Sun, Aug 6, 2023 at 2:57 PM PJ Fanning <fannin...@apache.org>
> > wrote:
> > > > >
> > > > > > And we can release voted on milestones.
> > > > > >
> > > > > > In the meantime, we have:
> > > > > >
> > > > > >
> > > > > >
> > > >
> >
> https://cwiki.apache.org/confluence/display/PEKKO/Testing+with+Pekko+Snapshot+Jars
> > > > > >
> > > > > > Which is not ideal but it does provide some help.
> > > > > >
> > > > > > On Sun, 6 Aug 2023 at 13:38, Matthew de Detrich
> > > > > > <matthew.dedetr...@aiven.io.invalid> wrote:
> > > > > > >
> > > > > > > > Nobody is threatening to delete our recent snapshots. There
> is
> > a
> > > > > > > question about what to do about older snapshots - which is
> > separate
> > > > > > > from this topic.
> > > > > > >
> > > > > > > I am not saying that people will be deleting our snapshots, but
> > > > rather
> > > > > > > there is a current problem with them being pruned (or more
> > correctly
> > > > > > > not being pruned as expected) which needs to be resolved.
> > > > > > >
> > > > > > > > Anyone who wants to test the latest pekko 1.1.0 preview can
> > track
> > > > down
> > > > > > > versions at
> > > > > > > The core point is while its possible for users to manually
> track
> > > > > > snapshots
> > > > > > > and update them as new ones get released/old ones get deleted
> > this
> > > > is not
> > > > > > > practical for the use case I mentioned earlier where we want to
> > set
> > > > the
> > > > > > > downstream Pekko modules versions to a more frequently built
> non
> > > > release
> > > > > > > version so we can catch regressions and/or make sure that newly
> > > > merged
> > > > > > > features work as expected.
> > > > > > >
> > > > > > > If we use snapshots for this then we have the overhead of
> having
> > to
> > > > > > > manually update snapshots versions when builds break due to
> > > > > > > snapshots not existing anymore (note that this hasn't been a
> > problem
> > > > for
> > > > > > now
> > > > > > > because the snapshot pruning is not working correctly which is
> > the
> > > > > > problem
> > > > > > > I was referring to before).
> > > > > > >
> > > > > > > Furthermore if developers want to test newly merged features
> into
> > > > > > > development/staging/prod systems to see that there aren't
> > > > regressions,
> > > > > > > assuming the snapshots are working correctly (which they
> > currently
> > > > > > aren't)
> > > > > > > they can also hit the same issues regarding the snapshots being
> > > > pruned.
> > > > > > >
> > > > > > > At least to me, this is one of the core problems that the
> > milestone
> > > > > > > concept is intended to solve, snapshots are too
> > frequent/granular and
> > > > > > > releases/release candidates are too infrequent.
> > > > > > >
> > > > > > > On Sun, Aug 6, 2023 at 2:23 PM PJ Fanning <
> fannin...@apache.org>
> > > > wrote:
> > > > > > >
> > > > > > > > Nobody is threatening to delete our recent snapshots. There
> is
> > a
> > > > > > > > question about what to do about older snapshots - which is
> > separate
> > > > > > > > from this topic.
> > > > > > > >
> > > > > > > > Anyone who wants to test the latest pekko 1.1.0 preview can
> > track
> > > > down
> > > > > > > > versions at
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > >
> > > >
> >
> https://repository.apache.org/content/groups/snapshots/org/apache/pekko/pekko-actor-typed_2.13/
> > > > > > > >
> > > > > > > > 1.1.0-M0+5-964dcf53-SNAPSHOT is the most recent - but there
> > will be
> > > > > > > > new snapshots published most nights (any day that has commits
> > to
> > > > the
> > > > > > > > main branch).
> > > > > > > >
> > > > > > > > On Sun, 6 Aug 2023 at 13:08, Matthew de Detrich
> > > > > > > > <matthew.dedetr...@aiven.io.invalid> wrote:
> > > > > > > > >
> > > > > > > > > > The question to ask is, will these artefacts be used by
> > users
> > > > or
> > > > > > > > intended
> > > > > > > > > for their use? i.e. people who are not subscribed to the
> > mailing
> > > > > > list or
> > > > > > > > > not committers or PMC members? If so, then by ASF
> definition,
> > > > that
> > > > > > > > artefact
> > > > > > > > > is a release and needs to be voted on by the PPMC/IPMC.
> > > > > > > > >
> > > > > > > > > No they aren't, as I said they are treated in the exact
> same
> > way
> > > > > > > > snapshots
> > > > > > > > > are currently treated. The **ONLY** difference is that they
> > will
> > > > be
> > > > > > > > > distributed to the Apache Nexus main repository and this is
> > > > mainly
> > > > > > due to
> > > > > > > > > technical reasons stated before (i.e. snapshots are
> > > > semi-permanent
> > > > > > since
> > > > > > > > > they need to be pruned).
> > > > > > > > >
> > > > > > > > > The primary motivation for the milestone artifacts are
> > > > > > > > >
> > > > > > > > > * Internal testing between the Pekko dependencies (i.e.
> > > > publishing an
> > > > > > > > > artifact of Pekko core and using that artifact in in other
> > Pekko
> > > > > > modules
> > > > > > > > to
> > > > > > > > > catch regressions before a release is marked)
> > > > > > > > > * For Pekko developers to have the ability to test their
> > features
> > > > > > which
> > > > > > > > > were merged into main in their production systems without
> > having
> > > > to
> > > > > > > > > manually build Pekko (and possibly all of Pekko's
> > dependencies)
> > > > > > > > themselves,
> > > > > > > > > which also as stated previously is quite difficult
> > > > > > > > >
> > > > > > > > > There is no intention of the milestones having any formal
> > release
> > > > > > > > > announcement.
> > > > > > > > >
> > > > > > > > > > Have other incubating projects made non-ASF releases
> while
> > in
> > > > > > > > incubation?
> > > > > > > > > In a small number of cases, yes, mostly while they were
> > getting
> > > > their
> > > > > > > > code
> > > > > > > > > base in order but not after they had made an ASF release,
> and
> > > > there
> > > > > > we
> > > > > > > > very
> > > > > > > > > clearly labelled as non-ASF releases.
> > > > > > > > >
> > > > > > > > > Yes and this is not meant to be a formal ASF release, hence
> > the
> > > > > > previous
> > > > > > > > > suggestion of making this clear even in the DISCLAIMER file
> > > > > > > > >
> > > > > > > > > > Re official documentation/process that describes this
> > > > distinction?
> > > > > > Yes,
> > > > > > > > > that policy page sets that out. See, for instance [1] and
> > [2] for
> > > > > > why it
> > > > > > > > > needs to be this way.
> > > > > > > > >
> > > > > > > > > [1] Is talking about source packages, this discussion is
> > about
> > > > binary
> > > > > > > > > artifacts. In the last line they mention this
> > > > > > > > >
> > > > > > > > > > Nightly Builds that are not release candidates can be
> > hosted at
> > > > > > > > > ci.apache.org projects area, just file an INFRA ticket.
> > > > > > > > >
> > > > > > > > > The links to ci.apache.org aren't even working and I doubt
> > we
> > > > can
> > > > > > even
> > > > > > > > use
> > > > > > > > > such a repository since we are dealing with JVM jar's
> > > > specifically
> > > > > > > > >
> > > > > > > > > > Re official documentation/process that describes this
> > > > distinction?
> > > > > > Yes,
> > > > > > > > > that policy page sets that out. See, for instance [1] and
> > [2] for
> > > > > > why it
> > > > > > > > > needs to be this way. The Incubator distribution guideline
> > also
> > > > > > covers
> > > > > > > > this
> > > > > > > > > [3]. You see that at [4] "Release candidates, nightlys and
> > > > snapshots
> > > > > > must
> > > > > > > > > not be advertised to the general public.” and you can read
> > [5]
> > > > for
> > > > > > why.
> > > > > > > > The
> > > > > > > > > release distribution policy also touches on this e.g. [6]
> > > > > > > > >
> > > > > > > > > As was clarified earlier, milestones are **NOT** releases,
> > they
> > > > are
> > > > > > > > treated
> > > > > > > > > the exact same way as snapshots, nightlies or release
> > candidates
> > > > > > with the
> > > > > > > > > **ONLY** critical difference being that they are deployed
> in
> > a
> > > > > > repository
> > > > > > > > > that is not snapshots and they are published less
> frequently
> > > > then a
> > > > > > > > > snapshot/nightly is.
> > > > > > > > >
> > > > > > > > > With this being said I am increasingly of the opinion that
> > this
> > > > is
> > > > > > more
> > > > > > > > of
> > > > > > > > > a technical INFRA question than an ASF policy question
> since
> > the
> > > > > > issues
> > > > > > > > > being discussed are technical in nature. There was never a
> > > > > > suggestion of
> > > > > > > > > making an alternative formal type of ASF release.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > On Sun, Aug 6, 2023 at 10:51 AM Justin Mclean <
> > > > > > jus...@classsoftware.com>
> > > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > Hi,
> > > > > > > > > >
> > > > > > > > > > The question to ask is, will these artefacts be used by
> > users
> > > > or
> > > > > > > > intended
> > > > > > > > > > for their use? i.e. people who are not subscribed to the
> > > > mailing
> > > > > > list
> > > > > > > > or
> > > > > > > > > > not committers or PMC members? If so, then by ASF
> > definition,
> > > > that
> > > > > > > > artefact
> > > > > > > > > > is a release and needs to be voted on by the PPMC/IPMC.
> > > > > > > > > >
> > > > > > > > > > Have other incubating projects made non-ASF releases
> while
> > in
> > > > > > > > incubation?
> > > > > > > > > > In a small number of cases, yes, mostly while they were
> > getting
> > > > > > their
> > > > > > > > code
> > > > > > > > > > base in order but not after they had made an ASF release,
> > and
> > > > > > there we
> > > > > > > > very
> > > > > > > > > > clearly labelled as non-ASF releases.
> > > > > > > > > >
> > > > > > > > > > Re official documentation/process that describes this
> > > > distinction?
> > > > > > Yes,
> > > > > > > > > > that policy page sets that out. See, for instance [1] and
> > [2]
> > > > for
> > > > > > why
> > > > > > > > it
> > > > > > > > > > needs to be this way. The Incubator distribution
> guideline
> > also
> > > > > > covers
> > > > > > > > this
> > > > > > > > > > [3]. You see that at [4] "Release candidates, nightlys
> and
> > > > > > snapshots
> > > > > > > > must
> > > > > > > > > > not be advertised to the general public.” and you can
> read
> > [5]
> > > > for
> > > > > > > > why. The
> > > > > > > > > > release distribution policy also touches on this e.g. [6]
> > > > > > > > > >
> > > > > > > > > > Kind Regards,
> > > > > > > > > > Justin
> > > > > > > > > >
> > > > > > > > > > 1.
> > https://www.apache.org/legal/release-policy.html#host-rc
> > > > > > > > > > 2. https://www.apache.org/legal/release-policy.html#why
> > > > > > > > > > 3.
> > > > > > > > > >
> > > > > > > >
> > > > > >
> > > >
> >
> https://cwiki.apache.org/confluence/display/INCUBATOR/Distribution+Guidelines
> > > > > > > > > > 4.
> > > > > > > >
> > > > > >
> > > >
> > https://incubator.apache.org/guides/distribution.html#release_platforms
> > > > > > > > > > 5.
> > > > > > https://incubator.apache.org/guides/distribution.html#motivation
> > > > > > > > > > 6.
> > https://infra.apache.org/release-distribution.html#maven
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > >
> > ---------------------------------------------------------------------
> > > > > > > > > > To unsubscribe, e-mail: dev-unsubscr...@pekko.apache.org
> > > > > > > > > > For additional commands, e-mail:
> dev-h...@pekko.apache.org
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > > --
> > > > > > > > >
> > > > > > > > > Matthew de Detrich
> > > > > > > > >
> > > > > > > > > *Aiven Deutschland GmbH*
> > > > > > > > >
> > > > > > > > > Immanuelkirchstraße 26, 10405 Berlin
> > > > > > > > >
> > > > > > > > > Amtsgericht Charlottenburg, HRB 209739 B
> > > > > > > > >
> > > > > > > > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
> > > > > > > > >
> > > > > > > > > *m:* +491603708037
> > > > > > > > >
> > > > > > > > > *w:* aiven.io *e:* matthew.dedetr...@aiven.io
> > > > > > > >
> > > > > > > >
> > > > ---------------------------------------------------------------------
> > > > > > > > To unsubscribe, e-mail: dev-unsubscr...@pekko.apache.org
> > > > > > > > For additional commands, e-mail: dev-h...@pekko.apache.org
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > >
> > > > > > > Matthew de Detrich
> > > > > > >
> > > > > > > *Aiven Deutschland GmbH*
> > > > > > >
> > > > > > > Immanuelkirchstraße 26, 10405 Berlin
> > > > > > >
> > > > > > > Amtsgericht Charlottenburg, HRB 209739 B
> > > > > > >
> > > > > > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
> > > > > > >
> > > > > > > *m:* +491603708037
> > > > > > >
> > > > > > > *w:* aiven.io *e:* matthew.dedetr...@aiven.io
> > > > > >
> > > > > >
> > ---------------------------------------------------------------------
> > > > > > To unsubscribe, e-mail: dev-unsubscr...@pekko.apache.org
> > > > > > For additional commands, e-mail: dev-h...@pekko.apache.org
> > > > > >
> > > > > >
> > > > >
> > > > > --
> > > > >
> > > > > Matthew de Detrich
> > > > >
> > > > > *Aiven Deutschland GmbH*
> > > > >
> > > > > Immanuelkirchstraße 26, 10405 Berlin
> > > > >
> > > > > Amtsgericht Charlottenburg, HRB 209739 B
> > > > >
> > > > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
> > > > >
> > > > > *m:* +491603708037
> > > > >
> > > > > *w:* aiven.io *e:* matthew.dedetr...@aiven.io
> > > >
> > > > ---------------------------------------------------------------------
> > > > To unsubscribe, e-mail: dev-unsubscr...@pekko.apache.org
> > > > For additional commands, e-mail: dev-h...@pekko.apache.org
> > > >
> > > >
> > >
> > > --
> > >
> > > Matthew de Detrich
> > >
> > > *Aiven Deutschland GmbH*
> > >
> > > Immanuelkirchstraße 26, 10405 Berlin
> > >
> > > Amtsgericht Charlottenburg, HRB 209739 B
> > >
> > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
> > >
> > > *m:* +491603708037
> > >
> > > *w:* aiven.io *e:* matthew.dedetr...@aiven.io
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscr...@pekko.apache.org
> > For additional commands, e-mail: dev-h...@pekko.apache.org
> >
> >
>
> --
>
> Matthew de Detrich
>
> *Aiven Deutschland GmbH*
>
> Immanuelkirchstraße 26, 10405 Berlin
>
> Amtsgericht Charlottenburg, HRB 209739 B
>
> Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
>
> *m:* +491603708037
>
> *w:* aiven.io *e:* matthew.dedetr...@aiven.io
>
