> I don't think we can have full releases that depend on milestone releases of transitive dependencies.
Technically speaking process wise there is nothing wrong with this (it doesn't break anything ASF, ASF does not care about the distinction between milestone or non milestone) although it obviously will raise some eyebrows because of what it semantically implies. > Essentially, we are blocked from doing 1.1 releases of anything until Pekko 1.1.0 is released (other than milestone releases). > I think we should have a full discussion and vote about the idea of changing all of our builds to use Pekko 1.1.0(-M1). It was always my view that I didn't want to have to do months of releases of all the modules after Pekko 1.1 was released But we are already in this situation (albeit not fully) anyways. For example pekko-project will require pekko-persistence-jdbc 1.1.x because of a breaking update from Slick 3.3 to Slick 3.5 in pekko-persistence-jdbc (which pekko-projection has a dependency on). Transitively this will also mean pekko-connectors should also depend on pekko core 1.1.x series because it also has slick integration which got updated to 3.5 We will also need to do this kind of testing matrix due to differences in Pekko Core i.e. see logback 1 vs 2 (which means we now have different versions of pekko-testkit) and there are other cases such as Jackson. Even with pekko grpc, if I understand correctly i.e. from your original OP > With the 2 gRPC based Pekko Connectors that we maintain, we are already blocked from upgrading some Google libs - because they are incompatible with the Pekko gRPC jars. Pekko Connectors will have to depend on pekko-grpc 1.1.x There is also probably more stuff we are missing, i.e. we also wanted to do cross JDK testing[1] whose solution is the same as what we are talking about (even if the problem is different). The takeway I am trying to make here is that we have to deal with this even when completely ignoring the inliner changes since we have enough of these fundamental incompatibilities (often, but not always due to updated dependencies in the 1.1.x series not being compatible with those same dependencies 1.0.x series). Ontop of that there is an argument that with such fundamental big version changes in these dependencies (which gRPC is one of, it seems) that it warrants an -M1 (again completely ignoring inliner). Ontop of that, another point to boot is that pekko-connectors is likely not going to have a non milestone release a while[2] which means that it will be stuck in milestone (-M1, -M2 etc etc) for that time and I don't think pekko-connectors-M1 depending on pekko-grpc-M1 is going to be controversial. Also finally, our bottleneck when making Pekko Release has historically been the incubator itself, the PPMC on the other hand has almost always made the vote within 3 days (I think there were 2-3 exceptions to that). 1: https://github.com/apache/incubator-pekko/issues/1118 2: https://github.com/apache/incubator-pekko-connectors/discussions/443 On Tue, Mar 19, 2024 at 1:24 PM PJ Fanning <fannin...@apache.org> wrote: > > If we go the route of releasing pekko grpc 1.1.x by building with > pekko 1.1.0-M1 then the grpc release will need to be a milestone too. > I don't think we can have full releases that depend on milestone > releases of transitive dependencies. Essentially, we are blocked from > doing 1.1 releases of anything until Pekko 1.1.0 is released (other > than milestone releases). > > I think we should have a full discussion and vote about the idea of > changing all of our builds to use Pekko 1.1.0(-M1). It was always my > view that I didn't want to have to do months of releases of all the > modules after Pekko 1.1 was released. I think we are going to get a > lot of frustrated users wondering why they are getting version > conflicts when they inadvertently end up with a mix of pekko > dependencies, some 1.0, some 1.1. > > I, for one, am -1 on going down this route. > > > On Tue, 19 Mar 2024 at 13:15, Matthew de Detrich > <matthew.dedetr...@aiven.io.invalid> wrote: > > > > > pekko 1.1.0 to run pekko (which is what > > will appear in the pom)* > > > > pekko 1.1.0 to run pekko-grpc (which is what > > will appear in the pom) > > > > On Tue, Mar 19, 2024 at 1:08 PM Matthew de Detrich > > <matthew.dedetr...@aiven.io> wrote: > > > > > > > I disagree. I want to see a full description of what the new release > > > pipeline looks like. I want to know exactly what appears in our poms. > > > > > > I really don't see what's problematic about setting the pekko core > > > dependency > > > as 1.1.0[1]. This doesn't mean you need pekko 1.1.0 to run pekko (which > > > is what > > > will appear in the pom), it's just what gets resolved by default and > > > what it's built with. > > > > > > Akka-Http has done this multiple times, they bumped the core Akka version > > > within > > > Akka-Http for various reasons. > > > > > > > I stayed out of the discussion because I thought it had a low impact. > > > Now it is blocking our ability to do releases. Up until today, there > > > was no indication that Pekko 1.1 was going to require major changes to > > > our builds and release pipelines. > > > > > > This is untrue, it was discussed multiple times which is not the same > > > as it not being > > > noticed. I just spent 5 minutes looking for the first instance and tbh I > > > am not > > > exactly motivated to trawl through all of the discussions to find all > > > instances of this > > > as its not a productive use of time > > > > > > 1: > > > https://github.com/apache/incubator-pekko-grpc/blob/792f886381d5a5539901c2da4f99917b57389209/project/PekkoCoreDependency.scala#L25 > > > > > > On Tue, Mar 19, 2024 at 12:59 PM PJ Fanning <fannin...@gmail.com> wrote: > > > > > > > > I disagree. I want to see a full description of what the new release > > > > pipeline looks like. I want to know exactly what appears in our poms. > > > > > > > > Of all the features in the Pekko 1.1.0-M1, the scala 2 inliner is of > > > > the least interest to me. Johannes Rudolph has said on a number of > > > > times that Akka team did not prioritise it because the Hotspot > > > > compiler would be expected to make most of the optimisations anyway. > > > > > > > > I stayed out of the discussion because I thought it had a low impact. > > > > Now it is blocking our ability to do releases. Up until today, there > > > > was no indication that Pekko 1.1 was going to require major changes to > > > > our builds and release pipelines. > > > > > > > > > > > > > > > > On Tue, 19 Mar 2024 at 12:53, Matthew de Detrich > > > > <matthew.dedetr...@aiven.io.invalid> wrote: > > > > > > > > > > > I agree, no release of a module should depend on an unreleased > > > > > > version of > > > > > Pekko. > > > > > > > > > > This is incorrect, -M1 is technically an ASF release. Having > > > > > pekko-grpc rely > > > > > on a non released version of Pekko is not what is being discussed and > > > > > never was on the table. > > > > > > > > > > > This Pekko 1.1 build set up has never been agreed on. I'm against it > > > > > due to its complexity and unproven benefits. > > > > > > > > > > This is the same build setup that is required to cross test pekko > > > > > against > > > > > multiple pekko versions which was also discussed in multiple other > > > > > places. > > > > > > > > > > I have a suspicion that you have an inaccurate impression of whats > > > > > being > > > > > discussed, its really not that complex. The minimum build version > > > > > will be set > > > > > to Pekko 1.1.0 or Pekko 1.1.0-M1 and another pipeline will be added > > > > > to run > > > > > it against Pekko 1.0.x which we were planning to do anyways. > > > > > > > > > > On Tue, Mar 19, 2024 at 12:46 PM Nicolas Vollmar <nvoll...@gmail.com> > > > > > wrote: > > > > > > > > > > > > I agree, no release of a module should depend on an unreleased > > > > > > version of > > > > > > Pekko. > > > > > > People would end up with unintended milestones in their class path. > > > > > > > > > > > > If gRPC 1.1 needs Pekko 1.1 we should push for that release first. > > > > > > > > > > > > On Tue, 19 Mar 2024 at 12:01, PJ Fanning <fannin...@gmail.com> > > > > > > wrote: > > > > > > > > > > > > > This Pekko 1.1 build set up has never been agreed on. I'm against > > > > > > > it > > > > > > > due to its complexity and unproven benefits. > > > > > > > > > > > > > > On Tue, 19 Mar 2024 at 11:54, Matthew de Detrich > > > > > > > <matthew.dedetr...@aiven.io.invalid> wrote: > > > > > > > > > > > > > > > > This was discussed previously multiple times, in order for the > > > > > > > > inlining to be fully > > > > > > > > enabled it needs to be built with Pekko 1.1.0-M1 because that's > > > > > > > > the > > > > > > > > version of Pekko > > > > > > > > with the fixed inlining. This does not mean that a pekko gRPC > > > > > > > > built with Pekko 1.1.0-M1 won't work with Pekko 1.0.x, it just > > > > > > > > means it > > > > > > > needs to > > > > > > > > be built with it. > > > > > > > > > > > > > > > > Note that this also implies we need to add a test to make sure > > > > > > > > a pekko > > > > > > > > gRPC built > > > > > > > > with Pekko 1.1.0-M1 works when linked with Pekko 1.0.x at > > > > > > > > runtime > > > > > > > > > > > > > > > > On Tue, Mar 19, 2024 at 10:36 AM PJ Fanning > > > > > > > > <fannin...@apache.org> > > > > > > > wrote: > > > > > > > > > > > > > > > > > > I don't agree that we want to build Pekko gRPC 1.1 off of > > > > > > > > > Pekko > > > > > > > > > 1.1.0-M1. Pekko 1.1.0-M1 is > > > > > > > > > * a milestone - not a proper release > > > > > > > > > * I thought we were trying to maintain it so that our modules > > > > > > > > > would be > > > > > > > > > compatible with Pekko 1.0 > > > > > > > > > > > > > > > > > > I don't see the need for a milestone release of Pekko gRPC. > > > > > > > > > There is > > > > > > > > > nothing experimental in the main branch. The only reason to > > > > > > > > > make this > > > > > > > > > a 1.1 release instead of 1.0.3 release is that the changes in > > > > > > > > > our > > > > > > > > > Protobuf dependencies are not trivial. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Tue, 19 Mar 2024 at 07:12, Matthew de Detrich > > > > > > > > > <matthew.dedetr...@aiven.io.invalid> wrote: > > > > > > > > > > > > > > > > > > > > I would wait till Pekko Core 1.1.0-M1 release so that gRPC > > > > > > > > > > 1.1.0 can > > > > > > > > > > build off of Pekko Core 1.1.0-M1 > > > > > > > > > > as this will enable the full cross inlining[1]. > > > > > > > > > > > > > > > > > > > > Also not sure if we want to do -M1 first before full > > > > > > > > > > release (or > > > > > > > not). > > > > > > > > > > The reasons here are weaker than > > > > > > > > > > other projects but I may be missing something. > > > > > > > > > > > > > > > > > > > > 1: > > > > > > > https://github.com/pjfanning/sbt-pekko-build/blob/main/src/main/scala/com/github/pjfanning/pekkobuild/PekkoInlinePlugin.scala#L49-L56 > > > > > > > > > > > > > > > > > > > > On Mon, Mar 18, 2024 at 9:26 PM PJ Fanning > > > > > > > > > > <fannin...@apache.org> > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > Hi everyone, > > > > > > > > > > > > > > > > > > > > > > The gRPC code is heavily dependent on protobuf and there > > > > > > > > > > > are > > > > > > > upgrades > > > > > > > > > > > to grpc-protobuf and protoc in the main branch. With the > > > > > > > > > > > 2 gRPC > > > > > > > based > > > > > > > > > > > Pekko Connectors that we maintain, we are already blocked > > > > > > > > > > > from > > > > > > > > > > > upgrading some Google libs - because they are > > > > > > > > > > > incompatible with > > > > > > > the > > > > > > > > > > > Pekko gRPC jars. These connectors will compile with the > > > > > > > > > > > latest > > > > > > > Google > > > > > > > > > > > jars if Pekko gRPC 1.1 snapshots are used. > > > > > > > > > > > > > > > > > > > > > > These projects demo this: > > > > > > > > > > > * > > > > > > > https://github.com/pjfanning/pekko-connectors-google-cloud-pub-sub-grpc/ > > > > > > > > > > > * > > > > > > > https://github.com/pjfanning/pekko-connectors-google-cloud-bigquery-storage > > > > > > > > > > > > > > > > > > > > > > There has recently been a new protobuf-java 4.0.26 > > > > > > > > > > > release but the > > > > > > > > > > > gRPC and other libs have been changed to support this > > > > > > > > > > > incompatible > > > > > > > > > > > version. I suspect that we might need a Pekko gRPC > > > > > > > > > > > release for > > > > > > > these > > > > > > > > > > > new protobuf-java 4.0,x releases at some point also. > > > > > > > > > > > > > > > > > > > > > > Does anyone have any thoughts about this? > > > > > > > > > > > > > > > > > > > > > > Regards, > > > > > > > > > > > PJ > > > > > > > > > > > > > > > > > > > > > > > > > > > > > --------------------------------------------------------------------- > > > > > > > > > > > To unsubscribe, e-mail: dev-unsubscr...@pekko.apache.org > > > > > > > > > > > For additional commands, e-mail: dev-h...@pekko.apache.org > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > > > > > > > > > > > Matthew de Detrich > > > > > > > > > > > > > > > > > > > > Aiven Deutschland GmbH > > > > > > > > > > > > > > > > > > > > Immanuelkirchstraße 26, 10405 Berlin > > > > > > > > > > > > > > > > > > > > Alexanderufer 3-7, 10117 Berlin > > > > > > > > > > > > > > > > > > > > Amtsgericht Charlottenburg, HRB 209739 B > > > > > > > > > > > > > > > > > > > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen > > > > > > > > > > > > > > > > > > > > m: +491603708037 > > > > > > > > > > > > > > > > > > > > w: aiven.io e: matthew.dedetr...@aiven.io > > > > > > > > > > > > > > > > > > > > --------------------------------------------------------------------- > > > > > > > > > > To unsubscribe, e-mail: dev-unsubscr...@pekko.apache.org > > > > > > > > > > For additional commands, e-mail: dev-h...@pekko.apache.org > > > > > > > > > > > > > > > > > > > > > > > > > > > > --------------------------------------------------------------------- > > > > > > > > > To unsubscribe, e-mail: dev-unsubscr...@pekko.apache.org > > > > > > > > > For additional commands, e-mail: dev-h...@pekko.apache.org > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > > > > > > > Matthew de Detrich > > > > > > > > > > > > > > > > Aiven Deutschland GmbH > > > > > > > > > > > > > > > > Immanuelkirchstraße 26, 10405 Berlin > > > > > > > > > > > > > > > > Alexanderufer 3-7, 10117 Berlin > > > > > > > > > > > > > > > > Amtsgericht Charlottenburg, HRB 209739 B > > > > > > > > > > > > > > > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen > > > > > > > > > > > > > > > > m: +491603708037 > > > > > > > > > > > > > > > > w: aiven.io e: matthew.dedetr...@aiven.io > > > > > > > > > > > > > > > > --------------------------------------------------------------------- > > > > > > > > To unsubscribe, e-mail: dev-unsubscr...@pekko.apache.org > > > > > > > > For additional commands, e-mail: dev-h...@pekko.apache.org > > > > > > > > > > > > > > > > > > > > > > --------------------------------------------------------------------- > > > > > > > To unsubscribe, e-mail: dev-unsubscr...@pekko.apache.org > > > > > > > For additional commands, e-mail: dev-h...@pekko.apache.org > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > Matthew de Detrich > > > > > > > > > > Aiven Deutschland GmbH > > > > > > > > > > Immanuelkirchstraße 26, 10405 Berlin > > > > > > > > > > Alexanderufer 3-7, 10117 Berlin > > > > > > > > > > Amtsgericht Charlottenburg, HRB 209739 B > > > > > > > > > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen > > > > > > > > > > m: +491603708037 > > > > > > > > > > w: aiven.io e: matthew.dedetr...@aiven.io > > > > > > > > > > --------------------------------------------------------------------- > > > > > To unsubscribe, e-mail: dev-unsubscr...@pekko.apache.org > > > > > For additional commands, e-mail: dev-h...@pekko.apache.org > > > > > > > > > > > > > --------------------------------------------------------------------- > > > > To unsubscribe, e-mail: dev-unsubscr...@pekko.apache.org > > > > For additional commands, e-mail: dev-h...@pekko.apache.org > > > > > > > > > > > > > -- > > > > > > Matthew de Detrich > > > > > > Aiven Deutschland GmbH > > > > > > Immanuelkirchstraße 26, 10405 Berlin > > > > > > Alexanderufer 3-7, 10117 Berlin > > > > > > Amtsgericht Charlottenburg, HRB 209739 B > > > > > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen > > > > > > m: +491603708037 > > > > > > w: aiven.io e: matthew.dedetr...@aiven.io > > > > > > > > -- > > > > Matthew de Detrich > > > > Aiven Deutschland GmbH > > > > Immanuelkirchstraße 26, 10405 Berlin > > > > Alexanderufer 3-7, 10117 Berlin > > > > Amtsgericht Charlottenburg, HRB 209739 B > > > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen > > > > m: +491603708037 > > > > w: aiven.io e: matthew.dedetr...@aiven.io > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: dev-unsubscr...@pekko.apache.org > > For additional commands, e-mail: dev-h...@pekko.apache.org > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@pekko.apache.org > For additional commands, e-mail: dev-h...@pekko.apache.org > -- Matthew de Detrich Aiven Deutschland GmbH Immanuelkirchstraße 26, 10405 Berlin Alexanderufer 3-7, 10117 Berlin Amtsgericht Charlottenburg, HRB 209739 B Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen m: +491603708037 w: aiven.io e: matthew.dedetr...@aiven.io --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@pekko.apache.org For additional commands, e-mail: dev-h...@pekko.apache.org
