> I didn't follow this closely, but does this mean pekko-persistence-jdbc does not adhere to https://pekko.apache.org/docs/pekko/current/common/binary-compatibility-rules.html ?
Yes because it can't, the changes from Slick 3.3.x to Slick 3.5.x was so extensive that it completely broke binary compatibility rules. And by that I mean the upgrade to Slick 3.5.x forced us to rewrite some code that technically broke MiMa and also the upgrade itself caused MiMa to complain even in cases where the source code was not changed. On Mon, Mar 25, 2024 at 2:24 PM Arnout Engelen <enge...@apache.org> wrote: > > > For example pekko-project will require pekko-persistence-jdbc 1.1.x > because of a breaking update from Slick 3.3 to Slick 3.5 in > pekko-persistence-jdbc > (which pekko-projection has a dependency on). > > I didn't follow this closely, but does this mean pekko-persistence-jdbc > does not adhere to > https://pekko.apache.org/docs/pekko/current/common/binary-compatibility-rules.html > ? > > Traditionally we've always been rather strict about binary compatibility in > Akka. This has been very valuable (indeed there was a lot of flexibility of > upgrading libraries in various orders), but it was also challenging > sometimes. I think for Pekko it would make sense to move a little faster, > and expect users to "just upgrade" in some cases. We should document > clearly what we do and do not promise, though. > > I would also be OK with being more liberal with major version numbers, > especially in the 'satellite' projects: should the Slick 3.5 upgrade > perhaps mean pekko-persistence-jdbc 2.0.0? > > > Kind regards, > > Arnout > > On Tue, Mar 19, 2024 at 1:51 PM Matthew de Detrich > <matthew.dedetr...@aiven.io.invalid> wrote: > > > > I don't think we can have full releases that depend on milestone > > releases of transitive dependencies. > > > > Technically speaking process wise there is nothing wrong with this (it > > doesn't break anything ASF, ASF does not care about the > > distinction between milestone or non milestone) although it obviously > > will raise some eyebrows because of what it semantically implies. > > > > > Essentially, we are blocked from > > doing 1.1 releases of anything until Pekko 1.1.0 is released (other > > than milestone releases). > > > > > I think we should have a full discussion and vote about the idea of > > changing all of our builds to use Pekko 1.1.0(-M1). It was always my > > view that I didn't want to have to do months of releases of all the > > modules after Pekko 1.1 was released > > > > But we are already in this situation (albeit not fully) anyways. For > > example > > pekko-project will require pekko-persistence-jdbc 1.1.x because > > of a breaking update from Slick 3.3 to Slick 3.5 in > > pekko-persistence-jdbc (which > > pekko-projection has a dependency on). Transitively this will also mean > > pekko-connectors should also depend on pekko core 1.1.x series because it > > also has slick integration which got updated to 3.5 > > > > We will also need to do this kind of testing matrix due to differences > > in Pekko Core > > i.e. see logback 1 vs 2 (which means we now have different versions of > > pekko-testkit) > > and there are other cases such as Jackson. > > > > Even with pekko grpc, if I understand correctly i.e. from your original OP > > > > > With the 2 gRPC based > > Pekko Connectors that we maintain, we are already blocked from > > upgrading some Google libs - because they are incompatible with the > > Pekko gRPC jars. > > > > Pekko Connectors will have to depend on pekko-grpc 1.1.x There is also > > probably more stuff we are missing, i.e. we also wanted to do cross JDK > > testing[1] whose solution is the same as what we are talking about (even if > > the problem is different). > > > > The takeway I am trying to make here is that we have to deal with this > > even when completely ignoring the inliner changes since we have enough of > > these fundamental incompatibilities (often, but not always due to updated > > dependencies in the 1.1.x series not being compatible with those same > > dependencies 1.0.x series). Ontop of that there is an argument that with > > such fundamental big version changes in these dependencies (which gRPC > > is one of, it seems) that it warrants an -M1 (again completely > > ignoring inliner). > > Ontop of that, another point to boot is that pekko-connectors is likely > > not going to have a non milestone release a while[2] which means that it > > will be stuck in milestone (-M1, -M2 etc etc) for that time and I don't > > think > > pekko-connectors-M1 depending on pekko-grpc-M1 is going to be > > controversial. > > > > Also finally, our bottleneck when making Pekko Release has historically > > been > > the incubator itself, the PPMC on the other hand has almost always made > > the vote within 3 days (I think there were 2-3 exceptions to that). > > > > 1: https://github.com/apache/incubator-pekko/issues/1118 > > 2: https://github.com/apache/incubator-pekko-connectors/discussions/443 > > > > On Tue, Mar 19, 2024 at 1:24 PM PJ Fanning <fannin...@apache.org> wrote: > > > > > > If we go the route of releasing pekko grpc 1.1.x by building with > > > pekko 1.1.0-M1 then the grpc release will need to be a milestone too. > > > I don't think we can have full releases that depend on milestone > > > releases of transitive dependencies. Essentially, we are blocked from > > > doing 1.1 releases of anything until Pekko 1.1.0 is released (other > > > than milestone releases). > > > > > > I think we should have a full discussion and vote about the idea of > > > changing all of our builds to use Pekko 1.1.0(-M1). It was always my > > > view that I didn't want to have to do months of releases of all the > > > modules after Pekko 1.1 was released. I think we are going to get a > > > lot of frustrated users wondering why they are getting version > > > conflicts when they inadvertently end up with a mix of pekko > > > dependencies, some 1.0, some 1.1. > > > > > > I, for one, am -1 on going down this route. > > > > > > > > > On Tue, 19 Mar 2024 at 13:15, Matthew de Detrich > > > <matthew.dedetr...@aiven.io.invalid> wrote: > > > > > > > > > pekko 1.1.0 to run pekko (which is what > > > > will appear in the pom)* > > > > > > > > pekko 1.1.0 to run pekko-grpc (which is what > > > > will appear in the pom) > > > > > > > > On Tue, Mar 19, 2024 at 1:08 PM Matthew de Detrich > > > > <matthew.dedetr...@aiven.io> wrote: > > > > > > > > > > > I disagree. I want to see a full description of what the new > > release > > > > > pipeline looks like. I want to know exactly what appears in our poms. > > > > > > > > > > I really don't see what's problematic about setting the pekko core > > dependency > > > > > as 1.1.0[1]. This doesn't mean you need pekko 1.1.0 to run pekko > > (which is what > > > > > will appear in the pom), it's just what gets resolved by default and > > > > > what it's built with. > > > > > > > > > > Akka-Http has done this multiple times, they bumped the core Akka > > version within > > > > > Akka-Http for various reasons. > > > > > > > > > > > I stayed out of the discussion because I thought it had a low > > impact. > > > > > Now it is blocking our ability to do releases. Up until today, there > > > > > was no indication that Pekko 1.1 was going to require major changes > > to > > > > > our builds and release pipelines. > > > > > > > > > > This is untrue, it was discussed multiple times which is not the same > > > > > as it not being > > > > > noticed. I just spent 5 minutes looking for the first instance and > > tbh I am not > > > > > exactly motivated to trawl through all of the discussions to find all > > > > > instances of this > > > > > as its not a productive use of time > > > > > > > > > > 1: > > https://github.com/apache/incubator-pekko-grpc/blob/792f886381d5a5539901c2da4f99917b57389209/project/PekkoCoreDependency.scala#L25 > > > > > > > > > > On Tue, Mar 19, 2024 at 12:59 PM PJ Fanning <fannin...@gmail.com> > > wrote: > > > > > > > > > > > > I disagree. I want to see a full description of what the new > > release > > > > > > pipeline looks like. I want to know exactly what appears in our > > poms. > > > > > > > > > > > > Of all the features in the Pekko 1.1.0-M1, the scala 2 inliner is > > of > > > > > > the least interest to me. Johannes Rudolph has said on a number of > > > > > > times that Akka team did not prioritise it because the Hotspot > > > > > > compiler would be expected to make most of the optimisations > > anyway. > > > > > > > > > > > > I stayed out of the discussion because I thought it had a low > > impact. > > > > > > Now it is blocking our ability to do releases. Up until today, > > there > > > > > > was no indication that Pekko 1.1 was going to require major > > changes to > > > > > > our builds and release pipelines. > > > > > > > > > > > > > > > > > > > > > > > > On Tue, 19 Mar 2024 at 12:53, Matthew de Detrich > > > > > > <matthew.dedetr...@aiven.io.invalid> wrote: > > > > > > > > > > > > > > > I agree, no release of a module should depend on an unreleased > > version of > > > > > > > Pekko. > > > > > > > > > > > > > > This is incorrect, -M1 is technically an ASF release. Having > > pekko-grpc rely > > > > > > > on a non released version of Pekko is not what is being > > discussed and > > > > > > > never was on the table. > > > > > > > > > > > > > > > This Pekko 1.1 build set up has never been agreed on. I'm > > against it > > > > > > > due to its complexity and unproven benefits. > > > > > > > > > > > > > > This is the same build setup that is required to cross test > > pekko against > > > > > > > multiple pekko versions which was also discussed in multiple > > other places. > > > > > > > > > > > > > > I have a suspicion that you have an inaccurate impression of > > whats being > > > > > > > discussed, its really not that complex. The minimum build > > version will be set > > > > > > > to Pekko 1.1.0 or Pekko 1.1.0-M1 and another pipeline will be > > added to run > > > > > > > it against Pekko 1.0.x which we were planning to do anyways. > > > > > > > > > > > > > > On Tue, Mar 19, 2024 at 12:46 PM Nicolas Vollmar < > > nvoll...@gmail.com> wrote: > > > > > > > > > > > > > > > > I agree, no release of a module should depend on an unreleased > > version of > > > > > > > > Pekko. > > > > > > > > People would end up with unintended milestones in their class > > path. > > > > > > > > > > > > > > > > If gRPC 1.1 needs Pekko 1.1 we should push for that release > > first. > > > > > > > > > > > > > > > > On Tue, 19 Mar 2024 at 12:01, PJ Fanning <fannin...@gmail.com> > > wrote: > > > > > > > > > > > > > > > > > This Pekko 1.1 build set up has never been agreed on. I'm > > against it > > > > > > > > > due to its complexity and unproven benefits. > > > > > > > > > > > > > > > > > > On Tue, 19 Mar 2024 at 11:54, Matthew de Detrich > > > > > > > > > <matthew.dedetr...@aiven.io.invalid> wrote: > > > > > > > > > > > > > > > > > > > > This was discussed previously multiple times, in order for > > the > > > > > > > > > > inlining to be fully > > > > > > > > > > enabled it needs to be built with Pekko 1.1.0-M1 because > > that's the > > > > > > > > > > version of Pekko > > > > > > > > > > with the fixed inlining. This does not mean that a pekko > > gRPC > > > > > > > > > > built with Pekko 1.1.0-M1 won't work with Pekko 1.0.x, it > > just means it > > > > > > > > > needs to > > > > > > > > > > be built with it. > > > > > > > > > > > > > > > > > > > > Note that this also implies we need to add a test to make > > sure a pekko > > > > > > > > > > gRPC built > > > > > > > > > > with Pekko 1.1.0-M1 works when linked with Pekko 1.0.x at > > runtime > > > > > > > > > > > > > > > > > > > > On Tue, Mar 19, 2024 at 10:36 AM PJ Fanning < > > fannin...@apache.org> > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > I don't agree that we want to build Pekko gRPC 1.1 off > > of Pekko > > > > > > > > > > > 1.1.0-M1. Pekko 1.1.0-M1 is > > > > > > > > > > > * a milestone - not a proper release > > > > > > > > > > > * I thought we were trying to maintain it so that our > > modules would be > > > > > > > > > > > compatible with Pekko 1.0 > > > > > > > > > > > > > > > > > > > > > > I don't see the need for a milestone release of Pekko > > gRPC. There is > > > > > > > > > > > nothing experimental in the main branch. The only reason > > to make this > > > > > > > > > > > a 1.1 release instead of 1.0.3 release is that the > > changes in our > > > > > > > > > > > Protobuf dependencies are not trivial. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Tue, 19 Mar 2024 at 07:12, Matthew de Detrich > > > > > > > > > > > <matthew.dedetr...@aiven.io.invalid> wrote: > > > > > > > > > > > > > > > > > > > > > > > > I would wait till Pekko Core 1.1.0-M1 release so that > > gRPC 1.1.0 can > > > > > > > > > > > > build off of Pekko Core 1.1.0-M1 > > > > > > > > > > > > as this will enable the full cross inlining[1]. > > > > > > > > > > > > > > > > > > > > > > > > Also not sure if we want to do -M1 first before full > > release (or > > > > > > > > > not). > > > > > > > > > > > > The reasons here are weaker than > > > > > > > > > > > > other projects but I may be missing something. > > > > > > > > > > > > > > > > > > > > > > > > 1: > > > > > > > > > > > https://github.com/pjfanning/sbt-pekko-build/blob/main/src/main/scala/com/github/pjfanning/pekkobuild/PekkoInlinePlugin.scala#L49-L56 > > > > > > > > > > > > > > > > > > > > > > > > On Mon, Mar 18, 2024 at 9:26 PM PJ Fanning < > > fannin...@apache.org> > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > Hi everyone, > > > > > > > > > > > > > > > > > > > > > > > > > > The gRPC code is heavily dependent on protobuf and > > there are > > > > > > > > > upgrades > > > > > > > > > > > > > to grpc-protobuf and protoc in the main branch. With > > the 2 gRPC > > > > > > > > > based > > > > > > > > > > > > > Pekko Connectors that we maintain, we are already > > blocked from > > > > > > > > > > > > > upgrading some Google libs - because they are > > incompatible with > > > > > > > > > the > > > > > > > > > > > > > Pekko gRPC jars. These connectors will compile with > > the latest > > > > > > > > > Google > > > > > > > > > > > > > jars if Pekko gRPC 1.1 snapshots are used. > > > > > > > > > > > > > > > > > > > > > > > > > > These projects demo this: > > > > > > > > > > > > > * > > > > > > > > > > > https://github.com/pjfanning/pekko-connectors-google-cloud-pub-sub-grpc/ > > > > > > > > > > > > > * > > > > > > > > > > > https://github.com/pjfanning/pekko-connectors-google-cloud-bigquery-storage > > > > > > > > > > > > > > > > > > > > > > > > > > There has recently been a new protobuf-java 4.0.26 > > release but the > > > > > > > > > > > > > gRPC and other libs have been changed to support > > this incompatible > > > > > > > > > > > > > version. I suspect that we might need a Pekko gRPC > > release for > > > > > > > > > these > > > > > > > > > > > > > new protobuf-java 4.0,x releases at some point also. > > > > > > > > > > > > > > > > > > > > > > > > > > Does anyone have any thoughts about this? > > > > > > > > > > > > > > > > > > > > > > > > > > Regards, > > > > > > > > > > > > > PJ > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > --------------------------------------------------------------------- > > > > > > > > > > > > > To unsubscribe, e-mail: > > dev-unsubscr...@pekko.apache.org > > > > > > > > > > > > > For additional commands, e-mail: > > dev-h...@pekko.apache.org > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > > > > > > > > > > > > > > > Matthew de Detrich > > > > > > > > > > > > > > > > > > > > > > > > Aiven Deutschland GmbH > > > > > > > > > > > > > > > > > > > > > > > > Immanuelkirchstraße 26, 10405 Berlin > > > > > > > > > > > > > > > > > > > > > > > > Alexanderufer 3-7, 10117 Berlin > > > > > > > > > > > > > > > > > > > > > > > > Amtsgericht Charlottenburg, HRB 209739 B > > > > > > > > > > > > > > > > > > > > > > > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen > > > > > > > > > > > > > > > > > > > > > > > > m: +491603708037 > > > > > > > > > > > > > > > > > > > > > > > > w: aiven.io e: matthew.dedetr...@aiven.io > > > > > > > > > > > > > > > > > > > > > > > > > > --------------------------------------------------------------------- > > > > > > > > > > > > To unsubscribe, e-mail: > > dev-unsubscr...@pekko.apache.org > > > > > > > > > > > > For additional commands, e-mail: > > dev-h...@pekko.apache.org > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > --------------------------------------------------------------------- > > > > > > > > > > > To unsubscribe, e-mail: dev-unsubscr...@pekko.apache.org > > > > > > > > > > > For additional commands, e-mail: > > dev-h...@pekko.apache.org > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > > > > > > > > > > > Matthew de Detrich > > > > > > > > > > > > > > > > > > > > Aiven Deutschland GmbH > > > > > > > > > > > > > > > > > > > > Immanuelkirchstraße 26, 10405 Berlin > > > > > > > > > > > > > > > > > > > > Alexanderufer 3-7, 10117 Berlin > > > > > > > > > > > > > > > > > > > > Amtsgericht Charlottenburg, HRB 209739 B > > > > > > > > > > > > > > > > > > > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen > > > > > > > > > > > > > > > > > > > > m: +491603708037 > > > > > > > > > > > > > > > > > > > > w: aiven.io e: matthew.dedetr...@aiven.io > > > > > > > > > > > > > > > > > > > > > > --------------------------------------------------------------------- > > > > > > > > > > To unsubscribe, e-mail: dev-unsubscr...@pekko.apache.org > > > > > > > > > > For additional commands, e-mail: dev-h...@pekko.apache.org > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > --------------------------------------------------------------------- > > > > > > > > > To unsubscribe, e-mail: dev-unsubscr...@pekko.apache.org > > > > > > > > > For additional commands, e-mail: dev-h...@pekko.apache.org > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > > > > > Matthew de Detrich > > > > > > > > > > > > > > Aiven Deutschland GmbH > > > > > > > > > > > > > > Immanuelkirchstraße 26, 10405 Berlin > > > > > > > > > > > > > > Alexanderufer 3-7, 10117 Berlin > > > > > > > > > > > > > > Amtsgericht Charlottenburg, HRB 209739 B > > > > > > > > > > > > > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen > > > > > > > > > > > > > > m: +491603708037 > > > > > > > > > > > > > > w: aiven.io e: matthew.dedetr...@aiven.io > > > > > > > > > > > > > > > > --------------------------------------------------------------------- > > > > > > > To unsubscribe, e-mail: dev-unsubscr...@pekko.apache.org > > > > > > > For additional commands, e-mail: dev-h...@pekko.apache.org > > > > > > > > > > > > > > > > > > > > > --------------------------------------------------------------------- > > > > > > To unsubscribe, e-mail: dev-unsubscr...@pekko.apache.org > > > > > > For additional commands, e-mail: dev-h...@pekko.apache.org > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > Matthew de Detrich > > > > > > > > > > Aiven Deutschland GmbH > > > > > > > > > > Immanuelkirchstraße 26, 10405 Berlin > > > > > > > > > > Alexanderufer 3-7, 10117 Berlin > > > > > > > > > > Amtsgericht Charlottenburg, HRB 209739 B > > > > > > > > > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen > > > > > > > > > > m: +491603708037 > > > > > > > > > > w: aiven.io e: matthew.dedetr...@aiven.io > > > > > > > > > > > > > > > > -- > > > > > > > > Matthew de Detrich > > > > > > > > Aiven Deutschland GmbH > > > > > > > > Immanuelkirchstraße 26, 10405 Berlin > > > > > > > > Alexanderufer 3-7, 10117 Berlin > > > > > > > > Amtsgericht Charlottenburg, HRB 209739 B > > > > > > > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen > > > > > > > > m: +491603708037 > > > > > > > > w: aiven.io e: matthew.dedetr...@aiven.io > > > > > > > > --------------------------------------------------------------------- > > > > To unsubscribe, e-mail: dev-unsubscr...@pekko.apache.org > > > > For additional commands, e-mail: dev-h...@pekko.apache.org > > > > > > > > > > --------------------------------------------------------------------- > > > To unsubscribe, e-mail: dev-unsubscr...@pekko.apache.org > > > For additional commands, e-mail: dev-h...@pekko.apache.org > > > > > > > > > -- > > > > Matthew de Detrich > > > > Aiven Deutschland GmbH > > > > Immanuelkirchstraße 26, 10405 Berlin > > > > Alexanderufer 3-7, 10117 Berlin > > > > Amtsgericht Charlottenburg, HRB 209739 B > > > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen > > > > m: +491603708037 > > > > w: aiven.io e: matthew.dedetr...@aiven.io > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: dev-unsubscr...@pekko.apache.org > > For additional commands, e-mail: dev-h...@pekko.apache.org > > > > > > -- > Arnout Engelen > ASF Security Response > Apache Pekko PMC member, ASF Member > NixOS Committer > Independent Open Source consultant -- Matthew de Detrich Aiven Deutschland GmbH Immanuelkirchstraße 26, 10405 Berlin Alexanderufer 3-7, 10117 Berlin Amtsgericht Charlottenburg, HRB 209739 B Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen m: +491603708037 w: aiven.io e: matthew.dedetr...@aiven.io --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@pekko.apache.org For additional commands, e-mail: dev-h...@pekko.apache.org
