> Regarding the JDK version, we're already on JDK17 and I wouldn't recommend downgrading.
For the relevant performance optimizations that the Scala inliner solves, the modt critical ones are in JDK 21/GraalVM enterprise (which is now "free"), i.e. see https://docs.oracle.com/en/graalvm/jdk/21/docs/graalvm-as-a-platform/language-implementation-framework/splitting/ReportingPolymorphism/ As an example Unfortunately it's also the case that a lot of users still use JDK 1.8 (and its **those** users who ironically care about performance than most since they are enterprise users). Not sure if this changes anything but it's food for thought. > The main question for me would be what's more important to the project: a) keeping a direct comparison with akka or b) showing a good overall score. If a) and b) then we'd need to apply code changes to both pekko and akka submissions Selfishly I want both a) and b) because even aside from the Scala 2 inliner, there were seperate performance improvements or 1.1.x series which I'd like to see compared On Sat, 18 Nov 2023, 12:05 Oliver Trosien, <o...@pocket-design.de> wrote: > Regarding the JDK version, we're already on JDK17 and I wouldn't recommend > downgrading. The main question for me would be what's more important to the > project: a) keeping a direct comparison with akka or b) showing a good > overall score. If a) and b) then we'd need to apply code changes to both > pekko and akka submissions. One example showing how important it is to > understand the benchmark physics is the akka-http-slick-postgres > submission, which even for a single-query per request is performing 50% max > RPS of the blocking implementation in akka-http, and is failing the updates > test when slick starts rejecting tasks based on max queue length (*). > > Finally, some frameworks actually use these benchmarks to find and dissolve > bottlenecks in hotspots of their code, see Netty in the latest blog post > which btw also features pekko as new submission (**). > > Oliver > > (*) > > https://tfb-status.techempower.com/unzip/results.2023-11-12-22-36-41-240.zip/results/20231106203152/akka-http-slick-postgres/run/akka-http-slick-postgres.log > > (**) > https://www.techempower.com/blog/2023/11/15/framework-benchmarks-round-22/ > > > Am Sa., 18. Nov. 2023 um 10:12 Uhr schrieb Matthew de Detrich > <matthew.dedetr...@aiven.io.invalid>: > > > I am not ontop of this too much, but one major feature that is planned to > > be added specifically in Pekko 1.1.x release (and also in Pekko modules > > 1.1.x) is enabling the Scala 2 inliner which should provide at least some > > performance improvement (improvements would be more noticeable on early > > JDK's, i.e. 8,11) so maybe it's a good idea to have a separate benchmarks > > specifically for the 1.1.x series? > > > > On Tue, Nov 14, 2023 at 9:52 PM Oliver Trosien <o...@pocket-design.de> > > wrote: > > > > > Hi, > > > > > > another quick update regarding the TechEmpower benchmark(eting) > > > contribution. While the final blog post is yet to come, the benchmark > > data > > > is already available (1), and overall pekko-http got a decent overall > > > score, ending up somewhere in the middle of the field (slightly above > > Akka > > > of course :). The screenshot attached shows maximum RPS for the > following > > > test types (2): JSON: 197k, 1-query: 141k, 20-query: 10.5k, Fortunes: > > 96k, > > > Updates: 1.8k and Plaintext: 2.6M. > > > > > > (1) > > > > > > https://www.techempower.com/benchmarks/#hw=ph&test=composite§ion=data-r22 > > > (2) > > > > > > https://github.com/TechEmpower/FrameworkBenchmarks/wiki/Project-Information-Framework-Tests-Overview > > > > > > If we wanted to improve the score, there's for sure some low-hanging > > > fruit, including fixing unreasonable parallelism on the DB updates test > > > (multiple parallel update queries for each request without any > > > transactional context) which I intentionally didn't touch because my > > > primary goal was a direct comparison to the Akka implementation. After > > that > > > I would kindly ask the project to take over the annual polishing, > > including > > > upgrades and fixes. > > > > > > Cheers, > > > Oliver > > > > > > > > > > > > Am Mo., 4. Sept. 2023 um 08:46 Uhr schrieb Oliver Trosien < > > > o...@pocket-design.de>: > > > > > >> Hi, > > >> > > >> quick update: pekko-http got merged into the test suite (1), and a > first > > >> preliminary test round (2) shows it's on par with akka-http (as > > expected - > > >> we didn't change the logic of the code itself). I'll post another > update > > >> when the final numbers are ready. > > >> > > >> Oliver > > >> > > >> (1) > > >> > > > https://github.com/TechEmpower/FrameworkBenchmarks/tree/master/frameworks/Scala/pekko-http > > >> (2) > > >> > > > https://www.techempower.com/benchmarks/#section=test&runid=074e8a70-d6fb-4f10-82f3-43e57c0965b5&test=json&l=zik0zj-35b&hw=ph&p=zik0z3-zik0zj-zik0zj-zik0zj-v2qiv3-1 > > >> > > >> Am Mi., 23. Aug. 2023 um 14:57 Uhr schrieb PJ Fanning < > > >> fannin...@gmail.com>: > > >> > > >>> I think we should be able to keep track of this in future. > > >>> > > >>> On Wed, 23 Aug 2023 at 13:20, Oliver Trosien <o...@pocket-design.de> > > >>> wrote: > > >>> > > > >>> > Hi PJ, > > >>> > > > >>> > sure, I can do the initial submission. My main point is that it > only > > >>> makes > > >>> > sense if the project is willing to take over the long-term > > >>> maintainence. > > >>> > > > >>> > Oliver > > >>> > > > >>> > > > >>> > Am Mi., 23. Aug. 2023 um 11:15 Uhr schrieb PJ Fanning < > > >>> fannin...@apache.org > > >>> > >: > > >>> > > > >>> > > Hi Olivier, > > >>> > > > > >>> > > Thanks for sharing your code. Are you planning on submitting that > > to > > >>> > > TechEmpower/FrameworkBenchmarks ? > > >>> > > > > >>> > > On Wed, 23 Aug 2023 at 07:13, Oliver Trosien < > o...@pocket-design.de > > > > > >>> wrote: > > >>> > > > > > >>> > > > Hi, > > >>> > > > > > >>> > > > fair point. I have working code here (just the plain variant, > not > > >>> the one > > >>> > > > with slick from Lightbend), > > >>> > > > which is as you pointed out basically changing the imports plus > > >>> doing a > > >>> > > bit > > >>> > > > of cleanup: > > >>> > > > https://github.com/otrosien/FrameworkBenchmarks/pull/1/ > > >>> > > > > > >>> > > > Main difference to the akka version is that it runs on JDK17. > > Feel > > >>> free > > >>> > > to > > >>> > > > use this as a baseline. > > >>> > > > > > >>> > > > Regards, > > >>> > > > Oliver > > >>> > > > > > >>> > > > > > >>> > > > Am Di., 22. Aug. 2023 um 19:31 Uhr schrieb PJ Fanning < > > >>> > > fannin...@apache.org > > >>> > > > >: > > >>> > > > > > >>> > > > > Thanks Olivier. We have Pekko and Pekko HTTP releases that > > could > > >>> be > > >>> > > > > benchmarked. The Akka code could probably be pretty easily > > >>> forked and > > >>> > > > > used to create Pekko equivalents (mainly just changing the > > >>> imports and > > >>> > > > > possibly some config names). > > >>> > > > > > > >>> > > > > I wouldn't expect there to be much difference between the > Akka > > >>> and > > >>> > > > > Pekko performance numbers. > > >>> > > > > > > >>> > > > > If someone has time to do the initial setup, it should be > > fairly > > >>> > > > > straightforward for us to maintain it going forward. > > >>> > > > > > > >>> > > > > For my part, I'm more interested in getting all the Pekko > > modules > > >>> > > > > released and helping to get widely used Akka ecosystem libs > to > > >>> support > > >>> > > > > Pekko too (or to create Pekko based forks). > > >>> > > > > > > >>> > > > > On Tue, 22 Aug 2023 at 18:10, Oliver Trosien < > > >>> o...@pocket-design.de> > > >>> > > wrote: > > >>> > > > > > > > >>> > > > > > Hi, > > >>> > > > > > > > >>> > > > > > I wanted to ask the project team if they want to maintain > an > > >>> > > > > implementation > > >>> > > > > > of the framework benchmark run for pekko (you can see the > > akka > > >>> > > > > > implementation here: (1)) > > >>> > > > > > > > >>> > > > > > I would totally understand if you'd object this kind of > > >>> > > "Benchmarketing", > > >>> > > > > > but it still would give the project some additional > > >>> visibility. I do > > >>> > > > > have a > > >>> > > > > > local version which basically is a copy of the akka one. > > While > > >>> there > > >>> > > are > > >>> > > > > > some framework submissions driven by individuals, they > > usually > > >>> don't > > >>> > > have > > >>> > > > > > insights or long-term interest in maintaining these > > >>> submissions and > > >>> > > as a > > >>> > > > > > result performance regressions can shed bad light on the > > >>> project (2). > > >>> > > > > > > > >>> > > > > > They want to close submissions to the next round soon (3) - > > >>> and it > > >>> > > > > usually > > >>> > > > > > only happens once a year, so we would need to be quick to > get > > >>> > > included. > > >>> > > > > > WDYT? > > >>> > > > > > > > >>> > > > > > Regards, > > >>> > > > > > Oliver > > >>> > > > > > > > >>> > > > > > (1) > > >>> > > > > > > > >>> > > > > > > >>> > > > > >>> > > > https://github.com/TechEmpower/FrameworkBenchmarks/tree/master/frameworks/Scala/akka-http > > >>> > > > > > (2) > > >>> > > > > > > > >>> > > > > > > >>> > > > > >>> > > > https://github.com/TechEmpower/FrameworkBenchmarks/issues/7475#issuecomment-1607479108 > > >>> > > > > > (3) > > >>> > > > > > > > >>> > > > > > > >>> > > > > >>> > > > https://github.com/TechEmpower/FrameworkBenchmarks/issues/7475#issuecomment-1684087157 > > >>> > > > > > > >>> > > > > > > >>> --------------------------------------------------------------------- > > >>> > > > > To unsubscribe, e-mail: dev-unsubscr...@pekko.apache.org > > >>> > > > > For additional commands, e-mail: dev-h...@pekko.apache.org > > >>> > > > > > > >>> > > > > > > >>> > > > > >>> > > > > --------------------------------------------------------------------- > > >>> > > To unsubscribe, e-mail: dev-unsubscr...@pekko.apache.org > > >>> > > For additional commands, e-mail: dev-h...@pekko.apache.org > > >>> > > > > >>> > > > > >>> > > >>> --------------------------------------------------------------------- > > >>> To unsubscribe, e-mail: dev-unsubscr...@pekko.apache.org > > >>> For additional commands, e-mail: dev-h...@pekko.apache.org > > >>> > > >>> > > > --------------------------------------------------------------------- > > > To unsubscribe, e-mail: dev-unsubscr...@pekko.apache.org > > > For additional commands, e-mail: dev-h...@pekko.apache.org > > > > > > > > -- > > > > Matthew de Detrich > > > > *Aiven Deutschland GmbH* > > > > Immanuelkirchstraße 26, 10405 Berlin > > > > Alexanderufer 3-7, 10117 Berlin > > > > Amtsgericht Charlottenburg, HRB 209739 B > > > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen > > > > *m:* +491603708037 > > > > *w:* aiven.io *e:* matthew.dedetr...@aiven.io > > > >
