timw opened a new pull request, #862:
URL: https://github.com/apache/pekko-grpc/pull/862
# Motivation
This work started slightly accidentally, as I was primarily trying to
complete some work on the full grpc/grpc-web client <-> server RPC flows, and
when I rebased found that the protocol reader/writer internals I was evolving
had been exposed to the code generation.
There are a couple of "reverts" in this MR, but I hope they're seen as
building on the previous contributions - I think the addition of JMH
benchmarking and the unary/strict optimisation ideas are significant, and this
MR is primarily trying to retain and build on those benefits with simpler/more
maintainable implementations.
# Description of changes
This MR improves the benchmark suite, and simplifies and builds on the
implementation of the recent unary RPC performance improvements:
- the handler benchmarks are extended to all RPC styles and both the Java
and Scala generated handlers - before and after benchmarks are presented below.
- the unary response optimisation is generalised to support all protocol
variants and compression codecs while encapsulating the strict encoding concern
a bit better in the protocol implementations.
- some inlines and general use of FastFuture have been added that have
minor performance boosts.
- the Scala GrpcMarshalling.handleUnary implementation is greatly
simplified. The vast majority of the benefit of the previous implementation can
be achieved with FastFuture, and it can be improved slightly over that with
judicious use of parasitic executors for subsequent transforms. The net
performance is marginally faster now, with much less code.
- the Java GrpcMarshalling.handleUnaryResponse implementation has also been
simplified - CompletionStage already implements FastFuture like immediate
transforms for completed futures, so the optimisations implemented had minimal
benefit relative to their complexity.
- the Java unary RPC code generation has been simplified - the generated
code implementations for these exposed a lot of internal APIs in the protocol
reader/writer, making evolving those tricky, and had minimal/no impact on
benchmark performance.
- the duplicated handler generation in the Scala code generation has also
been reverted - I couldn't see any difference in benchmark performance between
a specific handler and the lifted/defaulted partial, but happy to discuss that
one if someone has seen a difference in benchmarks.
- a minor change to avoid buffer copying in gzip compress/uncompress
# Benchmark Comparison
These are benchmarks with the "full" profile run a few times to make sure
the results are concordant.
Results are from an M4 Macbook Pro with Java 21.
## Benchmarks before
```
GrpcMarshallingBenchmark.handleUnaryStrict
thrpt 30 53399977.378 ± 236563.959 ops/s
GrpcMarshallingBenchmark.marshall
thrpt 30 75091936.170 ± 396471.159 ops/s
GrpcMarshallingBenchmark.marshallStream
thrpt 30 873235.788 ± 3800.080 ops/s
GrpcMarshallingBenchmark.unmarshallJavaStrict
thrpt 30 4162386.536 ± 29487.458 ops/s
GrpcMarshallingBenchmark.unmarshallJavaStrictStreamed
thrpt 30 80746.708 ± 840.984 ops/s
GrpcMarshallingBenchmark.unmarshallStrict
thrpt 30 47609563.052 ± 457915.633 ops/s
JavaHandlerBenchmark.generatedBidiStreamingNonstrictRequestProcessing
thrpt 30 31026.673 ± 775.275 ops/s
JavaHandlerBenchmark.generatedBidiStreamingStrictRequestProcessing
thrpt 30 33088.268 ± 271.861 ops/s
JavaHandlerBenchmark.generatedClientStreamingNonstrictRequestProcessing
thrpt 30 44649.643 ± 1424.900 ops/s
JavaHandlerBenchmark.generatedClientStreamingStrictRequestProcessing
thrpt 30 49516.739 ± 2797.536 ops/s
JavaHandlerBenchmark.generatedServerStreamingNonstrictRequestProcessing
thrpt 30 22387.423 ± 285.747 ops/s
JavaHandlerBenchmark.generatedServerStreamingStrictRequestProcessing
thrpt 30 35161.133 ± 227.209 ops/s
JavaHandlerBenchmark.generatedUnaryNonstrictRequestProcessing
thrpt 30 49822.353 ± 784.518 ops/s
JavaHandlerBenchmark.generatedUnaryStrictRequestProcessing
thrpt 30 468428.800 ± 3426.273 ops/s
JavaHandlerBenchmark.oldStyleUnaryNonstrictRequestProcessing
thrpt 30 50196.435 ± 815.611 ops/s
JavaHandlerBenchmark.oldStyleUnaryStrictRequestProcessing
thrpt 30 455882.911 ± 3345.942 ops/s
ScalaHandlerBenchmark.generatedBidiStreamingNonstrictRequestProcessing
thrpt 30 25468.353 ± 260.275 ops/s
ScalaHandlerBenchmark.generatedBidiStreamingStrictRequestProcessing
thrpt 30 27334.586 ± 277.373 ops/s
ScalaHandlerBenchmark.generatedClientStreamingNonstrictRequestProcessing
thrpt 30 47795.740 ± 1224.781 ops/s
ScalaHandlerBenchmark.generatedClientStreamingStrictRequestProcessing
thrpt 30 52320.519 ± 1715.364 ops/s
ScalaHandlerBenchmark.generatedServerStreamingNonstrictRequestProcessing
thrpt 30 23030.293 ± 250.371 ops/s
ScalaHandlerBenchmark.generatedServerStreamingStrictRequestProcessing
thrpt 30 29067.371 ± 307.424 ops/s
ScalaHandlerBenchmark.generatedUnaryNonstrictRequestProcessing
thrpt 30 62822.560 ± 1118.223 ops/s
ScalaHandlerBenchmark.generatedUnaryStrictRequestProcessing
thrpt 30 508877.824 ± 4035.343 ops/s
ScalaHandlerBenchmark.oldStyleUnaryNonstrictRequestProcessing
thrpt 30 63475.816 ± 668.029 ops/s
ScalaHandlerBenchmark.oldStyleUnaryStrictRequestProcessing
thrpt 30 99281.076 ± 1255.909 ops/s
ScalaHandlerBenchmark.syntheticClientStreamingNonstrictRequestProcessing
thrpt 30 53280.750 ± 1396.899 ops/s
ScalaHandlerBenchmark.syntheticClientStreamingStrictRequestProcessing
thrpt 30 59851.853 ± 582.574 ops/s
ScalaHandlerBenchmark.syntheticUnaryNonstrictRequestProcessing
thrpt 30 71707.502 ± 1382.759 ops/s
ScalaHandlerBenchmark.syntheticUnaryStrictRequestProcessing
thrpt 30 29881100.522 ± 588646.009 ops/s
```
## Benchmarks after
```
GrpcMarshallingBenchmark.handleUnaryStrict
thrpt 30 23005007.241 ± 473057.466 ops/s
GrpcMarshallingBenchmark.marshall
thrpt 30 130375569.815 ± 830605.090 ops/s
GrpcMarshallingBenchmark.marshallStream
thrpt 30 793487.445 ± 9462.529 ops/s
GrpcMarshallingBenchmark.unmarshallJavaStrict
thrpt 30 4265269.981 ± 4172.738 ops/s
GrpcMarshallingBenchmark.unmarshallJavaStrictStreamed
thrpt 30 84973.568 ± 742.753 ops/s
GrpcMarshallingBenchmark.unmarshallStrict
thrpt 30 48102785.062 ± 145163.062 ops/s
JavaHandlerBenchmark.generatedBidiStreamingNonstrictRequestProcessing
thrpt 30 31139.727 ± 163.322 ops/s
JavaHandlerBenchmark.generatedBidiStreamingStrictRequestProcessing
thrpt 30 34658.776 ± 189.357 ops/s
JavaHandlerBenchmark.generatedClientStreamingNonstrictRequestProcessing
thrpt 30 45756.087 ± 980.167 ops/s
JavaHandlerBenchmark.generatedClientStreamingStrictRequestProcessing
thrpt 30 51655.524 ± 934.029 ops/s
JavaHandlerBenchmark.generatedServerStreamingNonstrictRequestProcessing
thrpt 30 23278.436 ± 405.283 ops/s
JavaHandlerBenchmark.generatedServerStreamingStrictRequestProcessing
thrpt 30 36367.638 ± 504.533 ops/s
JavaHandlerBenchmark.generatedUnaryNonstrictRequestProcessing
thrpt 30 52138.210 ± 768.502 ops/s
JavaHandlerBenchmark.generatedUnaryStrictRequestProcessing
thrpt 30 468946.623 ± 3685.495 ops/s
JavaHandlerBenchmark.oldStyleUnaryNonstrictRequestProcessing
thrpt 30 51820.826 ± 522.548 ops/s
JavaHandlerBenchmark.oldStyleUnaryStrictRequestProcessing
thrpt 30 459969.700 ± 1485.594 ops/s
ScalaHandlerBenchmark.generatedBidiStreamingNonstrictRequestProcessing
thrpt 30 32508.149 ± 273.993 ops/s
ScalaHandlerBenchmark.generatedBidiStreamingStrictRequestProcessing
thrpt 30 35353.414 ± 252.348 ops/s
ScalaHandlerBenchmark.generatedClientStreamingNonstrictRequestProcessing
thrpt 30 53125.037 ± 876.229 ops/s
ScalaHandlerBenchmark.generatedClientStreamingStrictRequestProcessing
thrpt 30 66494.721 ± 894.347 ops/s
ScalaHandlerBenchmark.generatedServerStreamingNonstrictRequestProcessing
thrpt 30 24302.384 ± 363.502 ops/s
ScalaHandlerBenchmark.generatedServerStreamingStrictRequestProcessing
thrpt 30 35497.762 ± 2977.292 ops/s
ScalaHandlerBenchmark.generatedUnaryNonstrictRequestProcessing
thrpt 30 62792.381 ± 2802.438 ops/s
ScalaHandlerBenchmark.generatedUnaryStrictRequestProcessing
thrpt 30 533037.363 ± 2054.773 ops/s
ScalaHandlerBenchmark.oldStyleUnaryNonstrictRequestProcessing
thrpt 30 65645.750 ± 442.854 ops/s
ScalaHandlerBenchmark.oldStyleUnaryStrictRequestProcessing
thrpt 30 519181.866 ± 4505.690 ops/s
ScalaHandlerBenchmark.syntheticClientStreamingNonstrictRequestProcessing
thrpt 30 61688.996 ± 380.460 ops/s
ScalaHandlerBenchmark.syntheticClientStreamingStrictRequestProcessing
thrpt 30 72524.001 ± 708.499 ops/s
ScalaHandlerBenchmark.syntheticUnaryNonstrictRequestProcessing
thrpt 30 74921.041 ± 814.497 ops/s
ScalaHandlerBenchmark.syntheticUnaryStrictRequestProcessing
thrpt 30 24562915.594 ± 261286.650 ops/s
```
## Benchmark changes summary
GrpcMarshallingBenchmarks have some dips, but these are probably too fine
grained and the higher level benchmarks show improvements - (you can see in the
synthetic tests that very fine grained benchmarks see really large throughput
variations with minor changes, so probably aren't usefully representative).
Java request processing is largely unchanged - consistently 0-5% improvement
across the board.
Scala request processing shows some notable improvements:
- Bidi streaming 28% improvement for strict and non-strict entities
- Client streaming 27%/11% improvement for strict/non-strict entities
- Server streaming 26%/6% improvement for strict/non-strict entities
- Unary 0-5% improvement for strict entities, unchanged for non-strict
There is also a 423% improvement for existing (pekko-grpc < 2.0.0) "old
style" generated handlers without re-generation, which might be useful if the
2.0.0 API/ABI doesn't change - essentially introducing FastFuture into the
unmarshal brings much of the benefits to existing code without having to
regenerate to use the new `handleUnary`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
