anuraaga commented on issue #2579: Benchmark bytes / bytebuffer, protobuf vs zipkin vs wire. URL: https://github.com/apache/incubator-zipkin/pull/2579#issuecomment-491252980 Yeah it'd definitely only be for the server. And while `protobuf-java-util` has some dependencies, just `protobuf-java` is dependency free and I believe has never broken backwards compatibility so I wouldn't write it off so fast ;) I added a wire-based decoder too, with the same code flow as the protobuf-based one, just uses wire for the decoding. Now that it is doing hex decoding, it's a proper comparison. Wire is quite a bit slower, probably since it has to do an array copy before converting to hex since it doesn't expose the ability to read single bytes, and maybe because protobuf uses `Unsafe` though I don't know if that really makes a big impact on modern JVMs (it would have a larger impact with java 8 than java 11 I guess). Protobuf vs zipkin seems to show protobuf a few percent faster. Garbage-wise, as expected the lack of an initial array copy does remove a whole buffer's worth of allocation, but due to the lots of allocations to create hex strings, it's a very small percentage overall, ~3%, so just a little improvement. This is after applying a change to reuse `char[]`, which had a similar improvement in the base case too. There do seem to be some differences shown here, though they're not so huge as to make it clear whether any approach is the best. ``` Benchmark (num) Mode Cnt Score Error Units ProtoCodecBenchmarks.bytes_copy_protobufDecoder 10000 sample 1309 11510.855 ± 138.341 us/op ProtoCodecBenchmarks.bytes_copy_protobufDecoder:bytes_copy_protobufDecoder·p0.00 10000 sample 10010.624 us/op ProtoCodecBenchmarks.bytes_copy_protobufDecoder:bytes_copy_protobufDecoder·p0.50 10000 sample 10977.280 us/op ProtoCodecBenchmarks.bytes_copy_protobufDecoder:bytes_copy_protobufDecoder·p0.90 10000 sample 13156.352 us/op ProtoCodecBenchmarks.bytes_copy_protobufDecoder:bytes_copy_protobufDecoder·p0.95 10000 sample 15319.040 us/op ProtoCodecBenchmarks.bytes_copy_protobufDecoder:bytes_copy_protobufDecoder·p0.99 10000 sample 16967.270 us/op ProtoCodecBenchmarks.bytes_copy_protobufDecoder:bytes_copy_protobufDecoder·p0.999 10000 sample 24244.060 us/op ProtoCodecBenchmarks.bytes_copy_protobufDecoder:bytes_copy_protobufDecoder·p0.9999 10000 sample 25493.504 us/op ProtoCodecBenchmarks.bytes_copy_protobufDecoder:bytes_copy_protobufDecoder·p1.00 10000 sample 25493.504 us/op ProtoCodecBenchmarks.bytes_copy_protobufDecoder:·gc.alloc.rate 10000 sample 15 1212.778 ± 22.951 MB/sec ProtoCodecBenchmarks.bytes_copy_protobufDecoder:·gc.alloc.rate.norm 10000 sample 15 21949689.300 ± 213.505 B/op ProtoCodecBenchmarks.bytes_copy_protobufDecoder:·gc.churn.G1_Eden_Space 10000 sample 15 1110.207 ± 113.046 MB/sec ProtoCodecBenchmarks.bytes_copy_protobufDecoder:·gc.churn.G1_Eden_Space.norm 10000 sample 15 20094766.898 ± 2030230.821 B/op ProtoCodecBenchmarks.bytes_copy_protobufDecoder:·gc.churn.G1_Old_Gen 10000 sample 15 112.514 ± 11.349 MB/sec ProtoCodecBenchmarks.bytes_copy_protobufDecoder:·gc.churn.G1_Old_Gen.norm 10000 sample 15 2036516.677 ± 203864.604 B/op ProtoCodecBenchmarks.bytes_copy_protobufDecoder:·gc.churn.G1_Survivor_Space 10000 sample 15 0.443 ± 0.438 MB/sec ProtoCodecBenchmarks.bytes_copy_protobufDecoder:·gc.churn.G1_Survivor_Space.norm 10000 sample 15 8009.395 ± 7904.907 B/op ProtoCodecBenchmarks.bytes_copy_protobufDecoder:·gc.count 10000 sample 15 82.000 counts ProtoCodecBenchmarks.bytes_copy_protobufDecoder:·gc.time 10000 sample 15 346.000 ms ProtoCodecBenchmarks.bytes_copy_wireDecoder 10000 sample 1068 14170.764 ± 135.522 us/op ProtoCodecBenchmarks.bytes_copy_wireDecoder:bytes_copy_wireDecoder·p0.00 10000 sample 12648.448 us/op ProtoCodecBenchmarks.bytes_copy_wireDecoder:bytes_copy_wireDecoder·p0.50 10000 sample 13713.408 us/op ProtoCodecBenchmarks.bytes_copy_wireDecoder:bytes_copy_wireDecoder·p0.90 10000 sample 15810.560 us/op ProtoCodecBenchmarks.bytes_copy_wireDecoder:bytes_copy_wireDecoder·p0.95 10000 sample 16846.029 us/op ProtoCodecBenchmarks.bytes_copy_wireDecoder:bytes_copy_wireDecoder·p0.99 10000 sample 19930.808 us/op ProtoCodecBenchmarks.bytes_copy_wireDecoder:bytes_copy_wireDecoder·p0.999 10000 sample 24186.126 us/op ProtoCodecBenchmarks.bytes_copy_wireDecoder:bytes_copy_wireDecoder·p0.9999 10000 sample 24281.088 us/op ProtoCodecBenchmarks.bytes_copy_wireDecoder:bytes_copy_wireDecoder·p1.00 10000 sample 24281.088 us/op ProtoCodecBenchmarks.bytes_copy_wireDecoder:·gc.alloc.rate 10000 sample 15 1129.202 ± 28.012 MB/sec ProtoCodecBenchmarks.bytes_copy_wireDecoder:·gc.alloc.rate.norm 10000 sample 15 25120909.386 ± 202.273 B/op ProtoCodecBenchmarks.bytes_copy_wireDecoder:·gc.churn.G1_Eden_Space 10000 sample 15 1040.511 ± 54.956 MB/sec ProtoCodecBenchmarks.bytes_copy_wireDecoder:·gc.churn.G1_Eden_Space.norm 10000 sample 15 23153464.194 ± 1194946.337 B/op ProtoCodecBenchmarks.bytes_copy_wireDecoder:·gc.churn.G1_Old_Gen 10000 sample 15 90.519 ± 4.650 MB/sec ProtoCodecBenchmarks.bytes_copy_wireDecoder:·gc.churn.G1_Old_Gen.norm 10000 sample 15 2014291.467 ± 102517.266 B/op ProtoCodecBenchmarks.bytes_copy_wireDecoder:·gc.churn.G1_Survivor_Space 10000 sample 15 0.838 ± 0.323 MB/sec ProtoCodecBenchmarks.bytes_copy_wireDecoder:·gc.churn.G1_Survivor_Space.norm 10000 sample 15 18578.527 ± 6846.810 B/op ProtoCodecBenchmarks.bytes_copy_wireDecoder:·gc.count 10000 sample 15 76.000 counts ProtoCodecBenchmarks.bytes_copy_wireDecoder:·gc.time 10000 sample 15 173.000 ms ProtoCodecBenchmarks.bytes_protobufDecoder 10000 sample 1340 11252.910 ± 126.595 us/op ProtoCodecBenchmarks.bytes_protobufDecoder:bytes_protobufDecoder·p0.00 10000 sample 9912.320 us/op ProtoCodecBenchmarks.bytes_protobufDecoder:bytes_protobufDecoder·p0.50 10000 sample 10731.520 us/op ProtoCodecBenchmarks.bytes_protobufDecoder:bytes_protobufDecoder·p0.90 10000 sample 12546.867 us/op ProtoCodecBenchmarks.bytes_protobufDecoder:bytes_protobufDecoder·p0.95 10000 sample 14974.976 us/op ProtoCodecBenchmarks.bytes_protobufDecoder:bytes_protobufDecoder·p0.99 10000 sample 16912.548 us/op ProtoCodecBenchmarks.bytes_protobufDecoder:bytes_protobufDecoder·p0.999 10000 sample 19200.541 us/op ProtoCodecBenchmarks.bytes_protobufDecoder:bytes_protobufDecoder·p0.9999 10000 sample 19267.584 us/op ProtoCodecBenchmarks.bytes_protobufDecoder:bytes_protobufDecoder·p1.00 10000 sample 19267.584 us/op ProtoCodecBenchmarks.bytes_protobufDecoder:·gc.alloc.rate 10000 sample 15 1130.991 ± 25.879 MB/sec ProtoCodecBenchmarks.bytes_protobufDecoder:·gc.alloc.rate.norm 10000 sample 15 20009660.550 ± 259.757 B/op ProtoCodecBenchmarks.bytes_protobufDecoder:·gc.churn.G1_Eden_Space 10000 sample 15 1136.615 ± 111.000 MB/sec ProtoCodecBenchmarks.bytes_protobufDecoder:·gc.churn.G1_Eden_Space.norm 10000 sample 15 20105313.287 ± 1867925.592 B/op ProtoCodecBenchmarks.bytes_protobufDecoder:·gc.churn.G1_Old_Gen 10000 sample 15 0.319 ± 0.197 MB/sec ProtoCodecBenchmarks.bytes_protobufDecoder:·gc.churn.G1_Old_Gen.norm 10000 sample 15 5659.194 ± 3507.786 B/op ProtoCodecBenchmarks.bytes_protobufDecoder:·gc.churn.G1_Survivor_Space 10000 sample 15 0.354 ± 0.454 MB/sec ProtoCodecBenchmarks.bytes_protobufDecoder:·gc.churn.G1_Survivor_Space.norm 10000 sample 15 6261.311 ± 8110.558 B/op ProtoCodecBenchmarks.bytes_protobufDecoder:·gc.count 10000 sample 15 84.000 counts ProtoCodecBenchmarks.bytes_protobufDecoder:·gc.time 10000 sample 15 352.000 ms ProtoCodecBenchmarks.bytes_wireDecoder 10000 sample 1027 14696.033 ± 274.890 us/op ProtoCodecBenchmarks.bytes_wireDecoder:bytes_wireDecoder·p0.00 10000 sample 12697.600 us/op ProtoCodecBenchmarks.bytes_wireDecoder:bytes_wireDecoder·p0.50 10000 sample 13991.936 us/op ProtoCodecBenchmarks.bytes_wireDecoder:bytes_wireDecoder·p0.90 10000 sample 16420.045 us/op ProtoCodecBenchmarks.bytes_wireDecoder:bytes_wireDecoder·p0.95 10000 sample 17812.685 us/op ProtoCodecBenchmarks.bytes_wireDecoder:bytes_wireDecoder·p0.99 10000 sample 26746.552 us/op ProtoCodecBenchmarks.bytes_wireDecoder:bytes_wireDecoder·p0.999 10000 sample 63427.052 us/op ProtoCodecBenchmarks.bytes_wireDecoder:bytes_wireDecoder·p0.9999 10000 sample 64225.280 us/op ProtoCodecBenchmarks.bytes_wireDecoder:bytes_wireDecoder·p1.00 10000 sample 64225.280 us/op ProtoCodecBenchmarks.bytes_wireDecoder:·gc.alloc.rate 10000 sample 15 1003.950 ± 33.760 MB/sec ProtoCodecBenchmarks.bytes_wireDecoder:·gc.alloc.rate.norm 10000 sample 15 23180993.238 ± 327.812 B/op ProtoCodecBenchmarks.bytes_wireDecoder:·gc.churn.G1_Eden_Space 10000 sample 15 1001.456 ± 113.305 MB/sec ProtoCodecBenchmarks.bytes_wireDecoder:·gc.churn.G1_Eden_Space.norm 10000 sample 15 23105838.211 ± 2265066.780 B/op ProtoCodecBenchmarks.bytes_wireDecoder:·gc.churn.G1_Old_Gen 10000 sample 15 0.585 ± 0.150 MB/sec ProtoCodecBenchmarks.bytes_wireDecoder:·gc.churn.G1_Old_Gen.norm 10000 sample 15 13499.422 ± 3403.060 B/op ProtoCodecBenchmarks.bytes_wireDecoder:·gc.churn.G1_Survivor_Space 10000 sample 15 0.752 ± 0.249 MB/sec ProtoCodecBenchmarks.bytes_wireDecoder:·gc.churn.G1_Survivor_Space.norm 10000 sample 15 17272.859 ± 5177.973 B/op ProtoCodecBenchmarks.bytes_wireDecoder:·gc.count 10000 sample 15 73.000 counts ProtoCodecBenchmarks.bytes_wireDecoder:·gc.time 10000 sample 15 158.000 ms ProtoCodecBenchmarks.bytes_zipkinDecoder 10000 sample 1220 12361.137 ± 243.588 us/op ProtoCodecBenchmarks.bytes_zipkinDecoder:bytes_zipkinDecoder·p0.00 10000 sample 9863.168 us/op ProtoCodecBenchmarks.bytes_zipkinDecoder:bytes_zipkinDecoder·p0.50 10000 sample 11452.416 us/op ProtoCodecBenchmarks.bytes_zipkinDecoder:bytes_zipkinDecoder·p0.90 10000 sample 15382.938 us/op ProtoCodecBenchmarks.bytes_zipkinDecoder:bytes_zipkinDecoder·p0.95 10000 sample 18246.861 us/op ProtoCodecBenchmarks.bytes_zipkinDecoder:bytes_zipkinDecoder·p0.99 10000 sample 22308.127 us/op ProtoCodecBenchmarks.bytes_zipkinDecoder:bytes_zipkinDecoder·p0.999 10000 sample 26914.292 us/op ProtoCodecBenchmarks.bytes_zipkinDecoder:bytes_zipkinDecoder·p0.9999 10000 sample 27066.368 us/op ProtoCodecBenchmarks.bytes_zipkinDecoder:bytes_zipkinDecoder·p1.00 10000 sample 27066.368 us/op ProtoCodecBenchmarks.bytes_zipkinDecoder:·gc.alloc.rate 10000 sample 15 984.706 ± 72.183 MB/sec ProtoCodecBenchmarks.bytes_zipkinDecoder:·gc.alloc.rate.norm 10000 sample 15 19129801.132 ± 226.638 B/op ProtoCodecBenchmarks.bytes_zipkinDecoder:·gc.churn.G1_Eden_Space 10000 sample 15 993.104 ± 126.048 MB/sec ProtoCodecBenchmarks.bytes_zipkinDecoder:·gc.churn.G1_Eden_Space.norm 10000 sample 15 19270497.313 ± 1683336.845 B/op ProtoCodecBenchmarks.bytes_zipkinDecoder:·gc.churn.G1_Old_Gen 10000 sample 15 0.434 ± 0.268 MB/sec ProtoCodecBenchmarks.bytes_zipkinDecoder:·gc.churn.G1_Old_Gen.norm 10000 sample 15 8434.596 ± 5290.023 B/op ProtoCodecBenchmarks.bytes_zipkinDecoder:·gc.churn.G1_Survivor_Space 10000 sample 15 0.753 ± 0.651 MB/sec ProtoCodecBenchmarks.bytes_zipkinDecoder:·gc.churn.G1_Survivor_Space.norm 10000 sample 15 14673.383 ± 12535.509 B/op ProtoCodecBenchmarks.bytes_zipkinDecoder:·gc.count 10000 sample 15 83.000 counts ProtoCodecBenchmarks.bytes_zipkinDecoder:·gc.time 10000 sample 15 335.000 ms
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
