I've just committed the first pass of a benchmarking tool to trunk.
Everything is under "benchmarks" - see the readme.txt for usage
guidelines.

I have a bit of tidying up to do before the committed C# version is
*exactly* equivalent (I've changed my build file around a bit) but
I've got some initial results. There are two .proto files,
representing the same messages (Message1 and Message2) but with one
optimized for speed and the other for size. (This is a great case
where a command-line option to override the optimization flag would be
handy :) There are two data files, one for each message. The benchmark
results below are running each version with each message file, with
the Java code and my current C# code.

I haven't optimised with a profiler very recently - I suspect there
are some improvements which could be made by skipping the null
handling when merging/parsing (as it should be unnecessary). I didn't
use any particular options when running the Java version (1.6.0_11-
b03) so I'm sure there are tweaks to be made there too.

Java:
Benchmarking benchmarks.GoogleSize$SizeMessage1 with file ../
google_message1.dat
Serialize to byte string: 1077894 iterations in 30.03s; 7.80469MB/s
Serialize to byte array: 1044398 iterations in 28.86s; 7.86873MB/s
Serialize to memory stream: 981322 iterations in 30.172s; 7.0720015MB/
s
Deserialize from byte string: 1378737 iterations in 30.048s;
9.977018MB/s
Deserialize from byte array: 1374400 iterations in 30.581s; 9.772291MB/
s
Deserialize from memory stream: 1212133 iterations in 30.478s;
8.647663MB/s

Benchmarking benchmarks.GoogleSpeed$SpeedMessage1 with file ../
google_message1.dat
Serialize to byte string: 8876207 iterations in 30.358s; 63.575417MB/s
Serialize to byte array: 8977534 iterations in 30.044s; 64.9732MB/s
Serialize to memory stream: 5715348 iterations in 30.485s; 40.765373MB/
s
Deserialize from byte string: 9295886 iterations in 30.159s;
67.020676MB/s
Deserialize from byte array: 9003228 iterations in 29.266s; 66.89133MB/
s
Deserialize from memory stream: 5888670 iterations in 30.466s;
42.027805MB/s

Benchmarking benchmarks.GoogleSize$SizeMessage2 with file ../
google_message2.dat
Serialize to byte string: 2598 iterations in 29.922s; 7.002691MB/s
Serialize to byte array: 2602 iterations in 30.545s; 6.8704243MB/s
Serialize to memory stream: 2412 iterations in 30.257s; 6.429362MB/s
Deserialize from byte string: 1831 iterations in 30.37s; 4.8625035MB/s
Deserialize from byte array: 1853 iterations in 30.383s; 4.918823MB/s
Deserialize from memory stream: 1839 iterations in 30.754s;
4.8227696MB/s

Benchmarking benchmarks.GoogleSpeed$SpeedMessage2 with file ../
google_message2.dat
Serialize to byte string: 18033 iterations in 30.088s; 48.33827MB/s
Serialize to byte array: 17808 iterations in 30.264s; 47.457542MB/s
Serialize to memory stream: 16569 iterations in 30.037s; 44.48936MB/s
Deserialize from byte string: 13837 iterations in 29.609s; 37.690735MB/
s
Deserialize from byte array: 14163 iterations in 30.208s; 37.813744MB/
s
Deserialize from memory stream: 12945 iterations in 28.333s;
36.84902MB/s


C#:
Benchmarking
Google.ProtocolBuffers.ProtoBench.SizeMessage1,BenchmarkTypes with
file google_message1.dat
Serialize to byte string: 1929013 iterations in 30.191s; 13.893MB/s
Serialize to byte array: 1939375 iterations in 30.179s; 13.973MB/s
Serialize to memory stream: 1759325 iterations in 29.956s; 12.770MB/s
Deserialize from byte string: 1876302 iterations in 28.483s; 14.323MB/
s
Deserialize from byte array: 1995484 iterations in 30.554s; 14.201MB/s
Deserialize from memory stream: 1831091 iterations in 30.731s;
12.956MB/s

Benchmarking
Google.ProtocolBuffers.ProtoBench.SpeedMessage1,BenchmarkTypes with
file google_message1.dat
Serialize to byte string: 17117282 iterations in 30.787s; 120.893MB/s
Serialize to byte array: 17551170 iterations in 30.469s; 125.251MB/s
Serialize to memory stream: 10226358 iterations in 28.105s; 79.117MB/s
Deserialize from byte string: 12752420 iterations in 29.695s; 93.379MB/
s
Deserialize from byte array: 12713411 iterations in 29.833s; 92.661MB/
s
Deserialize from memory stream: 8628148 iterations in 39.201s;
47.859MB/s

Benchmarking
Google.ProtocolBuffers.ProtoBench.SizeMessage2,BenchmarkTypes with
file google_message2.dat
Serialize to byte string: 4735 iterations in 29.045s; 13.148MB/s
Serialize to byte array: 4873 iterations in 30.436s; 12.913MB/s
Serialize to memory stream: 4417 iterations in 30.081s; 11.843MB/s
Deserialize from byte string: 3119 iterations in 30.270s; 8.310MB/s
Deserialize from byte array: 3123 iterations in 30.153s; 8.353MB/s
Deserialize from memory stream: 3088 iterations in 30.299s; 8.220MB/s

Benchmarking
Google.ProtocolBuffers.ProtoBench.SpeedMessage2,BenchmarkTypes with
file google_message2.dat
Serialize to byte string: 32605 iterations in 30.184s; 87.122MB/s
Serialize to byte array: 32445 iterations in 30.360s; 86.191MB/s
Serialize to memory stream: 19121 iterations in 30.093s; 51.245MB/s
Deserialize from byte string: 16237 iterations in 28.680s; 45.660MB/s
Deserialize from byte array: 17019 iterations in 30.160s; 45.512MB/s
Deserialize from memory stream: 16830 iterations in 29.851s; 45.472MB/
s

This is only an initial cut, but I wanted to get it out into the
community quickly rather than polishing build scripts etc. You
currently have to build by hand, but there are instructions in the
readme.

Thanks,
Jon


--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to 
protobuf+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en
-~----------~----~----~----~------~----~------~--~---

Reply via email to