On Mar 6, 3:27 am, lahike...@gmail.com wrote:
> I appreciate this, as i've been wanting to see some benchmarks between
> the implementations for a long time. Of course, as a C advocate (i'm
> the author of protobuf-c), I'm hoping (and frankly expecting) that
> it'll win in the size AND speed category.
> I also like the style of separating the test data (.proto and sample
> data) from the test itself (like serialize v deserialize).
> Maybe we could have a standard across platforms for a benchmark utilty
> program, e.g.
> benchmark-program TASK N_ITERATIONS PROTO_FILE DATA_FILE
Being able to specify an individual task (instead of running all 6
that I currently support) would indeed be very useful. I was doing
some optimisation yesterday (improved the C# deserialization by
10-20%!) and ended up commenting out the tasks I didn't want.
Specifying them on the command line is clearly nicer :)
On the other hand, by the time you've got all of that several times
for different variations, it becomes quite verbose. I'm also wary of
putting the "N_ITERATIONS" bit in - without wishing to blow my own
trumpet, I like the fact that the framework does it for you, making
sure you get a reasonable sample. Again though, that's a bit of a pain
when it comes to optimisation where you want a short run to just see
whether it looks like it's helping.
Maybe we need a sort of pseudo language for benchmarking:
proto = foo.proto
type = GoogleSpeed.Message1
data = message1.dat
iterations = 100000
iterations = 0 // back to auto-detect
Then just pass in a file with that stuff - or even allow the user to
type it into stdin if they really want :)
We could *possibly* even allow "javatype = ..." "dotnettype=..." etc
and then share "standard" benchmarking files.
Is this all going a bit over the top, perhaps?
You received this message because you are subscribed to the Google Groups
"Protocol Buffers" group.
To post to this group, send email to email@example.com
To unsubscribe from this group, send email to
For more options, visit this group at