Re: RMI Performance Tests
For such a small message, you may find that if you use flat arrays rather
than stream I/O, it may be significantly faster. That is, when serializing,
call toByteArray() and then write the array to the stream, and on the
parsing end, read the array first, then call parseFrom() on that.
Admittedly, the stream I/O methods should be doing this automatically if it
is faster.
On Tue, Aug 18, 2009 at 7:43 AM, Tai wrote:
>
> What I am measuring in writeObject() and readObject() is:
>
> 1) For PB I only measure the writeDelimitedTo() and parseDelimitedFrom
> public class PojoUsingPB extends AbstractPojo implements Serializable
> {
> ...
> private void writeObject(java.io.ObjectOutputStream out) throws
> IOException {
> ...
>long start = System.nanoTime();
>message.writeDelimitedTo(out);
>long end = System.nanoTime();
> ...
> }
>
>private void readObject(java.io.ObjectInputStream in)
>throws ClassNotFoundException, IOException {
> long start = System.nanoTime();
>PojoMessage.Pojo message = PojoMessage.Pojo.parseDelimitedFrom
> (in);
>long end = System.nanoTime();
> ...
>
> 2) In Java I measure the ObjectOutputStream.writeObject() and
> ObjectInputStream.readObject():
> public class PojoUsingJava extends AbstractPojo implements
> Serializable {
> ...
> private void writeObject(java.io.ObjectOutputStream out) throws
> IOException {
> long start = System.nanoTime();
>out.writeObject(key);
>out.writeObject(value);
>out.writeBoolean(someBoolean);
>long end = System.nanoTime();
> ...
> }
>
>private void readObject(java.io.ObjectInputStream in)
>throws ClassNotFoundException, IOException {
> long start = System.nanoTime();
>key = (String) in.readObject();
>value = (String) in.readObject();
>someBoolean = in.readBoolean();
>long end = System.nanoTime();
> ...
>
> I don't know where I could have done something wrong. The results I am
> getting is:
> Duration using Java > Protocol Buffers
> Average remote time = 111309 > 122173 nanosecs
> Average serialization time = 3609 > 6586 nanosecs
> Average de-serialization time = 3533 > 9287 nanosecs
>
> Based on that PB seems to be slower and I am not coming to the same
> conclusion as stated in other performance tests?!
>
> Tai
>
>
>
> On Aug 17, 11:24 pm, Kenton Varda wrote:
> > A message with three primitive fields should only take a couple hundred
> > nanoseconds to parse or serialize, unless the strings are *huge*. I have
> to
> > believe that something is wrong with the setup.
> >
> > On Mon, Aug 17, 2009 at 1:53 PM, Tai wrote:
> >
> > > I have now also tried parseDelimitedFrom() and writeDelimitedTo. The
> > > performance difference is still by a factor of almost 2:
> >
> > > Average Java time: 154333 nanosecs
> > > Average PB time: 293566 nanosecs
> >
> > > Average Java time: 149934 nanosecs
> > > Average PB time: 275076 nanosecs
> >
> > > I will try with a bigger message but I doubt that PB will get any
> > > close to Java's total performance (serialization, deserialization and
> > > object creation).
> >
> > > I was hoping to give our RMI environment a boost but I doesn't look
> > > like this. I will give it another chance.
> >
> > > Tai
> >
> > > On 17 Aug., 22:07, Kenton Varda wrote:
> > > > Oops, I missed it when skimming.
> >
> > > > > PojoMessage.Pojo message = PojoMessage.Pojo.parseFrom(in);
> >
> > > > That line won't work. Protocol buffers are not self-delimiting, so
> > > > parseFrom() -- if it succeeds -- always reads the *entire* input.
> You
> > > > should use parseDelimitedFrom() instead, and use writeDelimitedTo()
> on
> > > the
> > > > sending side.
> >
> > > > BTW, with such a small message, the cost of serializing and parsing
> the
> > > > message is going to be extremely small compared to the cost of the
> RPC
> > > > system, network time, etc. I'd be pretty surprised if there is any
> > > > significant difference between the two approaches.
> >
> > > > On Mon, Aug 17, 2009 at 12:48 PM, Tai
> wrote:
> >
> > > > > I am using 2.1.0 and as shown above the proto file uses option
> > > > > optimize_for = SPEED;
> >
> > > > > On Aug 17, 8:56 pm, Kenton Varda wrote:
> > > > > > What version of protocol buffers are you using? If it's 2.0.3 or
> > > > > previous
> > > > > > then you need to put this line in your proto files:
> > > > > > option optimize_for = SPEED;
> >
> > > > > > In 2.1.0 and up, this is the default.
> >
> > > > > > On Mon, Aug 17, 2009 at 10:29 AM, Tai
> > > wrote:
> >
> > > > > > > Hi,
> >
> > > > > > > I did some simple test to compare Java vs. ProtocolBuffers. I
> know
> > > > > > > that there is already a very verbose benchmark measuring
> > > > > > > serialization, object creation and de-serialization here:
> >
> > >http://www.eishay.com/2009/03/more-on-benchmarking-java-serialization.
> > > > > ..
> >
> > > > > > > But when using RMI I am gettin
Re: RMI Performance Tests
Here the result when passing a list of 1000 pojos from the client to
the server:
Duration using Java Serial > Protocol Buffers Serial > Java External >
Protocol Buffers External
Average remote time = 3539624170 > 4691285205 > 3364781976 >
4805589816 nanosecs
Average serialization time = 4928 > 22203 > 2450 > 10963 nanosecs
Average de-serialization time = 2742 > 5354 > 2374 > 5790 nanosecs
Compared to passing a single pojo:
Duration using Java Serial > Protocol Buffers Serial > Java External >
Protocol Buffers External
Average remote time = 422374212 > 430878925 > 370977875 > 409553449
nanosecs
Average serialization time = 38552 > 2196927 > 35479 > 65651 nanosecs
Average de-serialization time = 4373 > 5000 > 3629 > 4785 nanosecs
The serialization using PB + Externalizable (10963) is by a factor 5
slower than using Java + Externalizable (2450)! For a single Pojo the
factor was a little less than a factor of 2 (35479 > 65651).
Tai
On Aug 18, 6:58 pm, Tai wrote:
> Hi Brice,
>
> I have considered your input and uses Externalizable. My results
> compares Java vs. PB and Serialization vs. Externalization:
>
> Duration using Java Serial > Protocol Buffers Serial > Java External >
> Protocol Buffers External
> 1. Average remote time = 398658488 > 443504551 > 372961228 > 380994372
> nanosecs
> 2. Average serialization time = 29613 > 2192178 > 27937 > 57270
> nanosecs
> 3. Average de-serialization time = 3439 > 4737 > 2989 > 4456 nanosecs
>
> For 1)
> As you can see the overall remote call is faster using Externalizable
> compared to Serizalizable. But in both cases PB is slightly slower
> than Java.
> For 2)
> The serialization using Externalizable is slightly better for Java
> (29613 / 27937). For PB it is a huge difference using Externalizable
> (2192178 / 57270). But PB compared to Java is still slower.
> For 3)
> Using Externalizable for de-serialization is slightly better. But here
> again PB is slower than Java.
>
> Your suggestion to create builder once in the constructor for
> immutable objects is good. In our case it is a no-opt since they are
> not immutable.
>
> In my performance test I have two String (with key "key" and value
> "value") and one boolean member. In a production environment my key is
> max 30 and the value 50 chars long. For that I get:
>
> Duration using Java Serial > Protocol Buffers Serial > Java External >
> Protocol Buffers External
> Average remote time = 422374212 > 430878925 > 370977875 > 409553449
> nanosecs
> Average serialization time = 38552 > 2196927 > 35479 > 65651 nanosecs
> Average de-serialization time = 4373 > 5000 > 3629 > 4785 nanosecs
>
> This result brings me to the same conclusion as mentioned above. The
> next thing I will try is to have Pojo with a lot more members. Also I
> will try to pass a list of pojos from the client to the server. This
> would then cover our production environment. Looking at the
> performance in general I think the biggest boost would be in the byte
> size of a PB message. The biggest performance bottleneck is the
> network transmission and therefor the byte size should quite have a
> big impact on the performance. The other performance issues for object
> creation, serialization and deserialization is quite small compared to
> the remote call.
>
> Tai
>
> On Aug 18, 5:12 pm, Brice Figureau wrote:
>
> > On Tue, 2009-08-18 at 07:43 -0700, Tai wrote:
> > > What I am measuring in writeObject() and readObject() is:
>
> > > 1) For PB I only measure the writeDelimitedTo() and parseDelimitedFrom
> > > public class PojoUsingPB extends AbstractPojo implements Serializable
> > > {
> > > ...
> > > private void writeObject(java.io.ObjectOutputStream out) throws
> > > IOException {
> > > ...
> > > long start = System.nanoTime();
> > > message.writeDelimitedTo(out);
> > > long end = System.nanoTime();
> > > ...
> > > }
>
> > > private void readObject(java.io.ObjectInputStream in)
> > > throws ClassNotFoundException, IOException {
> > > long start = System.nanoTime();
> > > PojoMessage.Pojo message = PojoMessage.Pojo.parseDelimitedFrom
> > > (in);
> > > long end = System.nanoTime();
> > > ...
>
> > > 2) In Java I measure the ObjectOutputStream.writeObject() and
> > > ObjectInputStream.readObject():
> > > public class PojoUsingJava extends AbstractPojo implements
> > > Serializable {
> > > ...
> > > private void writeObject(java.io.ObjectOutputStream out) throws
> > > IOException {
> > > long start = System.nanoTime();
> > > out.writeObject(key);
> > > out.writeObject(value);
> > > out.writeBoolean(someBoolean);
> > > long end = System.nanoTime();
> > > ...
> > > }
>
> > > private void readObject(java.io.ObjectInputStream in)
> > > throws ClassNotFoundException, IOException {
> > > long start = System.nanoTime();
> > > key = (String) in.readObject();
> > > value = (String) in.readObject();
>
Re: RMI Performance Tests
Hi Brice,
I have considered your input and uses Externalizable. My results
compares Java vs. PB and Serialization vs. Externalization:
Duration using Java Serial > Protocol Buffers Serial > Java External >
Protocol Buffers External
1. Average remote time = 398658488 > 443504551 > 372961228 > 380994372
nanosecs
2. Average serialization time = 29613 > 2192178 > 27937 > 57270
nanosecs
3. Average de-serialization time = 3439 > 4737 > 2989 > 4456 nanosecs
For 1)
As you can see the overall remote call is faster using Externalizable
compared to Serizalizable. But in both cases PB is slightly slower
than Java.
For 2)
The serialization using Externalizable is slightly better for Java
(29613 / 27937). For PB it is a huge difference using Externalizable
(2192178 / 57270). But PB compared to Java is still slower.
For 3)
Using Externalizable for de-serialization is slightly better. But here
again PB is slower than Java.
Your suggestion to create builder once in the constructor for
immutable objects is good. In our case it is a no-opt since they are
not immutable.
In my performance test I have two String (with key "key" and value
"value") and one boolean member. In a production environment my key is
max 30 and the value 50 chars long. For that I get:
Duration using Java Serial > Protocol Buffers Serial > Java External >
Protocol Buffers External
Average remote time = 422374212 > 430878925 > 370977875 > 409553449
nanosecs
Average serialization time = 38552 > 2196927 > 35479 > 65651 nanosecs
Average de-serialization time = 4373 > 5000 > 3629 > 4785 nanosecs
This result brings me to the same conclusion as mentioned above. The
next thing I will try is to have Pojo with a lot more members. Also I
will try to pass a list of pojos from the client to the server. This
would then cover our production environment. Looking at the
performance in general I think the biggest boost would be in the byte
size of a PB message. The biggest performance bottleneck is the
network transmission and therefor the byte size should quite have a
big impact on the performance. The other performance issues for object
creation, serialization and deserialization is quite small compared to
the remote call.
Tai
On Aug 18, 5:12 pm, Brice Figureau wrote:
> On Tue, 2009-08-18 at 07:43 -0700, Tai wrote:
> > What I am measuring in writeObject() and readObject() is:
>
> > 1) For PB I only measure the writeDelimitedTo() and parseDelimitedFrom
> > public class PojoUsingPB extends AbstractPojo implements Serializable
> > {
> > ...
> > private void writeObject(java.io.ObjectOutputStream out) throws
> > IOException {
> > ...
> > long start = System.nanoTime();
> > message.writeDelimitedTo(out);
> > long end = System.nanoTime();
> > ...
> > }
>
> > private void readObject(java.io.ObjectInputStream in)
> > throws ClassNotFoundException, IOException {
> > long start = System.nanoTime();
> > PojoMessage.Pojo message = PojoMessage.Pojo.parseDelimitedFrom
> > (in);
> > long end = System.nanoTime();
> > ...
>
> > 2) In Java I measure the ObjectOutputStream.writeObject() and
> > ObjectInputStream.readObject():
> > public class PojoUsingJava extends AbstractPojo implements
> > Serializable {
> > ...
> > private void writeObject(java.io.ObjectOutputStream out) throws
> > IOException {
> > long start = System.nanoTime();
> > out.writeObject(key);
> > out.writeObject(value);
> > out.writeBoolean(someBoolean);
> > long end = System.nanoTime();
> > ...
> > }
>
> > private void readObject(java.io.ObjectInputStream in)
> > throws ClassNotFoundException, IOException {
> > long start = System.nanoTime();
> > key = (String) in.readObject();
> > value = (String) in.readObject();
> > someBoolean = in.readBoolean();
> > long end = System.nanoTime();
> > ...
>
> > I don't know where I could have done something wrong. The results I am
> > getting is:
> > Duration using Java > Protocol Buffers
> > Average remote time = 111309 > 122173 nanosecs
> > Average serialization time = 3609 > 6586 nanosecs
> > Average de-serialization time = 3533 > 9287 nanosecs
>
> > Based on that PB seems to be slower and I am not coming to the same
> > conclusion as stated in other performance tests?!
>
> That's because you create a PB, builder and call build for each written
> object.
> The Java serialization doesn't have to do that, it just feeds the fields
> of your POJO directly to the stream.
>
> And the same applies to the deserialization.
>
> If you were writing much more than 2 fields, then you'd see the weight
> of creating the objects be less compared to the serialization time.
>
> I'm not familiar with Java Serialization (anymore), but I think I
> remember overriding writeObject and readObject still uses java
> serialization for some parts of the current object (including
> superclass, and so on).
> Shouldn't you use in
Re: RMI Performance Tests
On Tue, 2009-08-18 at 07:43 -0700, Tai wrote:
> What I am measuring in writeObject() and readObject() is:
>
> 1) For PB I only measure the writeDelimitedTo() and parseDelimitedFrom
> public class PojoUsingPB extends AbstractPojo implements Serializable
> {
> ...
> private void writeObject(java.io.ObjectOutputStream out) throws
> IOException {
> ...
> long start = System.nanoTime();
> message.writeDelimitedTo(out);
> long end = System.nanoTime();
> ...
> }
>
> private void readObject(java.io.ObjectInputStream in)
> throws ClassNotFoundException, IOException {
> long start = System.nanoTime();
> PojoMessage.Pojo message = PojoMessage.Pojo.parseDelimitedFrom
> (in);
> long end = System.nanoTime();
> ...
>
> 2) In Java I measure the ObjectOutputStream.writeObject() and
> ObjectInputStream.readObject():
> public class PojoUsingJava extends AbstractPojo implements
> Serializable {
> ...
> private void writeObject(java.io.ObjectOutputStream out) throws
> IOException {
> long start = System.nanoTime();
> out.writeObject(key);
> out.writeObject(value);
> out.writeBoolean(someBoolean);
> long end = System.nanoTime();
> ...
> }
>
> private void readObject(java.io.ObjectInputStream in)
> throws ClassNotFoundException, IOException {
> long start = System.nanoTime();
> key = (String) in.readObject();
> value = (String) in.readObject();
> someBoolean = in.readBoolean();
> long end = System.nanoTime();
> ...
>
> I don't know where I could have done something wrong. The results I am
> getting is:
> Duration using Java > Protocol Buffers
> Average remote time = 111309 > 122173 nanosecs
> Average serialization time = 3609 > 6586 nanosecs
> Average de-serialization time = 3533 > 9287 nanosecs
>
> Based on that PB seems to be slower and I am not coming to the same
> conclusion as stated in other performance tests?!
That's because you create a PB, builder and call build for each written
object.
The Java serialization doesn't have to do that, it just feeds the fields
of your POJO directly to the stream.
And the same applies to the deserialization.
If you were writing much more than 2 fields, then you'd see the weight
of creating the objects be less compared to the serialization time.
I'm not familiar with Java Serialization (anymore), but I think I
remember overriding writeObject and readObject still uses java
serialization for some parts of the current object (including
superclass, and so on).
Shouldn't you use instead an Externalizable object?
My suggestion:
* benchmark in situation or with larger close to production objects
* if your objects are immutable, create the builder in your
constructer, and build at that time. This way you can write multiple
time without incurring the creation cost.
* use an Externalizable object
HTH,
--
Brice Figureau
My Blog: http://www.masterzen.fr/
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Protocol Buffers" group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to
protobuf+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---
Re: RMI Performance Tests
What I am measuring in writeObject() and readObject() is:
1) For PB I only measure the writeDelimitedTo() and parseDelimitedFrom
public class PojoUsingPB extends AbstractPojo implements Serializable
{
...
private void writeObject(java.io.ObjectOutputStream out) throws
IOException {
...
long start = System.nanoTime();
message.writeDelimitedTo(out);
long end = System.nanoTime();
...
}
private void readObject(java.io.ObjectInputStream in)
throws ClassNotFoundException, IOException {
long start = System.nanoTime();
PojoMessage.Pojo message = PojoMessage.Pojo.parseDelimitedFrom
(in);
long end = System.nanoTime();
...
2) In Java I measure the ObjectOutputStream.writeObject() and
ObjectInputStream.readObject():
public class PojoUsingJava extends AbstractPojo implements
Serializable {
...
private void writeObject(java.io.ObjectOutputStream out) throws
IOException {
long start = System.nanoTime();
out.writeObject(key);
out.writeObject(value);
out.writeBoolean(someBoolean);
long end = System.nanoTime();
...
}
private void readObject(java.io.ObjectInputStream in)
throws ClassNotFoundException, IOException {
long start = System.nanoTime();
key = (String) in.readObject();
value = (String) in.readObject();
someBoolean = in.readBoolean();
long end = System.nanoTime();
...
I don't know where I could have done something wrong. The results I am
getting is:
Duration using Java > Protocol Buffers
Average remote time = 111309 > 122173 nanosecs
Average serialization time = 3609 > 6586 nanosecs
Average de-serialization time = 3533 > 9287 nanosecs
Based on that PB seems to be slower and I am not coming to the same
conclusion as stated in other performance tests?!
Tai
On Aug 17, 11:24 pm, Kenton Varda wrote:
> A message with three primitive fields should only take a couple hundred
> nanoseconds to parse or serialize, unless the strings are *huge*. I have to
> believe that something is wrong with the setup.
>
> On Mon, Aug 17, 2009 at 1:53 PM, Tai wrote:
>
> > I have now also tried parseDelimitedFrom() and writeDelimitedTo. The
> > performance difference is still by a factor of almost 2:
>
> > Average Java time: 154333 nanosecs
> > Average PB time: 293566 nanosecs
>
> > Average Java time: 149934 nanosecs
> > Average PB time: 275076 nanosecs
>
> > I will try with a bigger message but I doubt that PB will get any
> > close to Java's total performance (serialization, deserialization and
> > object creation).
>
> > I was hoping to give our RMI environment a boost but I doesn't look
> > like this. I will give it another chance.
>
> > Tai
>
> > On 17 Aug., 22:07, Kenton Varda wrote:
> > > Oops, I missed it when skimming.
>
> > > > PojoMessage.Pojo message = PojoMessage.Pojo.parseFrom(in);
>
> > > That line won't work. Protocol buffers are not self-delimiting, so
> > > parseFrom() -- if it succeeds -- always reads the *entire* input. You
> > > should use parseDelimitedFrom() instead, and use writeDelimitedTo() on
> > the
> > > sending side.
>
> > > BTW, with such a small message, the cost of serializing and parsing the
> > > message is going to be extremely small compared to the cost of the RPC
> > > system, network time, etc. I'd be pretty surprised if there is any
> > > significant difference between the two approaches.
>
> > > On Mon, Aug 17, 2009 at 12:48 PM, Tai wrote:
>
> > > > I am using 2.1.0 and as shown above the proto file uses option
> > > > optimize_for = SPEED;
>
> > > > On Aug 17, 8:56 pm, Kenton Varda wrote:
> > > > > What version of protocol buffers are you using? If it's 2.0.3 or
> > > > previous
> > > > > then you need to put this line in your proto files:
> > > > > option optimize_for = SPEED;
>
> > > > > In 2.1.0 and up, this is the default.
>
> > > > > On Mon, Aug 17, 2009 at 10:29 AM, Tai
> > wrote:
>
> > > > > > Hi,
>
> > > > > > I did some simple test to compare Java vs. ProtocolBuffers. I know
> > > > > > that there is already a very verbose benchmark measuring
> > > > > > serialization, object creation and de-serialization here:
>
> >http://www.eishay.com/2009/03/more-on-benchmarking-java-serialization.
> > > > ..
>
> > > > > > But when using RMI I am getting a total time where PB is slower
> > than
> > > > > > Java. My performance test measures the total time when passing a
> > Pojo
> > > > > > from a client to a remote agent. The results I get when running the
> > > > > > tests twice are:
> > > > > > Average Java time: 147421 nanosecs
> > > > > > Average PB time: 281617 nanosecs (factor 1.91 slower)
>
> > > > > > Average Java time: 232446 nanosecs
> > > > > > Average PB time: 467485 nanosecs (factor 2.01 slower)
>
> > > > > > I was expecting PB to be faster than Java?! But this seems to be
> > > > > > wrong.
>
> > > > > > My classes are (unfortunately I cannot attach to the source / you
> > can
> > > > > > e
Re: RMI Performance Tests
A message with three primitive fields should only take a couple hundred
nanoseconds to parse or serialize, unless the strings are *huge*. I have to
believe that something is wrong with the setup.
On Mon, Aug 17, 2009 at 1:53 PM, Tai wrote:
>
> I have now also tried parseDelimitedFrom() and writeDelimitedTo. The
> performance difference is still by a factor of almost 2:
>
> Average Java time: 154333 nanosecs
> Average PB time: 293566 nanosecs
>
> Average Java time: 149934 nanosecs
> Average PB time: 275076 nanosecs
>
> I will try with a bigger message but I doubt that PB will get any
> close to Java's total performance (serialization, deserialization and
> object creation).
>
> I was hoping to give our RMI environment a boost but I doesn't look
> like this. I will give it another chance.
>
> Tai
>
>
> On 17 Aug., 22:07, Kenton Varda wrote:
> > Oops, I missed it when skimming.
> >
> > > PojoMessage.Pojo message = PojoMessage.Pojo.parseFrom(in);
> >
> > That line won't work. Protocol buffers are not self-delimiting, so
> > parseFrom() -- if it succeeds -- always reads the *entire* input. You
> > should use parseDelimitedFrom() instead, and use writeDelimitedTo() on
> the
> > sending side.
> >
> > BTW, with such a small message, the cost of serializing and parsing the
> > message is going to be extremely small compared to the cost of the RPC
> > system, network time, etc. I'd be pretty surprised if there is any
> > significant difference between the two approaches.
> >
> > On Mon, Aug 17, 2009 at 12:48 PM, Tai wrote:
> >
> > > I am using 2.1.0 and as shown above the proto file uses option
> > > optimize_for = SPEED;
> >
> > > On Aug 17, 8:56 pm, Kenton Varda wrote:
> > > > What version of protocol buffers are you using? If it's 2.0.3 or
> > > previous
> > > > then you need to put this line in your proto files:
> > > > option optimize_for = SPEED;
> >
> > > > In 2.1.0 and up, this is the default.
> >
> > > > On Mon, Aug 17, 2009 at 10:29 AM, Tai
> wrote:
> >
> > > > > Hi,
> >
> > > > > I did some simple test to compare Java vs. ProtocolBuffers. I know
> > > > > that there is already a very verbose benchmark measuring
> > > > > serialization, object creation and de-serialization here:
> > > > >
> http://www.eishay.com/2009/03/more-on-benchmarking-java-serialization.
> > > ..
> >
> > > > > But when using RMI I am getting a total time where PB is slower
> than
> > > > > Java. My performance test measures the total time when passing a
> Pojo
> > > > > from a client to a remote agent. The results I get when running the
> > > > > tests twice are:
> > > > > Average Java time: 147421 nanosecs
> > > > > Average PB time: 281617 nanosecs (factor 1.91 slower)
> >
> > > > > Average Java time: 232446 nanosecs
> > > > > Average PB time: 467485 nanosecs (factor 2.01 slower)
> >
> > > > > I was expecting PB to be faster than Java?! But this seems to be
> > > > > wrong.
> >
> > > > > My classes are (unfortunately I cannot attach to the source / you
> can
> > > > > email me if you want a zip):
> >
> > > > > The rmi interface for the agent server:
> > > > > =
> > > > > package test.protocolbuffers;
> >
> > > > > import java.rmi.Remote;
> > > > > import java.rmi.RemoteException;
> >
> > > > > public interface IRemoteAgent extends Remote {
> >
> > > > >long doSomething(long start, Object pojo) throws
> RemoteException;
> >
> > > > > }
> > > > > =
> >
> > > > > The server agent implementing the interface
> > > > > =
> > > > > package test.protocolbuffers;
> >
> > > > > import java.rmi.RemoteException;
> > > > > import java.rmi.registry.LocateRegistry;
> > > > > import java.rmi.registry.Registry;
> > > > > import java.rmi.server.UnicastRemoteObject;
> >
> > > > > public class RemoteAgent implements IRemoteAgent {
> >
> > > > >public RemoteAgent() {
> > > > >}
> >
> > > > >public static void main(String[] args) throws Exception {
> > > > >RemoteAgent agent = new RemoteAgent();
> > > > >String name = "RemoteAgent1";
> > > > >UnicastRemoteObject.exportObject(agent, 0);
> > > > >Registry registry;
> > > > >try {
> > > > >registry = LocateRegistry.createRegistry(1099);
> > > > >} catch (Exception e) {
> > > > >registry = LocateRegistry.getRegistry();
> > > > >}
> > > > >registry.rebind(name, agent);
> > > > >System.out.println(name + " bound.");
> >
> > > > >}
> >
> > > > >public long doSomething(long start, Object pojo) throws
> > > > > RemoteException {
> > > > >long end = System.nanoTime();
> > > > >return end - start;
> > > > >}
> > > > > }
> > > > > =
> >
> > > > > The Java POJO for measuring the using standard Java serialization:
> > > > > =
Re: RMI Performance Tests
I have now also tried parseDelimitedFrom() and writeDelimitedTo. The
performance difference is still by a factor of almost 2:
Average Java time: 154333 nanosecs
Average PB time: 293566 nanosecs
Average Java time: 149934 nanosecs
Average PB time: 275076 nanosecs
I will try with a bigger message but I doubt that PB will get any
close to Java's total performance (serialization, deserialization and
object creation).
I was hoping to give our RMI environment a boost but I doesn't look
like this. I will give it another chance.
Tai
On 17 Aug., 22:07, Kenton Varda wrote:
> Oops, I missed it when skimming.
>
> > PojoMessage.Pojo message = PojoMessage.Pojo.parseFrom(in);
>
> That line won't work. Protocol buffers are not self-delimiting, so
> parseFrom() -- if it succeeds -- always reads the *entire* input. You
> should use parseDelimitedFrom() instead, and use writeDelimitedTo() on the
> sending side.
>
> BTW, with such a small message, the cost of serializing and parsing the
> message is going to be extremely small compared to the cost of the RPC
> system, network time, etc. I'd be pretty surprised if there is any
> significant difference between the two approaches.
>
> On Mon, Aug 17, 2009 at 12:48 PM, Tai wrote:
>
> > I am using 2.1.0 and as shown above the proto file uses option
> > optimize_for = SPEED;
>
> > On Aug 17, 8:56 pm, Kenton Varda wrote:
> > > What version of protocol buffers are you using? If it's 2.0.3 or
> > previous
> > > then you need to put this line in your proto files:
> > > option optimize_for = SPEED;
>
> > > In 2.1.0 and up, this is the default.
>
> > > On Mon, Aug 17, 2009 at 10:29 AM, Tai wrote:
>
> > > > Hi,
>
> > > > I did some simple test to compare Java vs. ProtocolBuffers. I know
> > > > that there is already a very verbose benchmark measuring
> > > > serialization, object creation and de-serialization here:
> > > >http://www.eishay.com/2009/03/more-on-benchmarking-java-serialization.
> > ..
>
> > > > But when using RMI I am getting a total time where PB is slower than
> > > > Java. My performance test measures the total time when passing a Pojo
> > > > from a client to a remote agent. The results I get when running the
> > > > tests twice are:
> > > > Average Java time: 147421 nanosecs
> > > > Average PB time: 281617 nanosecs (factor 1.91 slower)
>
> > > > Average Java time: 232446 nanosecs
> > > > Average PB time: 467485 nanosecs (factor 2.01 slower)
>
> > > > I was expecting PB to be faster than Java?! But this seems to be
> > > > wrong.
>
> > > > My classes are (unfortunately I cannot attach to the source / you can
> > > > email me if you want a zip):
>
> > > > The rmi interface for the agent server:
> > > > =
> > > > package test.protocolbuffers;
>
> > > > import java.rmi.Remote;
> > > > import java.rmi.RemoteException;
>
> > > > public interface IRemoteAgent extends Remote {
>
> > > > long doSomething(long start, Object pojo) throws RemoteException;
>
> > > > }
> > > > =
>
> > > > The server agent implementing the interface
> > > > =
> > > > package test.protocolbuffers;
>
> > > > import java.rmi.RemoteException;
> > > > import java.rmi.registry.LocateRegistry;
> > > > import java.rmi.registry.Registry;
> > > > import java.rmi.server.UnicastRemoteObject;
>
> > > > public class RemoteAgent implements IRemoteAgent {
>
> > > > public RemoteAgent() {
> > > > }
>
> > > > public static void main(String[] args) throws Exception {
> > > > RemoteAgent agent = new RemoteAgent();
> > > > String name = "RemoteAgent1";
> > > > UnicastRemoteObject.exportObject(agent, 0);
> > > > Registry registry;
> > > > try {
> > > > registry = LocateRegistry.createRegistry(1099);
> > > > } catch (Exception e) {
> > > > registry = LocateRegistry.getRegistry();
> > > > }
> > > > registry.rebind(name, agent);
> > > > System.out.println(name + " bound.");
>
> > > > }
>
> > > > public long doSomething(long start, Object pojo) throws
> > > > RemoteException {
> > > > long end = System.nanoTime();
> > > > return end - start;
> > > > }
> > > > }
> > > > =
>
> > > > The Java POJO for measuring the using standard Java serialization:
> > > > =
> > > > package test.protocolbuffers;
>
> > > > import java.io.Serializable;
>
> > > > public class PojoUsingJava implements Serializable {
> > > > private String key;
> > > > private String value;
> > > > private boolean someBoolean;
>
> > > > public PojoUsingJava(String key, String value, boolean
> > > > someBoolean) {
> > > > this.key = key;
> > > > this.value = value;
> > > > this.someBoolean = someBoolean;
> > > > }
>
>
Re: RMI Performance Tests
Oops, I missed it when skimming.
> PojoMessage.Pojo message = PojoMessage.Pojo.parseFrom(in);
That line won't work. Protocol buffers are not self-delimiting, so
parseFrom() -- if it succeeds -- always reads the *entire* input. You
should use parseDelimitedFrom() instead, and use writeDelimitedTo() on the
sending side.
BTW, with such a small message, the cost of serializing and parsing the
message is going to be extremely small compared to the cost of the RPC
system, network time, etc. I'd be pretty surprised if there is any
significant difference between the two approaches.
On Mon, Aug 17, 2009 at 12:48 PM, Tai wrote:
>
> I am using 2.1.0 and as shown above the proto file uses option
> optimize_for = SPEED;
>
> On Aug 17, 8:56 pm, Kenton Varda wrote:
> > What version of protocol buffers are you using? If it's 2.0.3 or
> previous
> > then you need to put this line in your proto files:
> > option optimize_for = SPEED;
> >
> > In 2.1.0 and up, this is the default.
> >
> > On Mon, Aug 17, 2009 at 10:29 AM, Tai wrote:
> >
> > > Hi,
> >
> > > I did some simple test to compare Java vs. ProtocolBuffers. I know
> > > that there is already a very verbose benchmark measuring
> > > serialization, object creation and de-serialization here:
> > >http://www.eishay.com/2009/03/more-on-benchmarking-java-serialization.
> ..
> >
> > > But when using RMI I am getting a total time where PB is slower than
> > > Java. My performance test measures the total time when passing a Pojo
> > > from a client to a remote agent. The results I get when running the
> > > tests twice are:
> > > Average Java time: 147421 nanosecs
> > > Average PB time: 281617 nanosecs (factor 1.91 slower)
> >
> > > Average Java time: 232446 nanosecs
> > > Average PB time: 467485 nanosecs (factor 2.01 slower)
> >
> > > I was expecting PB to be faster than Java?! But this seems to be
> > > wrong.
> >
> > > My classes are (unfortunately I cannot attach to the source / you can
> > > email me if you want a zip):
> >
> > > The rmi interface for the agent server:
> > > =
> > > package test.protocolbuffers;
> >
> > > import java.rmi.Remote;
> > > import java.rmi.RemoteException;
> >
> > > public interface IRemoteAgent extends Remote {
> >
> > >long doSomething(long start, Object pojo) throws RemoteException;
> >
> > > }
> > > =
> >
> > > The server agent implementing the interface
> > > =
> > > package test.protocolbuffers;
> >
> > > import java.rmi.RemoteException;
> > > import java.rmi.registry.LocateRegistry;
> > > import java.rmi.registry.Registry;
> > > import java.rmi.server.UnicastRemoteObject;
> >
> > > public class RemoteAgent implements IRemoteAgent {
> >
> > >public RemoteAgent() {
> > >}
> >
> > >public static void main(String[] args) throws Exception {
> > >RemoteAgent agent = new RemoteAgent();
> > >String name = "RemoteAgent1";
> > >UnicastRemoteObject.exportObject(agent, 0);
> > >Registry registry;
> > >try {
> > >registry = LocateRegistry.createRegistry(1099);
> > >} catch (Exception e) {
> > >registry = LocateRegistry.getRegistry();
> > >}
> > >registry.rebind(name, agent);
> > >System.out.println(name + " bound.");
> >
> > >}
> >
> > >public long doSomething(long start, Object pojo) throws
> > > RemoteException {
> > >long end = System.nanoTime();
> > >return end - start;
> > >}
> > > }
> > > =
> >
> > > The Java POJO for measuring the using standard Java serialization:
> > > =
> > > package test.protocolbuffers;
> >
> > > import java.io.Serializable;
> >
> > > public class PojoUsingJava implements Serializable {
> > >private String key;
> > >private String value;
> > >private boolean someBoolean;
> >
> > >public PojoUsingJava(String key, String value, boolean
> > > someBoolean) {
> > >this.key = key;
> > >this.value = value;
> > >this.someBoolean = someBoolean;
> > >}
> >
> > > }
> > > =
> >
> > > The PB Pojo:
> > > =
> > > package test.protocolbuffers;
> >
> > > import java.io.IOException;
> > > import java.io.Serializable;
> > > import test.protocolbuffers.PojoMessage.Pojo.Builder;
> >
> > > public class PojoUsingPB implements Serializable {
> > >private String key;
> > >private String value;
> > >private boolean someBoolean;
> >
> > >public PojoUsingPB(String key, String value, boolean someBoolean)
> > > {
> > >this.key = key;
> > >this.value = value;
> > >this.someBoolean = someBoolean;
> > >}
> >
> > >private vo
Re: RMI Performance Tests
I am using 2.1.0 and as shown above the proto file uses option
optimize_for = SPEED;
On Aug 17, 8:56 pm, Kenton Varda wrote:
> What version of protocol buffers are you using? If it's 2.0.3 or previous
> then you need to put this line in your proto files:
> option optimize_for = SPEED;
>
> In 2.1.0 and up, this is the default.
>
> On Mon, Aug 17, 2009 at 10:29 AM, Tai wrote:
>
> > Hi,
>
> > I did some simple test to compare Java vs. ProtocolBuffers. I know
> > that there is already a very verbose benchmark measuring
> > serialization, object creation and de-serialization here:
> >http://www.eishay.com/2009/03/more-on-benchmarking-java-serialization...
>
> > But when using RMI I am getting a total time where PB is slower than
> > Java. My performance test measures the total time when passing a Pojo
> > from a client to a remote agent. The results I get when running the
> > tests twice are:
> > Average Java time: 147421 nanosecs
> > Average PB time: 281617 nanosecs (factor 1.91 slower)
>
> > Average Java time: 232446 nanosecs
> > Average PB time: 467485 nanosecs (factor 2.01 slower)
>
> > I was expecting PB to be faster than Java?! But this seems to be
> > wrong.
>
> > My classes are (unfortunately I cannot attach to the source / you can
> > email me if you want a zip):
>
> > The rmi interface for the agent server:
> > =
> > package test.protocolbuffers;
>
> > import java.rmi.Remote;
> > import java.rmi.RemoteException;
>
> > public interface IRemoteAgent extends Remote {
>
> > long doSomething(long start, Object pojo) throws RemoteException;
>
> > }
> > =
>
> > The server agent implementing the interface
> > =
> > package test.protocolbuffers;
>
> > import java.rmi.RemoteException;
> > import java.rmi.registry.LocateRegistry;
> > import java.rmi.registry.Registry;
> > import java.rmi.server.UnicastRemoteObject;
>
> > public class RemoteAgent implements IRemoteAgent {
>
> > public RemoteAgent() {
> > }
>
> > public static void main(String[] args) throws Exception {
> > RemoteAgent agent = new RemoteAgent();
> > String name = "RemoteAgent1";
> > UnicastRemoteObject.exportObject(agent, 0);
> > Registry registry;
> > try {
> > registry = LocateRegistry.createRegistry(1099);
> > } catch (Exception e) {
> > registry = LocateRegistry.getRegistry();
> > }
> > registry.rebind(name, agent);
> > System.out.println(name + " bound.");
>
> > }
>
> > public long doSomething(long start, Object pojo) throws
> > RemoteException {
> > long end = System.nanoTime();
> > return end - start;
> > }
> > }
> > =
>
> > The Java POJO for measuring the using standard Java serialization:
> > =
> > package test.protocolbuffers;
>
> > import java.io.Serializable;
>
> > public class PojoUsingJava implements Serializable {
> > private String key;
> > private String value;
> > private boolean someBoolean;
>
> > public PojoUsingJava(String key, String value, boolean
> > someBoolean) {
> > this.key = key;
> > this.value = value;
> > this.someBoolean = someBoolean;
> > }
>
> > }
> > =
>
> > The PB Pojo:
> > =
> > package test.protocolbuffers;
>
> > import java.io.IOException;
> > import java.io.Serializable;
> > import test.protocolbuffers.PojoMessage.Pojo.Builder;
>
> > public class PojoUsingPB implements Serializable {
> > private String key;
> > private String value;
> > private boolean someBoolean;
>
> > public PojoUsingPB(String key, String value, boolean someBoolean)
> > {
> > this.key = key;
> > this.value = value;
> > this.someBoolean = someBoolean;
> > }
>
> > private void writeObject(java.io.ObjectOutputStream out) throws
> > IOException {
> > Builder builder = PojoMessage.Pojo.newBuilder();
> > builder.setKey(key);
> > builder.setValue(value);
> > builder.setSomeBoolean(someBoolean);
> > PojoMessage.Pojo message = builder.build();
> > message.writeTo(out);
> > }
>
> > private void readObject(java.io.ObjectInputStream in)
> > throws ClassNotFoundException, IOException {
> > PojoMessage.Pojo message = PojoMessage.Pojo.parseFrom(in);
> > key = message.getKey();
> > value = message.getValue();
> > someBoolean = message.getSomeBoolean();
> > }
>
> > }
> > =
>
> > The proto file being used in the PB Pojo:
> > =
> > package test.protocolbuffers;
>
> > option op
Re: RMI Performance Tests
What version of protocol buffers are you using? If it's 2.0.3 or previous
then you need to put this line in your proto files:
option optimize_for = SPEED;
In 2.1.0 and up, this is the default.
On Mon, Aug 17, 2009 at 10:29 AM, Tai wrote:
>
> Hi,
>
> I did some simple test to compare Java vs. ProtocolBuffers. I know
> that there is already a very verbose benchmark measuring
> serialization, object creation and de-serialization here:
> http://www.eishay.com/2009/03/more-on-benchmarking-java-serialization.html
>
> But when using RMI I am getting a total time where PB is slower than
> Java. My performance test measures the total time when passing a Pojo
> from a client to a remote agent. The results I get when running the
> tests twice are:
> Average Java time: 147421 nanosecs
> Average PB time: 281617 nanosecs (factor 1.91 slower)
>
> Average Java time: 232446 nanosecs
> Average PB time: 467485 nanosecs (factor 2.01 slower)
>
> I was expecting PB to be faster than Java?! But this seems to be
> wrong.
>
> My classes are (unfortunately I cannot attach to the source / you can
> email me if you want a zip):
>
> The rmi interface for the agent server:
> =
> package test.protocolbuffers;
>
> import java.rmi.Remote;
> import java.rmi.RemoteException;
>
> public interface IRemoteAgent extends Remote {
>
>long doSomething(long start, Object pojo) throws RemoteException;
>
> }
> =
>
> The server agent implementing the interface
> =
> package test.protocolbuffers;
>
> import java.rmi.RemoteException;
> import java.rmi.registry.LocateRegistry;
> import java.rmi.registry.Registry;
> import java.rmi.server.UnicastRemoteObject;
>
> public class RemoteAgent implements IRemoteAgent {
>
>public RemoteAgent() {
>}
>
>public static void main(String[] args) throws Exception {
>RemoteAgent agent = new RemoteAgent();
>String name = "RemoteAgent1";
>UnicastRemoteObject.exportObject(agent, 0);
>Registry registry;
>try {
>registry = LocateRegistry.createRegistry(1099);
>} catch (Exception e) {
>registry = LocateRegistry.getRegistry();
>}
>registry.rebind(name, agent);
>System.out.println(name + " bound.");
>
>}
>
>public long doSomething(long start, Object pojo) throws
> RemoteException {
>long end = System.nanoTime();
>return end - start;
>}
> }
> =
>
> The Java POJO for measuring the using standard Java serialization:
> =
> package test.protocolbuffers;
>
> import java.io.Serializable;
>
> public class PojoUsingJava implements Serializable {
>private String key;
>private String value;
>private boolean someBoolean;
>
>public PojoUsingJava(String key, String value, boolean
> someBoolean) {
>this.key = key;
>this.value = value;
>this.someBoolean = someBoolean;
>}
>
> }
> =
>
> The PB Pojo:
> =
> package test.protocolbuffers;
>
> import java.io.IOException;
> import java.io.Serializable;
> import test.protocolbuffers.PojoMessage.Pojo.Builder;
>
> public class PojoUsingPB implements Serializable {
>private String key;
>private String value;
>private boolean someBoolean;
>
>public PojoUsingPB(String key, String value, boolean someBoolean)
> {
>this.key = key;
>this.value = value;
>this.someBoolean = someBoolean;
>}
>
>private void writeObject(java.io.ObjectOutputStream out) throws
> IOException {
>Builder builder = PojoMessage.Pojo.newBuilder();
>builder.setKey(key);
>builder.setValue(value);
>builder.setSomeBoolean(someBoolean);
>PojoMessage.Pojo message = builder.build();
>message.writeTo(out);
>}
>
>private void readObject(java.io.ObjectInputStream in)
>throws ClassNotFoundException, IOException {
>PojoMessage.Pojo message = PojoMessage.Pojo.parseFrom(in);
>key = message.getKey();
>value = message.getValue();
>someBoolean = message.getSomeBoolean();
>}
>
> }
> =
>
> The proto file being used in the PB Pojo:
> =
> package test.protocolbuffers;
>
> option optimize_for = SPEED;
>
> message Pojo {
>
>optional string key = 1;
>optional string value = 2;
>optional bool someBoolean = 3;
> }=
>
> The test class for measuring the remote calls using Java and PB:
> =
> package test.protocolbuff
