Re: [protobuf] Support for J2ME
Hi Kenton,
*It seems to me that combining them would only be more work for both of us.
I would need you to be available...*
I haven't thought the whole thing through. You're right. It makes more sense
to keep my implementation in a separated project. Thanks for all the
advices.
Thanks,
-Gatis
On Mon, Mar 15, 2010 at 7:54 PM, Kenton Varda ken...@google.com wrote:
I wonder if we should consider making mutable-message mode a feature of the
base implementation, rather than j2me-specific. Some people have requested
this, even though I personally think the builder-based design is better.
On Fri, Mar 12, 2010 at 7:13 PM, Igor Gatis igorga...@gmail.com wrote:
My experience with J2ME says performance is not the most important feature
for the majority of the applications. Trust me when I say JAR size is the
one people care the most.
OK, you would know better than me.
I still need to go over all files to check code compliance with the style
guide and write tests. But besides that, do you think I still need to change
the design?
Based on my high-level look, what you have seems good. But, you don't need
my approval for anything.
Are you still hoping to add this to the official distribution? If so, can
I ask why you prefer this over maintaining your own project? It seems to me
that combining them would only be more work for both of us. I would need
you to be available to update and test your implementation any time I'm
trying to do a release or making Java code generator changes. Also, you
would not be able to release updates to your implementation separately from
the protobuf release cycle, which is probably far too slow for a new
project. With the decentralized approach, you do not have to depend on me
and I do not have to depend on you.
On Fri, Mar 12, 2010 at 9:10 PM, Kenton Varda ken...@google.com wrote:
This may solve the problem but adding code to every setter may have a
significant cost. It's harder to inline the setter this way. But it's hard
to say exactly what the cost will be without some sort of benchmarks.
On Fri, Mar 12, 2010 at 12:12 PM, Igor Gatis igorga...@gmail.comwrote:
I think I have a solution for the readonly messages.
Message.java now includes the following header:
public abstract class Message {
private boolean readOnly;
protected Message(boolean noInit) {
this.readOnly = true;
}
public Message() {}
protected void assertNotReadOnly() {
if (readOnly) {
throw new RuntimeException(Read only message!);
}
}
A generated message, HelloRequest, has:
public final class HelloRequest extends com.google.protobuf.Message {
public HelloRequest() {
initFields();
}
private HelloRequest(boolean noInit) { super(true); }
All methods that can modify HelloRequest look like this:
public void setName(java.lang.String value) {
assertNotReadOnly();
hasName = true;
name_ = value;
}
In other words, that example:
myMessage.getSubMessage().setFoo(1);
Would through an exception. just like
Collections.unmodifiableList(java.util.List)http://java.sun.com/j2se/1.4.2/docs/api/java/util/Collections.html#unmodifiableList(java.util.List)
does.
Do you think this solves the problem?
On Fri, Mar 12, 2010 at 4:16 PM, Igor Gatis igorga...@gmail.comwrote:
myMessage.getSubMessage().setFoo(1);
If they haven't previously called setSubMessage(new SubMessage()) then
this code will actually modify the shared default instance of SubMessage
which could cause all sorts of bugs around the system. Have you
considered
how to avoid this problem?
Uh, not really. But yeah, that's definitely a problem. Let me think
about that. I'll get back to you when I have a solution for this problem.
So the bottom line is that I had to squeeze the runtime to get it as
small as possible - this is a fully functional protobuf runtime
implementation that occupies 26KB against 173KB of standard Java
implementation.
Did you start from the lite implementation or the full one?
26k is pretty impressive.
Yep, I though LITE_RUNTIME would solve all my problems but generics and
for-each loops were a problem.
Flattening the Message class has a few side effects I haven't noticed
until I implemented equals() and hashCode() methods.
A bad effect is that now that there is no way to handle fields in a
generic way, these methods are generated for each message, which makes the
size per message bigger. For a small set of messages, that's not a
problem -
and that is the most common scenario for J2ME apps.
A good one is that obfuscators usually removed unused methods. If user
never uses equals or hashCode, they will be selectively removed. That was
not possible when AbstractMessage was there.
BTW, I removed the generic services generation too.
--
You received this message because you are subscribed to the Google Groups
Protocol Buffers group.
To post to this group, send email to
Re: [protobuf] Support for J2ME
Great! I look forward to seeing it.
On Tue, Mar 16, 2010 at 6:38 AM, Igor Gatis igorga...@gmail.com wrote:
Hi Kenton,
*It seems to me that combining them would only be more work for both of
us. I would need you to be available...*
I haven't thought the whole thing through. You're right. It makes more
sense to keep my implementation in a separated project. Thanks for all the
advices.
Thanks,
-Gatis
On Mon, Mar 15, 2010 at 7:54 PM, Kenton Varda ken...@google.com wrote:
I wonder if we should consider making mutable-message mode a feature of
the base implementation, rather than j2me-specific. Some people have
requested this, even though I personally think the builder-based design is
better.
On Fri, Mar 12, 2010 at 7:13 PM, Igor Gatis igorga...@gmail.com wrote:
My experience with J2ME says performance is not the most important
feature for the majority of the applications. Trust me when I say JAR size
is the one people care the most.
OK, you would know better than me.
I still need to go over all files to check code compliance with the style
guide and write tests. But besides that, do you think I still need to change
the design?
Based on my high-level look, what you have seems good. But, you don't
need my approval for anything.
Are you still hoping to add this to the official distribution? If so, can
I ask why you prefer this over maintaining your own project? It seems to me
that combining them would only be more work for both of us. I would need
you to be available to update and test your implementation any time I'm
trying to do a release or making Java code generator changes. Also, you
would not be able to release updates to your implementation separately from
the protobuf release cycle, which is probably far too slow for a new
project. With the decentralized approach, you do not have to depend on me
and I do not have to depend on you.
On Fri, Mar 12, 2010 at 9:10 PM, Kenton Varda ken...@google.com wrote:
This may solve the problem but adding code to every setter may have a
significant cost. It's harder to inline the setter this way. But it's
hard
to say exactly what the cost will be without some sort of benchmarks.
On Fri, Mar 12, 2010 at 12:12 PM, Igor Gatis igorga...@gmail.comwrote:
I think I have a solution for the readonly messages.
Message.java now includes the following header:
public abstract class Message {
private boolean readOnly;
protected Message(boolean noInit) {
this.readOnly = true;
}
public Message() {}
protected void assertNotReadOnly() {
if (readOnly) {
throw new RuntimeException(Read only message!);
}
}
A generated message, HelloRequest, has:
public final class HelloRequest extends com.google.protobuf.Message {
public HelloRequest() {
initFields();
}
private HelloRequest(boolean noInit) { super(true); }
All methods that can modify HelloRequest look like this:
public void setName(java.lang.String value) {
assertNotReadOnly();
hasName = true;
name_ = value;
}
In other words, that example:
myMessage.getSubMessage().setFoo(1);
Would through an exception. just like
Collections.unmodifiableList(java.util.List)http://java.sun.com/j2se/1.4.2/docs/api/java/util/Collections.html#unmodifiableList(java.util.List)
does.
Do you think this solves the problem?
On Fri, Mar 12, 2010 at 4:16 PM, Igor Gatis igorga...@gmail.comwrote:
myMessage.getSubMessage().setFoo(1);
If they haven't previously called setSubMessage(new SubMessage())
then this code will actually modify the shared default instance of
SubMessage which could cause all sorts of bugs around the system. Have
you
considered how to avoid this problem?
Uh, not really. But yeah, that's definitely a problem. Let me think
about that. I'll get back to you when I have a solution for this problem.
So the bottom line is that I had to squeeze the runtime to get it as
small as possible - this is a fully functional protobuf runtime
implementation that occupies 26KB against 173KB of standard Java
implementation.
Did you start from the lite implementation or the full one?
26k is pretty impressive.
Yep, I though LITE_RUNTIME would solve all my problems but generics
and for-each loops were a problem.
Flattening the Message class has a few side effects I haven't noticed
until I implemented equals() and hashCode() methods.
A bad effect is that now that there is no way to handle fields in a
generic way, these methods are generated for each message, which makes
the
size per message bigger. For a small set of messages, that's not a
problem -
and that is the most common scenario for J2ME apps.
A good one is that obfuscators usually removed unused methods. If user
never uses equals or hashCode, they will be selectively removed. That was
not possible when AbstractMessage was there.
BTW, I removed the generic services generation too.
--
You received this message
Re: [protobuf] Support for J2ME
I wonder if we should consider making mutable-message mode a feature of the
base implementation, rather than j2me-specific. Some people have requested
this, even though I personally think the builder-based design is better.
On Fri, Mar 12, 2010 at 7:13 PM, Igor Gatis igorga...@gmail.com wrote:
My experience with J2ME says performance is not the most important feature
for the majority of the applications. Trust me when I say JAR size is the
one people care the most.
OK, you would know better than me.
I still need to go over all files to check code compliance with the style
guide and write tests. But besides that, do you think I still need to change
the design?
Based on my high-level look, what you have seems good. But, you don't need
my approval for anything.
Are you still hoping to add this to the official distribution? If so, can I
ask why you prefer this over maintaining your own project? It seems to me
that combining them would only be more work for both of us. I would need
you to be available to update and test your implementation any time I'm
trying to do a release or making Java code generator changes. Also, you
would not be able to release updates to your implementation separately from
the protobuf release cycle, which is probably far too slow for a new
project. With the decentralized approach, you do not have to depend on me
and I do not have to depend on you.
On Fri, Mar 12, 2010 at 9:10 PM, Kenton Varda ken...@google.com wrote:
This may solve the problem but adding code to every setter may have a
significant cost. It's harder to inline the setter this way. But it's hard
to say exactly what the cost will be without some sort of benchmarks.
On Fri, Mar 12, 2010 at 12:12 PM, Igor Gatis igorga...@gmail.com wrote:
I think I have a solution for the readonly messages.
Message.java now includes the following header:
public abstract class Message {
private boolean readOnly;
protected Message(boolean noInit) {
this.readOnly = true;
}
public Message() {}
protected void assertNotReadOnly() {
if (readOnly) {
throw new RuntimeException(Read only message!);
}
}
A generated message, HelloRequest, has:
public final class HelloRequest extends com.google.protobuf.Message {
public HelloRequest() {
initFields();
}
private HelloRequest(boolean noInit) { super(true); }
All methods that can modify HelloRequest look like this:
public void setName(java.lang.String value) {
assertNotReadOnly();
hasName = true;
name_ = value;
}
In other words, that example:
myMessage.getSubMessage().setFoo(1);
Would through an exception. just like
Collections.unmodifiableList(java.util.List)http://java.sun.com/j2se/1.4.2/docs/api/java/util/Collections.html#unmodifiableList(java.util.List)
does.
Do you think this solves the problem?
On Fri, Mar 12, 2010 at 4:16 PM, Igor Gatis igorga...@gmail.com wrote:
myMessage.getSubMessage().setFoo(1);
If they haven't previously called setSubMessage(new SubMessage()) then
this code will actually modify the shared default instance of SubMessage
which could cause all sorts of bugs around the system. Have you
considered
how to avoid this problem?
Uh, not really. But yeah, that's definitely a problem. Let me think
about that. I'll get back to you when I have a solution for this problem.
So the bottom line is that I had to squeeze the runtime to get it as
small as possible - this is a fully functional protobuf runtime
implementation that occupies 26KB against 173KB of standard Java
implementation.
Did you start from the lite implementation or the full one?
26k is pretty impressive.
Yep, I though LITE_RUNTIME would solve all my problems but generics and
for-each loops were a problem.
Flattening the Message class has a few side effects I haven't noticed
until I implemented equals() and hashCode() methods.
A bad effect is that now that there is no way to handle fields in a
generic way, these methods are generated for each message, which makes the
size per message bigger. For a small set of messages, that's not a problem
-
and that is the most common scenario for J2ME apps.
A good one is that obfuscators usually removed unused methods. If user
never uses equals or hashCode, they will be selectively removed. That was
not possible when AbstractMessage was there.
BTW, I removed the generic services generation too.
--
You received this message because you are subscribed to the Google Groups
Protocol Buffers group.
To post to this group, send email to proto...@googlegroups.com.
To unsubscribe from this group, send email to
protobuf+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/protobuf?hl=en.
Re: [protobuf] Support for J2ME
I've uploaded my latest version.
http://codereview.appspot.com/445041/show
On Fri, Mar 12, 2010 at 11:13 PM, Igor Gatis igorga...@gmail.com wrote:
My experience with J2ME says performance is not the most important feature
for the majority of the applications. Trust me when I say JAR size is the
one people care the most.
Besides, the extra cost is a method call + condition check, that's fairly
cheap. Depending on the compiler and/or obfuscator, that condition could
even be inlined saving the extra method call.
I still need to go over all files to check code compliance with the style
guide and write tests. But besides that, do you think I still need to change
the design?
On Fri, Mar 12, 2010 at 9:10 PM, Kenton Varda ken...@google.com wrote:
This may solve the problem but adding code to every setter may have a
significant cost. It's harder to inline the setter this way. But it's hard
to say exactly what the cost will be without some sort of benchmarks.
On Fri, Mar 12, 2010 at 12:12 PM, Igor Gatis igorga...@gmail.com wrote:
I think I have a solution for the readonly messages.
Message.java now includes the following header:
public abstract class Message {
private boolean readOnly;
protected Message(boolean noInit) {
this.readOnly = true;
}
public Message() {}
protected void assertNotReadOnly() {
if (readOnly) {
throw new RuntimeException(Read only message!);
}
}
A generated message, HelloRequest, has:
public final class HelloRequest extends com.google.protobuf.Message {
public HelloRequest() {
initFields();
}
private HelloRequest(boolean noInit) { super(true); }
All methods that can modify HelloRequest look like this:
public void setName(java.lang.String value) {
assertNotReadOnly();
hasName = true;
name_ = value;
}
In other words, that example:
myMessage.getSubMessage().setFoo(1);
Would through an exception. just like
Collections.unmodifiableList(java.util.List)http://java.sun.com/j2se/1.4.2/docs/api/java/util/Collections.html#unmodifiableList(java.util.List)
does.
Do you think this solves the problem?
On Fri, Mar 12, 2010 at 4:16 PM, Igor Gatis igorga...@gmail.com wrote:
myMessage.getSubMessage().setFoo(1);
If they haven't previously called setSubMessage(new SubMessage()) then
this code will actually modify the shared default instance of SubMessage
which could cause all sorts of bugs around the system. Have you
considered
how to avoid this problem?
Uh, not really. But yeah, that's definitely a problem. Let me think
about that. I'll get back to you when I have a solution for this problem.
So the bottom line is that I had to squeeze the runtime to get it as
small as possible - this is a fully functional protobuf runtime
implementation that occupies 26KB against 173KB of standard Java
implementation.
Did you start from the lite implementation or the full one?
26k is pretty impressive.
Yep, I though LITE_RUNTIME would solve all my problems but generics and
for-each loops were a problem.
Flattening the Message class has a few side effects I haven't noticed
until I implemented equals() and hashCode() methods.
A bad effect is that now that there is no way to handle fields in a
generic way, these methods are generated for each message, which makes the
size per message bigger. For a small set of messages, that's not a problem
-
and that is the most common scenario for J2ME apps.
A good one is that obfuscators usually removed unused methods. If user
never uses equals or hashCode, they will be selectively removed. That was
not possible when AbstractMessage was there.
BTW, I removed the generic services generation too.
--
You received this message because you are subscribed to the Google Groups
Protocol Buffers group.
To post to this group, send email to proto...@googlegroups.com.
To unsubscribe from this group, send email to
protobuf+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/protobuf?hl=en.
Re: [protobuf] Support for J2ME
On Thu, Mar 11, 2010 at 1:52 PM, Igor Gatis igorga...@gmail.com wrote: *- Why did you eliminate the builder pattern?* To save jar space. J2ME environment is pretty restricted. Many devices have a few kilo bytes size limit (e.g 128K, 256K). An empty class adds about 200 bytes to jar file. The long time I worked with J2ME convinced me the less classes the better. I tried to keep it as simple as possible without hurting the OO design. Hmm, I'm surprised that you could even get anything resembling the Java protobuf implementation to fit in such a small space. You could possibly get it even smaller by using POJOs (avoid getters and setters). One thing that worries me about removing the builder pattern is that it changes the whole usage model in a way that could affect other design decisions. For example, if you have a message-typed field which has not been set yet, the getter for that field should return the message type's default instance. This was fine when default instances were immutable, but now that they are mutable, you might have bugs where someone writes: myMessage.getSubMessage().setFoo(1); If they haven't previously called setSubMessage(new SubMessage()) then this code will actually modify the shared default instance of SubMessage which could cause all sorts of bugs around the system. Have you considered how to avoid this problem? *- Why did you choose to implement generic services when this feature is deprecated anyway?* That was overseen. I need to fix that then. *Your diff shows the runtime classes as if they were completely new. Could you re-arrange the files so that it actually compares your versions of these classes against the official ones?* The some official java sources are not Java 1.3 compatible (for instance, they use generics and for-each loops). I've also flattened some of the classes hierarchy (e.g. Message vs AbstractMessage + GeneratedMessage + MessageLite, etc) to save jar space. I've also removed dependency to Descriptors so I had to change BlockingRpcChannel and RpcChannel to receive the method as a String rather than a descriptor. So that being said, the new set of files are tailored to J2ME environment and its limitations. So the bottom line is that I had to squeeze the runtime to get it as small as possible - this is a fully functional protobuf runtime implementation that occupies 26KB against 173KB of standard Java implementation. Did you start from the lite implementation or the full one? 26k is pretty impressive. On Thu, Mar 11, 2010 at 4:26 PM, Igor Gatis igorga...@gmail.com wrote: What exactly is deprecated in the services API? Is it only the Service and BlockingService interfaces? We are now encouraging RPC implementations to provide their own code generator plugins rather than work with the generic services. This way implementations can tailor the generated code to their purposes. Note that services were never part of the lite implementation. -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to proto...@googlegroups.com. To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/protobuf?hl=en.
Re: [protobuf] Support for J2ME
myMessage.getSubMessage().setFoo(1); If they haven't previously called setSubMessage(new SubMessage()) then this code will actually modify the shared default instance of SubMessage which could cause all sorts of bugs around the system. Have you considered how to avoid this problem? Uh, not really. But yeah, that's definitely a problem. Let me think about that. I'll get back to you when I have a solution for this problem. So the bottom line is that I had to squeeze the runtime to get it as small as possible - this is a fully functional protobuf runtime implementation that occupies 26KB against 173KB of standard Java implementation. Did you start from the lite implementation or the full one? 26k is pretty impressive. Yep, I though LITE_RUNTIME would solve all my problems but generics and for-each loops were a problem. Flattening the Message class has a few side effects I haven't noticed until I implemented equals() and hashCode() methods. A bad effect is that now that there is no way to handle fields in a generic way, these methods are generated for each message, which makes the size per message bigger. For a small set of messages, that's not a problem - and that is the most common scenario for J2ME apps. A good one is that obfuscators usually removed unused methods. If user never uses equals or hashCode, they will be selectively removed. That was not possible when AbstractMessage was there. BTW, I removed the generic services generation too. -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to proto...@googlegroups.com. To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/protobuf?hl=en.
Re: [protobuf] Support for J2ME
I think I have a solution for the readonly messages.
Message.java now includes the following header:
public abstract class Message {
private boolean readOnly;
protected Message(boolean noInit) {
this.readOnly = true;
}
public Message() {}
protected void assertNotReadOnly() {
if (readOnly) {
throw new RuntimeException(Read only message!);
}
}
A generated message, HelloRequest, has:
public final class HelloRequest extends com.google.protobuf.Message {
public HelloRequest() {
initFields();
}
private HelloRequest(boolean noInit) { super(true); }
All methods that can modify HelloRequest look like this:
public void setName(java.lang.String value) {
assertNotReadOnly();
hasName = true;
name_ = value;
}
In other words, that example:
myMessage.getSubMessage().setFoo(1);
Would through an exception. just like
Collections.unmodifiableList(java.util.List)http://java.sun.com/j2se/1.4.2/docs/api/java/util/Collections.html#unmodifiableList(java.util.List)
does.
Do you think this solves the problem?
On Fri, Mar 12, 2010 at 4:16 PM, Igor Gatis igorga...@gmail.com wrote:
myMessage.getSubMessage().setFoo(1);
If they haven't previously called setSubMessage(new SubMessage()) then
this code will actually modify the shared default instance of SubMessage
which could cause all sorts of bugs around the system. Have you considered
how to avoid this problem?
Uh, not really. But yeah, that's definitely a problem. Let me think about
that. I'll get back to you when I have a solution for this problem.
So the bottom line is that I had to squeeze the runtime to get it as
small as possible - this is a fully functional protobuf runtime
implementation that occupies 26KB against 173KB of standard Java
implementation.
Did you start from the lite implementation or the full one?
26k is pretty impressive.
Yep, I though LITE_RUNTIME would solve all my problems but generics and
for-each loops were a problem.
Flattening the Message class has a few side effects I haven't noticed until
I implemented equals() and hashCode() methods.
A bad effect is that now that there is no way to handle fields in a generic
way, these methods are generated for each message, which makes the size per
message bigger. For a small set of messages, that's not a problem - and that
is the most common scenario for J2ME apps.
A good one is that obfuscators usually removed unused methods. If user
never uses equals or hashCode, they will be selectively removed. That was
not possible when AbstractMessage was there.
BTW, I removed the generic services generation too.
--
You received this message because you are subscribed to the Google Groups
Protocol Buffers group.
To post to this group, send email to proto...@googlegroups.com.
To unsubscribe from this group, send email to
protobuf+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/protobuf?hl=en.
Re: [protobuf] Support for J2ME
This may solve the problem but adding code to every setter may have a
significant cost. It's harder to inline the setter this way. But it's hard
to say exactly what the cost will be without some sort of benchmarks.
On Fri, Mar 12, 2010 at 12:12 PM, Igor Gatis igorga...@gmail.com wrote:
I think I have a solution for the readonly messages.
Message.java now includes the following header:
public abstract class Message {
private boolean readOnly;
protected Message(boolean noInit) {
this.readOnly = true;
}
public Message() {}
protected void assertNotReadOnly() {
if (readOnly) {
throw new RuntimeException(Read only message!);
}
}
A generated message, HelloRequest, has:
public final class HelloRequest extends com.google.protobuf.Message {
public HelloRequest() {
initFields();
}
private HelloRequest(boolean noInit) { super(true); }
All methods that can modify HelloRequest look like this:
public void setName(java.lang.String value) {
assertNotReadOnly();
hasName = true;
name_ = value;
}
In other words, that example:
myMessage.getSubMessage().setFoo(1);
Would through an exception. just like
Collections.unmodifiableList(java.util.List)http://java.sun.com/j2se/1.4.2/docs/api/java/util/Collections.html#unmodifiableList(java.util.List)
does.
Do you think this solves the problem?
On Fri, Mar 12, 2010 at 4:16 PM, Igor Gatis igorga...@gmail.com wrote:
myMessage.getSubMessage().setFoo(1);
If they haven't previously called setSubMessage(new SubMessage()) then
this code will actually modify the shared default instance of SubMessage
which could cause all sorts of bugs around the system. Have you considered
how to avoid this problem?
Uh, not really. But yeah, that's definitely a problem. Let me think about
that. I'll get back to you when I have a solution for this problem.
So the bottom line is that I had to squeeze the runtime to get it as
small as possible - this is a fully functional protobuf runtime
implementation that occupies 26KB against 173KB of standard Java
implementation.
Did you start from the lite implementation or the full one?
26k is pretty impressive.
Yep, I though LITE_RUNTIME would solve all my problems but generics and
for-each loops were a problem.
Flattening the Message class has a few side effects I haven't noticed
until I implemented equals() and hashCode() methods.
A bad effect is that now that there is no way to handle fields in a
generic way, these methods are generated for each message, which makes the
size per message bigger. For a small set of messages, that's not a problem -
and that is the most common scenario for J2ME apps.
A good one is that obfuscators usually removed unused methods. If user
never uses equals or hashCode, they will be selectively removed. That was
not possible when AbstractMessage was there.
BTW, I removed the generic services generation too.
--
You received this message because you are subscribed to the Google Groups
Protocol Buffers group.
To post to this group, send email to proto...@googlegroups.com.
To unsubscribe from this group, send email to
protobuf+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/protobuf?hl=en.
Re: [protobuf] Support for J2ME
My experience with J2ME says performance is not the most important feature
for the majority of the applications. Trust me when I say JAR size is the
one people care the most.
Besides, the extra cost is a method call + condition check, that's fairly
cheap. Depending on the compiler and/or obfuscator, that condition could
even be inlined saving the extra method call.
I still need to go over all files to check code compliance with the style
guide and write tests. But besides that, do you think I still need to change
the design?
On Fri, Mar 12, 2010 at 9:10 PM, Kenton Varda ken...@google.com wrote:
This may solve the problem but adding code to every setter may have a
significant cost. It's harder to inline the setter this way. But it's hard
to say exactly what the cost will be without some sort of benchmarks.
On Fri, Mar 12, 2010 at 12:12 PM, Igor Gatis igorga...@gmail.com wrote:
I think I have a solution for the readonly messages.
Message.java now includes the following header:
public abstract class Message {
private boolean readOnly;
protected Message(boolean noInit) {
this.readOnly = true;
}
public Message() {}
protected void assertNotReadOnly() {
if (readOnly) {
throw new RuntimeException(Read only message!);
}
}
A generated message, HelloRequest, has:
public final class HelloRequest extends com.google.protobuf.Message {
public HelloRequest() {
initFields();
}
private HelloRequest(boolean noInit) { super(true); }
All methods that can modify HelloRequest look like this:
public void setName(java.lang.String value) {
assertNotReadOnly();
hasName = true;
name_ = value;
}
In other words, that example:
myMessage.getSubMessage().setFoo(1);
Would through an exception. just like
Collections.unmodifiableList(java.util.List)http://java.sun.com/j2se/1.4.2/docs/api/java/util/Collections.html#unmodifiableList(java.util.List)
does.
Do you think this solves the problem?
On Fri, Mar 12, 2010 at 4:16 PM, Igor Gatis igorga...@gmail.com wrote:
myMessage.getSubMessage().setFoo(1);
If they haven't previously called setSubMessage(new SubMessage()) then
this code will actually modify the shared default instance of SubMessage
which could cause all sorts of bugs around the system. Have you considered
how to avoid this problem?
Uh, not really. But yeah, that's definitely a problem. Let me think about
that. I'll get back to you when I have a solution for this problem.
So the bottom line is that I had to squeeze the runtime to get it as
small as possible - this is a fully functional protobuf runtime
implementation that occupies 26KB against 173KB of standard Java
implementation.
Did you start from the lite implementation or the full one?
26k is pretty impressive.
Yep, I though LITE_RUNTIME would solve all my problems but generics and
for-each loops were a problem.
Flattening the Message class has a few side effects I haven't noticed
until I implemented equals() and hashCode() methods.
A bad effect is that now that there is no way to handle fields in a
generic way, these methods are generated for each message, which makes the
size per message bigger. For a small set of messages, that's not a problem -
and that is the most common scenario for J2ME apps.
A good one is that obfuscators usually removed unused methods. If user
never uses equals or hashCode, they will be selectively removed. That was
not possible when AbstractMessage was there.
BTW, I removed the generic services generation too.
--
You received this message because you are subscribed to the Google Groups
Protocol Buffers group.
To post to this group, send email to proto...@googlegroups.com.
To unsubscribe from this group, send email to
protobuf+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/protobuf?hl=en.
Re: [protobuf] Support for J2ME
It is a stripped copy of the official Java implementation. Here are the list of files: BlockingRpcChannel.java BlockingService.java ByteString.java CodedInputStream.java CodedOutputStream.java InvalidProtocolBufferException.java Message.java RpcCallback.java RpcChannel.java RpcController.java ServiceException.java WireFormat.java Limitations: 1) Biggest limitation is the lack of support for extensions. I plan to fix it soon. 2) Also, for J2ME devices, (asynchronous) Services is not a huge requirement, so it is not there too (notice there is no Service.java in this list). As for the compiler, there is A LOT of shared code. I'm sending you a code review. Thanks, -Gatis On Wed, Mar 10, 2010 at 1:17 AM, Kenton Varda ken...@google.com wrote: Is your implementation a modification of the official Java implementation? If so, how much is shared? Could you send me a diff via codereview.appspot.com? On Tue, Mar 9, 2010 at 5:46 PM, Igor Gatis igorga...@gmail.com wrote: So it is totally out of the scope of this project to add such support? The changes I've made seem to match with the current architecture. I've added extended protoc to accept a new file option java_platform (where user may specify J2SE(default) or J2ME) which makes it produce J2ME compatible sources. On Tue, Mar 9, 2010 at 10:29 PM, Kenton Varda ken...@google.com wrote: There are three J2ME implementations on the third-party list: http://code.google.com/p/protobuf/wiki/ThirdPartyAddOns I'd be happy to list yours as well if you'd like. On Tue, Mar 9, 2010 at 5:17 PM, Igor Gatis igorga...@gmail.com wrote: Hi protobuf team, Have you guys considered supporting J2ME? I've extended protoc to generate J2ME compatible code and built its supporting J2ME runtime library. I was wondering if you guys would consider reviewing my changes. Thanks, -Gatis -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to proto...@googlegroups.com. To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.comprotobuf%2bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/protobuf?hl=en. -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to proto...@googlegroups.com. To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/protobuf?hl=en.
Re: [protobuf] Support for J2ME
Here is the code review: http://codereview.appspot.com/445041/show On Thu, Mar 11, 2010 at 2:09 PM, Igor Gatis igorga...@gmail.com wrote: It is a stripped copy of the official Java implementation. Here are the list of files: BlockingRpcChannel.java BlockingService.java ByteString.java CodedInputStream.java CodedOutputStream.java InvalidProtocolBufferException.java Message.java RpcCallback.java RpcChannel.java RpcController.java ServiceException.java WireFormat.java Limitations: 1) Biggest limitation is the lack of support for extensions. I plan to fix it soon. 2) Also, for J2ME devices, (asynchronous) Services is not a huge requirement, so it is not there too (notice there is no Service.java in this list). As for the compiler, there is A LOT of shared code. I'm sending you a code review. Thanks, -Gatis On Wed, Mar 10, 2010 at 1:17 AM, Kenton Varda ken...@google.com wrote: Is your implementation a modification of the official Java implementation? If so, how much is shared? Could you send me a diff via codereview.appspot.com? On Tue, Mar 9, 2010 at 5:46 PM, Igor Gatis igorga...@gmail.com wrote: So it is totally out of the scope of this project to add such support? The changes I've made seem to match with the current architecture. I've added extended protoc to accept a new file option java_platform (where user may specify J2SE(default) or J2ME) which makes it produce J2ME compatible sources. On Tue, Mar 9, 2010 at 10:29 PM, Kenton Varda ken...@google.com wrote: There are three J2ME implementations on the third-party list: http://code.google.com/p/protobuf/wiki/ThirdPartyAddOns I'd be happy to list yours as well if you'd like. On Tue, Mar 9, 2010 at 5:17 PM, Igor Gatis igorga...@gmail.com wrote: Hi protobuf team, Have you guys considered supporting J2ME? I've extended protoc to generate J2ME compatible code and built its supporting J2ME runtime library. I was wondering if you guys would consider reviewing my changes. Thanks, -Gatis -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to proto...@googlegroups.com. To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.comprotobuf%2bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/protobuf?hl=en. -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to proto...@googlegroups.com. To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/protobuf?hl=en.
Re: [protobuf] Support for J2ME
Can you explain some of the design decisions: - Why did you eliminate the builder pattern? - Why did you choose to implement generic services when this feature is deprecated anyway? Your diff shows the runtime classes as if they were completely new. Could you re-arrange the files so that it actually compares your versions of these classes against the official ones? = As far as making this part of the official project, my worry is about maintenance challenges. It will be much harder for us to make changes to the code generator if we always have to consider J2ME. For example, when coupled changes to the code generator and runtime library, unless the change is in a part that doesn't apply to J2ME, we would now have to update the J2ME library too. Not only does this require modifying more files, but it requires setting up J2ME tools for testing -- and our internal build system does not have any support for this AFAIK. So my thought is that the best way to run this is as a branch. Changes to the official implementation would not affect J2ME until the J2ME branch maintainers choose to update it, and then they could make sure those changes make sense. On Thu, Mar 11, 2010 at 9:16 AM, Igor Gatis igorga...@gmail.com wrote: Here is the code review: http://codereview.appspot.com/445041/show On Thu, Mar 11, 2010 at 2:09 PM, Igor Gatis igorga...@gmail.com wrote: It is a stripped copy of the official Java implementation. Here are the list of files: BlockingRpcChannel.java BlockingService.java ByteString.java CodedInputStream.java CodedOutputStream.java InvalidProtocolBufferException.java Message.java RpcCallback.java RpcChannel.java RpcController.java ServiceException.java WireFormat.java Limitations: 1) Biggest limitation is the lack of support for extensions. I plan to fix it soon. 2) Also, for J2ME devices, (asynchronous) Services is not a huge requirement, so it is not there too (notice there is no Service.java in this list). As for the compiler, there is A LOT of shared code. I'm sending you a code review. Thanks, -Gatis On Wed, Mar 10, 2010 at 1:17 AM, Kenton Varda ken...@google.com wrote: Is your implementation a modification of the official Java implementation? If so, how much is shared? Could you send me a diff via codereview.appspot.com? On Tue, Mar 9, 2010 at 5:46 PM, Igor Gatis igorga...@gmail.com wrote: So it is totally out of the scope of this project to add such support? The changes I've made seem to match with the current architecture. I've added extended protoc to accept a new file option java_platform (where user may specify J2SE(default) or J2ME) which makes it produce J2ME compatible sources. On Tue, Mar 9, 2010 at 10:29 PM, Kenton Varda ken...@google.comwrote: There are three J2ME implementations on the third-party list: http://code.google.com/p/protobuf/wiki/ThirdPartyAddOns I'd be happy to list yours as well if you'd like. On Tue, Mar 9, 2010 at 5:17 PM, Igor Gatis igorga...@gmail.comwrote: Hi protobuf team, Have you guys considered supporting J2ME? I've extended protoc to generate J2ME compatible code and built its supporting J2ME runtime library. I was wondering if you guys would consider reviewing my changes. Thanks, -Gatis -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to proto...@googlegroups.com. To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.comprotobuf%2bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/protobuf?hl=en. -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to proto...@googlegroups.com. To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/protobuf?hl=en.
Re: [protobuf] Support for J2ME
*- Why did you eliminate the builder pattern?* To save jar space. J2ME environment is pretty restricted. Many devices have a few kilo bytes size limit (e.g 128K, 256K). An empty class adds about 200 bytes to jar file. The long time I worked with J2ME convinced me the less classes the better. I tried to keep it as simple as possible without hurting the OO design. *- Why did you choose to implement generic services when this feature is deprecated anyway?* That was overseen. I need to fix that then. *Your diff shows the runtime classes as if they were completely new. Could you re-arrange the files so that it actually compares your versions of these classes against the official ones?* The some official java sources are not Java 1.3 compatible (for instance, they use generics and for-each loops). I've also flattened some of the classes hierarchy (e.g. Message vs AbstractMessage + GeneratedMessage + MessageLite, etc) to save jar space. I've also removed dependency to Descriptors so I had to change BlockingRpcChannel and RpcChannel to receive the method as a String rather than a descriptor. So that being said, the new set of files are tailored to J2ME environment and its limitations. So the bottom line is that I had to squeeze the runtime to get it as small as possible - this is a fully functional protobuf runtime implementation that occupies 26KB against 173KB of standard Java implementation. Does that make sense? On Thu, Mar 11, 2010 at 4:20 PM, Kenton Varda ken...@google.com wrote: Can you explain some of the design decisions: - Why did you eliminate the builder pattern? - Why did you choose to implement generic services when this feature is deprecated anyway? Your diff shows the runtime classes as if they were completely new. Could you re-arrange the files so that it actually compares your versions of these classes against the official ones? = As far as making this part of the official project, my worry is about maintenance challenges. It will be much harder for us to make changes to the code generator if we always have to consider J2ME. For example, when coupled changes to the code generator and runtime library, unless the change is in a part that doesn't apply to J2ME, we would now have to update the J2ME library too. Not only does this require modifying more files, but it requires setting up J2ME tools for testing -- and our internal build system does not have any support for this AFAIK. So my thought is that the best way to run this is as a branch. Changes to the official implementation would not affect J2ME until the J2ME branch maintainers choose to update it, and then they could make sure those changes make sense. On Thu, Mar 11, 2010 at 9:16 AM, Igor Gatis igorga...@gmail.com wrote: Here is the code review: http://codereview.appspot.com/445041/show On Thu, Mar 11, 2010 at 2:09 PM, Igor Gatis igorga...@gmail.com wrote: It is a stripped copy of the official Java implementation. Here are the list of files: BlockingRpcChannel.java BlockingService.java ByteString.java CodedInputStream.java CodedOutputStream.java InvalidProtocolBufferException.java Message.java RpcCallback.java RpcChannel.java RpcController.java ServiceException.java WireFormat.java Limitations: 1) Biggest limitation is the lack of support for extensions. I plan to fix it soon. 2) Also, for J2ME devices, (asynchronous) Services is not a huge requirement, so it is not there too (notice there is no Service.java in this list). As for the compiler, there is A LOT of shared code. I'm sending you a code review. Thanks, -Gatis On Wed, Mar 10, 2010 at 1:17 AM, Kenton Varda ken...@google.com wrote: Is your implementation a modification of the official Java implementation? If so, how much is shared? Could you send me a diff via codereview.appspot.com? On Tue, Mar 9, 2010 at 5:46 PM, Igor Gatis igorga...@gmail.com wrote: So it is totally out of the scope of this project to add such support? The changes I've made seem to match with the current architecture. I've added extended protoc to accept a new file option java_platform (where user may specify J2SE(default) or J2ME) which makes it produce J2ME compatible sources. On Tue, Mar 9, 2010 at 10:29 PM, Kenton Varda ken...@google.comwrote: There are three J2ME implementations on the third-party list: http://code.google.com/p/protobuf/wiki/ThirdPartyAddOns I'd be happy to list yours as well if you'd like. On Tue, Mar 9, 2010 at 5:17 PM, Igor Gatis igorga...@gmail.comwrote: Hi protobuf team, Have you guys considered supporting J2ME? I've extended protoc to generate J2ME compatible code and built its supporting J2ME runtime library. I was wondering if you guys would consider reviewing my changes. Thanks, -Gatis -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to proto...@googlegroups.com. To
Re: [protobuf] Support for J2ME
What exactly is deprecated in the services API? Is it only the Service and BlockingService interfaces? On Thu, Mar 11, 2010 at 6:52 PM, Igor Gatis igorga...@gmail.com wrote: *- Why did you eliminate the builder pattern?* To save jar space. J2ME environment is pretty restricted. Many devices have a few kilo bytes size limit (e.g 128K, 256K). An empty class adds about 200 bytes to jar file. The long time I worked with J2ME convinced me the less classes the better. I tried to keep it as simple as possible without hurting the OO design. *- Why did you choose to implement generic services when this feature is deprecated anyway?* That was overseen. I need to fix that then. *Your diff shows the runtime classes as if they were completely new. Could you re-arrange the files so that it actually compares your versions of these classes against the official ones?* The some official java sources are not Java 1.3 compatible (for instance, they use generics and for-each loops). I've also flattened some of the classes hierarchy (e.g. Message vs AbstractMessage + GeneratedMessage + MessageLite, etc) to save jar space. I've also removed dependency to Descriptors so I had to change BlockingRpcChannel and RpcChannel to receive the method as a String rather than a descriptor. So that being said, the new set of files are tailored to J2ME environment and its limitations. So the bottom line is that I had to squeeze the runtime to get it as small as possible - this is a fully functional protobuf runtime implementation that occupies 26KB against 173KB of standard Java implementation. Does that make sense? On Thu, Mar 11, 2010 at 4:20 PM, Kenton Varda ken...@google.com wrote: Can you explain some of the design decisions: - Why did you eliminate the builder pattern? - Why did you choose to implement generic services when this feature is deprecated anyway? Your diff shows the runtime classes as if they were completely new. Could you re-arrange the files so that it actually compares your versions of these classes against the official ones? = As far as making this part of the official project, my worry is about maintenance challenges. It will be much harder for us to make changes to the code generator if we always have to consider J2ME. For example, when coupled changes to the code generator and runtime library, unless the change is in a part that doesn't apply to J2ME, we would now have to update the J2ME library too. Not only does this require modifying more files, but it requires setting up J2ME tools for testing -- and our internal build system does not have any support for this AFAIK. So my thought is that the best way to run this is as a branch. Changes to the official implementation would not affect J2ME until the J2ME branch maintainers choose to update it, and then they could make sure those changes make sense. On Thu, Mar 11, 2010 at 9:16 AM, Igor Gatis igorga...@gmail.com wrote: Here is the code review: http://codereview.appspot.com/445041/show On Thu, Mar 11, 2010 at 2:09 PM, Igor Gatis igorga...@gmail.com wrote: It is a stripped copy of the official Java implementation. Here are the list of files: BlockingRpcChannel.java BlockingService.java ByteString.java CodedInputStream.java CodedOutputStream.java InvalidProtocolBufferException.java Message.java RpcCallback.java RpcChannel.java RpcController.java ServiceException.java WireFormat.java Limitations: 1) Biggest limitation is the lack of support for extensions. I plan to fix it soon. 2) Also, for J2ME devices, (asynchronous) Services is not a huge requirement, so it is not there too (notice there is no Service.java in this list). As for the compiler, there is A LOT of shared code. I'm sending you a code review. Thanks, -Gatis On Wed, Mar 10, 2010 at 1:17 AM, Kenton Varda ken...@google.comwrote: Is your implementation a modification of the official Java implementation? If so, how much is shared? Could you send me a diff via codereview.appspot.com? On Tue, Mar 9, 2010 at 5:46 PM, Igor Gatis igorga...@gmail.comwrote: So it is totally out of the scope of this project to add such support? The changes I've made seem to match with the current architecture. I've added extended protoc to accept a new file option java_platform (where user may specify J2SE(default) or J2ME) which makes it produce J2ME compatible sources. On Tue, Mar 9, 2010 at 10:29 PM, Kenton Varda ken...@google.comwrote: There are three J2ME implementations on the third-party list: http://code.google.com/p/protobuf/wiki/ThirdPartyAddOns I'd be happy to list yours as well if you'd like. On Tue, Mar 9, 2010 at 5:17 PM, Igor Gatis igorga...@gmail.comwrote: Hi protobuf team, Have you guys considered supporting J2ME? I've extended protoc to generate J2ME compatible code and built its supporting J2ME runtime library. I was wondering if you guys would consider reviewing my
Re: [protobuf] Support for J2ME
There are three J2ME implementations on the third-party list: http://code.google.com/p/protobuf/wiki/ThirdPartyAddOns I'd be happy to list yours as well if you'd like. On Tue, Mar 9, 2010 at 5:17 PM, Igor Gatis igorga...@gmail.com wrote: Hi protobuf team, Have you guys considered supporting J2ME? I've extended protoc to generate J2ME compatible code and built its supporting J2ME runtime library. I was wondering if you guys would consider reviewing my changes. Thanks, -Gatis -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to proto...@googlegroups.com. To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.comprotobuf%2bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/protobuf?hl=en. -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to proto...@googlegroups.com. To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/protobuf?hl=en.
Re: [protobuf] Support for J2ME
Is your implementation a modification of the official Java implementation? If so, how much is shared? Could you send me a diff via codereview.appspot.com? On Tue, Mar 9, 2010 at 5:46 PM, Igor Gatis igorga...@gmail.com wrote: So it is totally out of the scope of this project to add such support? The changes I've made seem to match with the current architecture. I've added extended protoc to accept a new file option java_platform (where user may specify J2SE(default) or J2ME) which makes it produce J2ME compatible sources. On Tue, Mar 9, 2010 at 10:29 PM, Kenton Varda ken...@google.com wrote: There are three J2ME implementations on the third-party list: http://code.google.com/p/protobuf/wiki/ThirdPartyAddOns I'd be happy to list yours as well if you'd like. On Tue, Mar 9, 2010 at 5:17 PM, Igor Gatis igorga...@gmail.com wrote: Hi protobuf team, Have you guys considered supporting J2ME? I've extended protoc to generate J2ME compatible code and built its supporting J2ME runtime library. I was wondering if you guys would consider reviewing my changes. Thanks, -Gatis -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to proto...@googlegroups.com. To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.comprotobuf%2bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/protobuf?hl=en. -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to proto...@googlegroups.com. To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/protobuf?hl=en.
