Re: [protobuf] Protobuf possible new features
Le mercredi 13 mai 2015 22:35:47 UTC+2, Feng Xiao a écrit : On Tue, May 12, 2015 at 7:07 AM, hyp...@donarproject.org javascript: wrote: Hello Protobuf team, first of all thank you for your great Job. We have been internally using protobuf in a big financial software company for a while and is has proven to be the best choice so far. As our need for multiplatform is growing, we have modified Protobuf and added some features. We reached a point where we though about sharing our need features and see if they can be integrated into the Protobuf mainline. Here are the features : - Generate a new interface for mutable proto objects: *OrBuilder.IBuilder. - Write interfaces (*OrBuilder and *.IBuilder) into separate files. This allows us to package interfaces into a standalone jar that can be referenced from modules that don't need to depend on the implementation. - Generate services for protobuf nano: we have an Android client using our internal RPC system built around protobuf services. - Be able to serialize and parse Proto objects to and from JSON strings. It seems this feature is partially implemented in proto3. - Generate GWT overlays and create a gwt-compatible protobuf jar (like the guava-gwt jar). The gwt-compatible protobuf only has the protobuf nano features. We have been exchanging with Ray Cromwell at the GWT.create conference this year and it seems that some teams at Google have developed there own JSON parser and overlay generator. Are those features relevant for integration into the protobuf mainline? Thanks for reaching out to us. We very much appreciate your contribution to protobuf, but unfortunately as protobuf is used very extensively inside Google, it's unlikely for us to integrate these changes because they either duplicate what we already have inside Google, or conflict with our view of how protobuf API should be. As what you have heard from Ray, internally inside Google we already have JSON serializers/parsers implemented. Actually we not only have one, but have several JSON implementations using different styles developed by different teams. For C++ and Java, I know of at least 3 JSON implementations for each of them. They do not get opensourced with protobuf because JSON is not part of protobuf library before proto3. For other features you mentioned, we also already have them internally. Some of them will get opensourced eventually as part of proto3 but most of them will not (or will be opensourced as a different project). JSON will be included in proto3 so we will select and open-source one of the JSON implementations we have been using internally. We have mutable API for Java implemented, but it will not be opensourced (you can find some traces of the mutable API here https://github.com/google/protobuf/blob/6ef984af4b0c63c1c33127a12dcfc8e6359f0c9e/java/src/main/java/com/google/protobuf/Extension.java#L56). For RPC, it will be part of the gRPC project: http://www.grpc.io/. Although we probably could not integrate your changes directly into protobuf, I think we can still explore whether we can make changes in protobuf in a way that would either eliminate the need to adding these features yourself, or make it easier for these features to interact with the protobuf library. As I already mentioned, we'll have JSON support added soon. For RPC, the gRPC project will not only cover protobuf nano but also every other language that protobuf supports. Maybe you can take a look at these, compare our implementations with yours and let us know what we can improve on our side. For the interface change, we won't be able to have that because it conflicts with the API we have internally. For GWT protobuf, right now that's not part of the protobuf project. I only know that GWT has its own protobuf implementation. You might want to contact Ray on that part. Thank you for your reply. First of all, we only depend on protobuf nano jar and generator (the one in the Android Open Source). What we need (in order of priority) : - *Interface isolation* : As we are developing using the Model View Presenter pattern, the presenter and view layers may only access model interfaces. Protobuf has a partial solution to it with the use of *OrBuilder. But it has been decided in our team that not only should we only see interfaces, we should only depend on a jar containing only interfaces. Thus, we modified the protobuf compiler to add the possibility to create java files containing only interfaces, with their implementation being generated using the regular way. What do you suggest ? *- Mutable interface* : using dagger or Guice injection framework, the presenter and view layers can instantiate implementation of the model interfaces. Since we need to mutate the state of the created model objects, and because of the Interface isolation, we added the
Re: [protobuf] Protobuf possible new features
On Fri, May 15, 2015 at 2:56 AM, hypn...@donarproject.org wrote: Le mercredi 13 mai 2015 22:35:47 UTC+2, Feng Xiao a écrit : On Tue, May 12, 2015 at 7:07 AM, hyp...@donarproject.org wrote: Hello Protobuf team, first of all thank you for your great Job. We have been internally using protobuf in a big financial software company for a while and is has proven to be the best choice so far. As our need for multiplatform is growing, we have modified Protobuf and added some features. We reached a point where we though about sharing our need features and see if they can be integrated into the Protobuf mainline. Here are the features : - Generate a new interface for mutable proto objects: *OrBuilder.IBuilder. - Write interfaces (*OrBuilder and *.IBuilder) into separate files. This allows us to package interfaces into a standalone jar that can be referenced from modules that don't need to depend on the implementation. - Generate services for protobuf nano: we have an Android client using our internal RPC system built around protobuf services. - Be able to serialize and parse Proto objects to and from JSON strings. It seems this feature is partially implemented in proto3. - Generate GWT overlays and create a gwt-compatible protobuf jar (like the guava-gwt jar). The gwt-compatible protobuf only has the protobuf nano features. We have been exchanging with Ray Cromwell at the GWT.create conference this year and it seems that some teams at Google have developed there own JSON parser and overlay generator. Are those features relevant for integration into the protobuf mainline? Thanks for reaching out to us. We very much appreciate your contribution to protobuf, but unfortunately as protobuf is used very extensively inside Google, it's unlikely for us to integrate these changes because they either duplicate what we already have inside Google, or conflict with our view of how protobuf API should be. As what you have heard from Ray, internally inside Google we already have JSON serializers/parsers implemented. Actually we not only have one, but have several JSON implementations using different styles developed by different teams. For C++ and Java, I know of at least 3 JSON implementations for each of them. They do not get opensourced with protobuf because JSON is not part of protobuf library before proto3. For other features you mentioned, we also already have them internally. Some of them will get opensourced eventually as part of proto3 but most of them will not (or will be opensourced as a different project). JSON will be included in proto3 so we will select and open-source one of the JSON implementations we have been using internally. We have mutable API for Java implemented, but it will not be opensourced (you can find some traces of the mutable API here https://github.com/google/protobuf/blob/6ef984af4b0c63c1c33127a12dcfc8e6359f0c9e/java/src/main/java/com/google/protobuf/Extension.java#L56). For RPC, it will be part of the gRPC project: http://www.grpc.io/. Although we probably could not integrate your changes directly into protobuf, I think we can still explore whether we can make changes in protobuf in a way that would either eliminate the need to adding these features yourself, or make it easier for these features to interact with the protobuf library. As I already mentioned, we'll have JSON support added soon. For RPC, the gRPC project will not only cover protobuf nano but also every other language that protobuf supports. Maybe you can take a look at these, compare our implementations with yours and let us know what we can improve on our side. For the interface change, we won't be able to have that because it conflicts with the API we have internally. For GWT protobuf, right now that's not part of the protobuf project. I only know that GWT has its own protobuf implementation. You might want to contact Ray on that part. Thank you for your reply. First of all, we only depend on protobuf nano jar and generator (the one in the Android Open Source). What we need (in order of priority) : - *Interface isolation* : As we are developing using the Model View Presenter pattern, the presenter and view layers may only access model interfaces. Protobuf has a partial solution to it with the use of *OrBuilder. But it has been decided in our team that not only should we only see interfaces, we should only depend on a jar containing only interfaces. What's the concern about depending on more than interfaces? Is it for minimizing compile time? Thus, we modified the protobuf compiler to add the possibility to create java files containing only interfaces, with their implementation being generated using the regular way. What do you suggest ? I thought you were talking about protobuf Java lite runtime rather than nano because protobuf nano does not have Builders stuff. What command line do you use when
Re: [protobuf] Protobuf possible new features
Le vendredi 15 mai 2015 19:46:57 UTC+2, Feng Xiao a écrit : On Fri, May 15, 2015 at 2:56 AM, hyp...@donarproject.org javascript: wrote: Le mercredi 13 mai 2015 22:35:47 UTC+2, Feng Xiao a écrit : On Tue, May 12, 2015 at 7:07 AM, hyp...@donarproject.org wrote: Hello Protobuf team, first of all thank you for your great Job. We have been internally using protobuf in a big financial software company for a while and is has proven to be the best choice so far. As our need for multiplatform is growing, we have modified Protobuf and added some features. We reached a point where we though about sharing our need features and see if they can be integrated into the Protobuf mainline. Here are the features : - Generate a new interface for mutable proto objects: *OrBuilder.IBuilder. - Write interfaces (*OrBuilder and *.IBuilder) into separate files. This allows us to package interfaces into a standalone jar that can be referenced from modules that don't need to depend on the implementation. - Generate services for protobuf nano: we have an Android client using our internal RPC system built around protobuf services. - Be able to serialize and parse Proto objects to and from JSON strings. It seems this feature is partially implemented in proto3. - Generate GWT overlays and create a gwt-compatible protobuf jar (like the guava-gwt jar). The gwt-compatible protobuf only has the protobuf nano features. We have been exchanging with Ray Cromwell at the GWT.create conference this year and it seems that some teams at Google have developed there own JSON parser and overlay generator. Are those features relevant for integration into the protobuf mainline? Thanks for reaching out to us. We very much appreciate your contribution to protobuf, but unfortunately as protobuf is used very extensively inside Google, it's unlikely for us to integrate these changes because they either duplicate what we already have inside Google, or conflict with our view of how protobuf API should be. As what you have heard from Ray, internally inside Google we already have JSON serializers/parsers implemented. Actually we not only have one, but have several JSON implementations using different styles developed by different teams. For C++ and Java, I know of at least 3 JSON implementations for each of them. They do not get opensourced with protobuf because JSON is not part of protobuf library before proto3. For other features you mentioned, we also already have them internally. Some of them will get opensourced eventually as part of proto3 but most of them will not (or will be opensourced as a different project). JSON will be included in proto3 so we will select and open-source one of the JSON implementations we have been using internally. We have mutable API for Java implemented, but it will not be opensourced (you can find some traces of the mutable API here https://github.com/google/protobuf/blob/6ef984af4b0c63c1c33127a12dcfc8e6359f0c9e/java/src/main/java/com/google/protobuf/Extension.java#L56). For RPC, it will be part of the gRPC project: http://www.grpc.io/. Although we probably could not integrate your changes directly into protobuf, I think we can still explore whether we can make changes in protobuf in a way that would either eliminate the need to adding these features yourself, or make it easier for these features to interact with the protobuf library. As I already mentioned, we'll have JSON support added soon. For RPC, the gRPC project will not only cover protobuf nano but also every other language that protobuf supports. Maybe you can take a look at these, compare our implementations with yours and let us know what we can improve on our side. For the interface change, we won't be able to have that because it conflicts with the API we have internally. For GWT protobuf, right now that's not part of the protobuf project. I only know that GWT has its own protobuf implementation. You might want to contact Ray on that part. Thank you for your reply. First of all, we only depend on protobuf nano jar and generator (the one in the Android Open Source). What we need (in order of priority) : - *Interface isolation* : As we are developing using the Model View Presenter pattern, the presenter and view layers may only access model interfaces. Protobuf has a partial solution to it with the use of *OrBuilder. But it has been decided in our team that not only should we only see interfaces, we should only depend on a jar containing only interfaces. What's the concern about depending on more than interfaces? Is it for minimizing compile time? Our team wanted to avoid the dependency on protobuf in presenter and view layers. One solution would have been to wrap the proto objects but it is much less maintainable, causes a lot of code duplication and for android, it causes an
Re: [protobuf] Protobuf possible new features
On Fri, May 15, 2015 at 11:08 AM, hypn...@donarproject.org wrote: Le vendredi 15 mai 2015 19:46:57 UTC+2, Feng Xiao a écrit : On Fri, May 15, 2015 at 2:56 AM, hyp...@donarproject.org wrote: Le mercredi 13 mai 2015 22:35:47 UTC+2, Feng Xiao a écrit : On Tue, May 12, 2015 at 7:07 AM, hyp...@donarproject.org wrote: Hello Protobuf team, first of all thank you for your great Job. We have been internally using protobuf in a big financial software company for a while and is has proven to be the best choice so far. As our need for multiplatform is growing, we have modified Protobuf and added some features. We reached a point where we though about sharing our need features and see if they can be integrated into the Protobuf mainline. Here are the features : - Generate a new interface for mutable proto objects: *OrBuilder.IBuilder. - Write interfaces (*OrBuilder and *.IBuilder) into separate files. This allows us to package interfaces into a standalone jar that can be referenced from modules that don't need to depend on the implementation. - Generate services for protobuf nano: we have an Android client using our internal RPC system built around protobuf services. - Be able to serialize and parse Proto objects to and from JSON strings. It seems this feature is partially implemented in proto3. - Generate GWT overlays and create a gwt-compatible protobuf jar (like the guava-gwt jar). The gwt-compatible protobuf only has the protobuf nano features. We have been exchanging with Ray Cromwell at the GWT.create conference this year and it seems that some teams at Google have developed there own JSON parser and overlay generator. Are those features relevant for integration into the protobuf mainline? Thanks for reaching out to us. We very much appreciate your contribution to protobuf, but unfortunately as protobuf is used very extensively inside Google, it's unlikely for us to integrate these changes because they either duplicate what we already have inside Google, or conflict with our view of how protobuf API should be. As what you have heard from Ray, internally inside Google we already have JSON serializers/parsers implemented. Actually we not only have one, but have several JSON implementations using different styles developed by different teams. For C++ and Java, I know of at least 3 JSON implementations for each of them. They do not get opensourced with protobuf because JSON is not part of protobuf library before proto3. For other features you mentioned, we also already have them internally. Some of them will get opensourced eventually as part of proto3 but most of them will not (or will be opensourced as a different project). JSON will be included in proto3 so we will select and open-source one of the JSON implementations we have been using internally. We have mutable API for Java implemented, but it will not be opensourced (you can find some traces of the mutable API here https://github.com/google/protobuf/blob/6ef984af4b0c63c1c33127a12dcfc8e6359f0c9e/java/src/main/java/com/google/protobuf/Extension.java#L56). For RPC, it will be part of the gRPC project: http://www.grpc.io/. Although we probably could not integrate your changes directly into protobuf, I think we can still explore whether we can make changes in protobuf in a way that would either eliminate the need to adding these features yourself, or make it easier for these features to interact with the protobuf library. As I already mentioned, we'll have JSON support added soon. For RPC, the gRPC project will not only cover protobuf nano but also every other language that protobuf supports. Maybe you can take a look at these, compare our implementations with yours and let us know what we can improve on our side. For the interface change, we won't be able to have that because it conflicts with the API we have internally. For GWT protobuf, right now that's not part of the protobuf project. I only know that GWT has its own protobuf implementation. You might want to contact Ray on that part. Thank you for your reply. First of all, we only depend on protobuf nano jar and generator (the one in the Android Open Source). What we need (in order of priority) : - *Interface isolation* : As we are developing using the Model View Presenter pattern, the presenter and view layers may only access model interfaces. Protobuf has a partial solution to it with the use of *OrBuilder. But it has been decided in our team that not only should we only see interfaces, we should only depend on a jar containing only interfaces. What's the concern about depending on more than interfaces? Is it for minimizing compile time? Our team wanted to avoid the dependency on protobuf in presenter and view layers. One solution would have been to wrap the proto objects but it is much less maintainable, causes a lot of code duplication and for android, it causes
Re: [protobuf] Protobuf possible new features
On Tue, May 12, 2015 at 7:07 AM, hypn...@donarproject.org wrote: Hello Protobuf team, first of all thank you for your great Job. We have been internally using protobuf in a big financial software company for a while and is has proven to be the best choice so far. As our need for multiplatform is growing, we have modified Protobuf and added some features. We reached a point where we though about sharing our need features and see if they can be integrated into the Protobuf mainline. Here are the features : - Generate a new interface for mutable proto objects: *OrBuilder.IBuilder. - Write interfaces (*OrBuilder and *.IBuilder) into separate files. This allows us to package interfaces into a standalone jar that can be referenced from modules that don't need to depend on the implementation. - Generate services for protobuf nano: we have an Android client using our internal RPC system built around protobuf services. - Be able to serialize and parse Proto objects to and from JSON strings. It seems this feature is partially implemented in proto3. - Generate GWT overlays and create a gwt-compatible protobuf jar (like the guava-gwt jar). The gwt-compatible protobuf only has the protobuf nano features. We have been exchanging with Ray Cromwell at the GWT.create conference this year and it seems that some teams at Google have developed there own JSON parser and overlay generator. Are those features relevant for integration into the protobuf mainline? Thanks for reaching out to us. We very much appreciate your contribution to protobuf, but unfortunately as protobuf is used very extensively inside Google, it's unlikely for us to integrate these changes because they either duplicate what we already have inside Google, or conflict with our view of how protobuf API should be. As what you have heard from Ray, internally inside Google we already have JSON serializers/parsers implemented. Actually we not only have one, but have several JSON implementations using different styles developed by different teams. For C++ and Java, I know of at least 3 JSON implementations for each of them. They do not get opensourced with protobuf because JSON is not part of protobuf library before proto3. For other features you mentioned, we also already have them internally. Some of them will get opensourced eventually as part of proto3 but most of them will not (or will be opensourced as a different project). JSON will be included in proto3 so we will select and open-source one of the JSON implementations we have been using internally. We have mutable API for Java implemented, but it will not be opensourced (you can find some traces of the mutable API here https://github.com/google/protobuf/blob/6ef984af4b0c63c1c33127a12dcfc8e6359f0c9e/java/src/main/java/com/google/protobuf/Extension.java#L56). For RPC, it will be part of the gRPC project: http://www.grpc.io/. Although we probably could not integrate your changes directly into protobuf, I think we can still explore whether we can make changes in protobuf in a way that would either eliminate the need to adding these features yourself, or make it easier for these features to interact with the protobuf library. As I already mentioned, we'll have JSON support added soon. For RPC, the gRPC project will not only cover protobuf nano but also every other language that protobuf supports. Maybe you can take a look at these, compare our implementations with yours and let us know what we can improve on our side. For the interface change, we won't be able to have that because it conflicts with the API we have internally. For GWT protobuf, right now that's not part of the protobuf project. I only know that GWT has its own protobuf implementation. You might want to contact Ray on that part. Thanks -- François -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To unsubscribe from this group and stop receiving emails from it, send an email to protobuf+unsubscr...@googlegroups.com. To post to this group, send email to protobuf@googlegroups.com. Visit this group at http://groups.google.com/group/protobuf. For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To unsubscribe from this group and stop receiving emails from it, send an email to protobuf+unsubscr...@googlegroups.com. To post to this group, send email to protobuf@googlegroups.com. Visit this group at http://groups.google.com/group/protobuf. For more options, visit https://groups.google.com/d/optout.
[protobuf] Protobuf possible new features
Hello Protobuf team, first of all thank you for your great Job. We have been internally using protobuf in a big financial software company for a while and is has proven to be the best choice so far. As our need for multiplatform is growing, we have modified Protobuf and added some features. We reached a point where we though about sharing our need features and see if they can be integrated into the Protobuf mainline. Here are the features : - Generate a new interface for mutable proto objects: *OrBuilder.IBuilder. - Write interfaces (*OrBuilder and *.IBuilder) into separate files. This allows us to package interfaces into a standalone jar that can be referenced from modules that don't need to depend on the implementation. - Generate services for protobuf nano: we have an Android client using our internal RPC system built around protobuf services. - Be able to serialize and parse Proto objects to and from JSON strings. It seems this feature is partially implemented in proto3. - Generate GWT overlays and create a gwt-compatible protobuf jar (like the guava-gwt jar). The gwt-compatible protobuf only has the protobuf nano features. We have been exchanging with Ray Cromwell at the GWT.create conference this year and it seems that some teams at Google have developed there own JSON parser and overlay generator. Are those features relevant for integration into the protobuf mainline ? Thanks -- François -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To unsubscribe from this group and stop receiving emails from it, send an email to protobuf+unsubscr...@googlegroups.com. To post to this group, send email to protobuf@googlegroups.com. Visit this group at http://groups.google.com/group/protobuf. For more options, visit https://groups.google.com/d/optout.
