On Wednesday, January 31, 2018 at 11:20:31 AM UTC-8, Walter Schulze wrote: > > Hi JT please see my inline replies. > > On Wed, 31 Jan 2018 at 19:05 <thebroke...@gmail.com <javascript:>> wrote: > >> Thank you, Walter, for your support. >> >> > gogo/protobuf is disappointed that golang/protobuf still thinks that >> runtime reflection is an efficient way of serializing structures. >> >> The table-driven implementation avoids reflect in the fast and common >> path. Instead, are you referring to the fact that we don't perform >> full-code generation of Marshal/Unmarshal like what gogo/protobuf does? We >> are aware that full-code generation will often out-perform the table-driven >> approach we took. However, full code-generation drastically bloats the >> binary size when you have many proto messages linked in. Keeping the binary >> size smaller was an important design decision for us and seemed to be a >> better default. >> > > Yes, I was referring to the speed of code generation over runtime > reflection. > What I struggle to understand is why the optimize_for file option that is > part of proto 2 and 3 is not considered by golang/protobuf as a way to > specify when code generation should be used over runtime reflection. > This seems to work for most other languages, including Java, which I heard > is quite popular among real software developers. >
Note that the code-generation approach used by other languages (mostly C++/Java) has its own problem. Mostly because of the tight coupling among generated code, runtime and embedded sub messages (generated by a different party using a different version of protoc). These problem don't exist inside of google as we use a single repo build system, but it cause significant issues in opensource. For instance, Hadoop is still shipping protobuf v2.5 generated code which is incompatible with later version of protobufs. All the projects using Hadoop are then version locked to v2.5, as an upgrade in any project (including Hadoop itself) will break the build. Version upgrade can only happen when all the transitive dependency closure upgrade together atomically. We solve this problem in protobuf-java v3.0+ by introducing ABI backward/forward compatibility guarantees on generated code and runtime. However, this introduced lots of overhead on code maintenance, reduced development velocity and limited the change we could do. We are now solving the issue by introduce table driven to Java. The recent benchmark result showed performance on par for android platforms, and hopefully we can release the new implementation in a few months. For Go, if we are going to introduce full generated code, I'd strongly recommend considering those complications. Major version bump is also expensive for protobufs as all the dependency libraries would have to bump their major version too. > > >> >> We are open to considering an option that allows user to specify >> full-code generation for select messages. >> > > This is exactly what gogo/protobuf allows users to do. > Using protobuf extensions gogo/protobuf allows the user to specify per > message or file whether they want to generate marshalers, unmarshalers, etc. > A user can also create a vanity binary to generate these methods if you do > not wish to use extensions and want to enforce a specific style across and > organization. > > >> >> > gogo/protobuf is still open to being merged back into golang/protobuf >> and has been since its inception 5 years ago. >> >> That is good to hear. I have not yet gone through all of gogo/protobuf to >> determine what it would to merge, or what should be merged. This will be >> future work. >> > > gogo/protobuf is also be open to only being partly merged. > > One other major advantage of gogo/protobuf is generating the structures > you want to use, by allowing you to modify the generated structure using > protobuf extensions like customtype. > This way you can avoid copying between the protobuf generated structure > and a user defined go structure that you actually want to use. > This is a huge speed and safety gain and probably the most important > feature of gogo/protobuf. > proto3 has addressed the biggest concern by allowing the generation of > fields without pointers, but there are other cases as well, including > casttype, customname for generating more lintable code and even not > generating the structure at all, for ultimate customization. > I would hope that merging some of these ideas will also be on the table. > > Looking forward to working together for a change > Please let me know how I can help > > Skeptically hopeful about a new era for protobufs in Go > Walter Schulze > > >> >> JT >> >> On Wednesday, January 31, 2018 at 7:13:38 AM UTC-8, Walter Schulze wrote: >>> >>> gogo/protobuf is happy to be acknowledged by Google as an entity in the >>> golang protobuf space. >>> gogo/protobuf welcomes golang/protobuf to the community and is extremely >>> happy to see this kind of transparency. >>> >>> gogo/protobuf will also merge these changes and as usual try to stay as >>> close as possible to golang/protobuf, >>> including also following the same version tagging. >>> >>> gogo/protobuf is disappointed that golang/protobuf still thinks that >>> runtime reflection is an efficient way of serializing structures. >>> >>> go Green go GoGoProtobuf >>> >>> PS >>> >>> gogo/protobuf is still open to being merged back into golang/protobuf >>> and has been since its inception 5 years ago. >>> gogo/protobuf feels for its users, especially those that are not >>> acknowledged by grpc-gateway and grpc-go, >>> and forced to employ work arounds, to preserve their missions of safety >>> and efficiency. >>> It knows that its existence is not something that anyone prefers, and it >>> welcomes death, >>> but only if it can preserve its legacy of fast serailization and >>> generating the structures you want to use. >>> >>> >>> On Tuesday, 30 January 2018 23:44:37 UTC+1, joe...@google.com wrote: >>>> >>>> Done. I tagged v1.0.0. When we perform the merge in the future, it will >>>> be tagged as v1.1.0. >>>> >>>> On Tuesday, January 30, 2018 at 9:37:23 AM UTC-8, Alexey Palazhchenko >>>> wrote: >>>>> >>>>> Hi, >>>>> >>>>> Can you please add tags to the repository before that? SemVer or even >>>>> tags with _any_ semantic would greatly help to rollback to the latest >>>>> working version when things break. >>>>> >>>>> –-– >>>>> Alexey «AlekSi» Palazhchenko >>>>> >>>>> -- >> You received this message because you are subscribed to a topic in the >> Google Groups "golang-nuts" group. >> To unsubscribe from this topic, visit >> https://groups.google.com/d/topic/golang-nuts/F5xFHTfwRnY/unsubscribe. >> To unsubscribe from this group and all its topics, send an email to >> golang-nuts...@googlegroups.com <javascript:>. >> For more options, visit https://groups.google.com/d/optout. >> > -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
