Re: [protobuf] What are well-known types?

[Josh Humphries]

What do you think about creating issues for them on the protobuf github
project and then include links to the design docs?

Also, the addition of boxed types seems to be back-pedaling a bit on the
decision to make zero and absent indistinguishable for primitive types.
This seems to be a concession that the proto2 behavior was useful, but at
the expense of on-the-wire bloat for present values.



*Josh Humphries*
Manager, Shared Systems  |  Platform Engineering
Atlanta, GA  |  678-400-4867
*Square* (www.squareup.com)

On Tue, Feb 23, 2016 at 6:34 PM, 'Feng Xiao' via Protocol Buffers <
protobuf@googlegroups.com> wrote:

> On Tue, Feb 23, 2016 at 12:55 PM, Zellyn  wrote:
>
>> There are increasing numbers of references to "well-known" types in
>> protos. For instance, I see changes in the Go implementation to support
>> them.
>> There were passing references in release notes in this group.
>>
>> However, the main protobuf site includes no narrative explanation that I
>> can find.
>>
>> The idea of a few well-known types to represent "Boxed" values since
>> proto3 removes the ability to null out fields makes sense, but the only
>> documentation I could find, in the reference section of the protobuf site
>> at
>> https://developers.google.com/protocol-buffers/docs/reference/google.protobuf,
>> includes all sorts of things like Struct, Method, Mixin, etc. that are
>> entirely unclear.
>>
>> Is there a conversation happening somewhere that I'm missing, or is it
>> Google-internal but not documented outside yet?
>>
> Sorry, the documentation for proto3 is not complete yet. We are still
> working on improving it. There are a couple of internal design docs for
> well-known types, but they are not yet ready to be included in the public
> developer guide.
>
>
>>
>> Thanks,
>>
>> Zellyn
>>
>> ps - the reintroduction of message types for primitives rather undermines
>> my belief in the arguments for removing optional fields in proto3 in the
>> first place. I'd like to give the benefit of the doubt to the folks
>> designing proto3: is the thinking articulated clearly somewhere?
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Protocol Buffers" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to protobuf+unsubscr...@googlegroups.com.
>> To post to this group, send email to protobuf@googlegroups.com.
>> Visit this group at https://groups.google.com/group/protobuf.
>> For more options, visit https://groups.google.com/d/optout.
>>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Protocol Buffers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to protobuf+unsubscr...@googlegroups.com.
> To post to this group, send email to protobuf@googlegroups.com.
> Visit this group at https://groups.google.com/group/protobuf.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to protobuf+unsubscr...@googlegroups.com.
To post to this group, send email to protobuf@googlegroups.com.
Visit this group at https://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.

Re: [protobuf] Any JSON coding question

[Tim Kientzle]

I think your current Any JSON design requires that protobuf serialization be 
able to fail.  This is definitely a major change from proto2 and something the 
conformance test does not explore.

1. Decode a valid JSON-encoded message with an Any field in an unknown schema.

2. Attempt to encode it in protobuf format.

If the descriptor cannot be fetched, then step #2 here must fail.

(The alternative would be for #1 to fail in this case, but that would imply 
that JSON-to-JSON transcoding must fail in some circumstances.)

If Any had been defined slightly differently, then transcoding would be 
trivial.  If you are unable to change the current definition of Any, please 
consider introducing an alternative:

package google.protobuf;

message Other {
  string type_url = 1;
  oneof encoded {
 bytes protobuf = 2;
 google.protobuf.Value json = 3;
  }
}

This is trivial to transcode:  it only requires that the final consumer be able 
to decode whatever format the originator attached.  Intermediate systems can 
freely transcode without needing to understand the schema at all, since the 
‘protobuf’ field here can be represented in JSON and the ‘json’ field can be 
encoded to protobuf.  The actual JSON serialization is very similar to the 
current Any serialization (with the addition of the extra “json” tag).

Cheers,

Tim



> On Feb 5, 2016, at 1:53 PM, Josh Haberman  wrote:
> 
> Hi Tim,
> 
> I think your analysis is correct. The answer to your question is that it's 
> not possible to transcode Any from JSON <-> protobuf unless you have the 
> schema. That is one of the reasons why the type URL exists as a way to fetch 
> schemas for types you otherwise wouldn't know about.
> 
> I'm glad the conformance suite was helpful!
> 
> Best,
> Josh
> 
> On Wednesday, February 3, 2016 at 1:37:40 PM UTC-8, Tim Kientzle wrote:
> I’m trying to implement Any, including proper JSON and protobuf encoding, and 
> the following scenario has me stumped:
> 
> 
> 
> I’m looking at the any.proto file, which includes this example:
> 
> 
> 
> Sample proto file:
> 
> // package google.profile;
> 
> // message Person {
> 
> //   string first_name = 1;
> 
> //   string last_name = 2;
> 
> // }
> 
> 
> 
> Resulting JSON encoding:
> 
> // {
> 
> //   "@type": "type.googleapis.com/google.profile.Person",
> 
> //   "firstName": ,
> 
> //   "lastName": 
> 
> // }
> 
> 
> 
> The scenario I’m trying to understand involves three systems:
> 
> 
> 
> Originating system:
> 
> * Knows google.profile and writes out the JSON above
> 
> 
> 
> Middle system:
> 
> * Reads the JSON example above
> 
> * Does *not* know google.profile (doesn’t have the descriptor and has no way 
> to get it)
> 
> * Writes out protobuf
> 
> 
> 
> Receiving system:
> 
> * Knows google.profile and reads the protobuf
> 
> 
> 
> In particular:  What is the protobuf serialization between the middle and 
> receiving systems?
> 
> 
> 
> Without the descriptor, the middle system cannot poduce the “real" protobuf 
> binary format (the field numbers aren’t in that format), so the only option 
> would seem to be to serialize the JSON data as a protobuf Struct 
> representation (which in turn has interesting implications for the receiving 
> system). Am I missing some other option?
> 
> 
> 
> Tim
> 
> 
> 
> P.S.  Nice work on the conformance test suite, by the way!  That clarified a 
> *lot* of details about the expected handling for protobuf JSON format.
> 
> 
> 
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Protocol Buffers" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to protobuf+unsubscr...@googlegroups.com.
> To post to this group, send email to protobuf@googlegroups.com.
> Visit this group at https://groups.google.com/group/protobuf.
> For more options, visit https://groups.google.com/d/optout.


-- 
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to protobuf+unsubscr...@googlegroups.com.
To post to this group, send email to protobuf@googlegroups.com.
Visit this group at https://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.

Re: [protobuf] Protobuf python packaging conflicts with AppEngine packaging

['Adrian Wan' via Protocol Buffers]

For the reference of others encountering this, the GitHub issue 
is https://github.com/google/protobuf/issues/1153

I'd be thankful for an update :)

On Tuesday, January 19, 2016 at 2:04:05 PM UTC-8, Feng Xiao wrote:
>
> Could you submit this as a github issue?
> https://github.com/google/protobuf/issues
>
> On Mon, Jan 18, 2016 at 2:36 PM, Dan Frame  > wrote:
>
>> I ran into an issue today with AppEngine imports and noticed that it was 
>> being caused by the PiPy package of protobufs.
>>
>> Since the Google protobuf package and the Google AppEngine package share 
>> the same google.* module, if the protobuf package is installed, all 
>> appengine imports fail due to the missing sub-modules (API, users, etc). I 
>> imagine that AppEngine would also cause issues with the protobuf package 
>> imports failing depending on the ordering on the Python path.
>>
>> It's simple to work around, but I wanted to bring this up if it hasn't 
>> been mentioned previously.
>>
>>
>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Protocol Buffers" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to protobuf+u...@googlegroups.com .
>> To post to this group, send email to prot...@googlegroups.com 
>> .
>> Visit this group at https://groups.google.com/group/protobuf.
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to protobuf+unsubscr...@googlegroups.com.
To post to this group, send email to protobuf@googlegroups.com.
Visit this group at https://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.

Re: [protobuf] Putting compiled protobufs into a python package

['Adrian Wan' via Protocol Buffers]

I believe I ran into this issue as well - 
https://github.com/google/protobuf/issues/881? 

On Monday, March 29, 2010 at 11:59:01 AM UTC-7, Kenton Varda wrote:
>
> Yeah, there's really no way to accomplish this without editing the .proto 
> files you are importing.  This is an issue in regular old Python as well 
> (or many other languages):  you can't really fix someone else's poor 
> package layout without editing their code.  I guess it's a bigger problem 
> in protobufs because people working in C++ may make decisions that aren't 
> so bad for C++ but are poor for Python.
>
> Instead of actually maintaining forks, you could write a sed script which 
> automatically fixes up the other project's .proto files and have your build 
> system automatically run this script.
>
> On Mon, Mar 29, 2010 at 10:00 AM, John Admanski  > wrote:
>
>> I have a situation where I want to make use of protocol buffers defined 
>> in a separate source tree; but when I compile the protobufs into python 
>> code all the compiled code assumes it's going to be exposed by putting it 
>> onto sys.path somehow (via one of the usual mechanisms). The problem is 
>> that due to the way the protobufs are defined in the original source tree 
>> this adds a bunch of new top-level packages and generally pollutes the 
>> global module namespace.
>>
>> So I wanted to work around this by putting all the compiled protobufs 
>> into a package, but this doesn't work with protoc; it assumes that if a 
>> protobuf references X/Y/Z.proto, then there's going to be an X.Y.Z_pb2 
>> module it can import. I can fix this by editing all the proto files to 
>> change all the references to be relative to this new package I'm defining 
>> but that basically means I have to fork the proto files from this other 
>> project.
>>
>> Is there any better mechanism when compiling protobufs that would allow 
>> me to put the compiled output into a package, rather than having to hang 
>> them off of sys.path or tweak all the proto files?
>>
>> -- John
>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Protocol Buffers" group.
>> To post to this group, send email to prot...@googlegroups.com 
>> .
>> To unsubscribe from this group, send email to 
>> protobuf+u...@googlegroups.com .
>> For more options, visit this group at 
>> http://groups.google.com/group/protobuf?hl=en.
>>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to protobuf+unsubscr...@googlegroups.com.
To post to this group, send email to protobuf@googlegroups.com.
Visit this group at https://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.

[protobuf] Compiling into relocatable Python packages

['Adrian Wan' via Protocol Buffers]

Relates to my (awan1) comment on this issue 
. I am struggling with 
including a Python package generated from a protobuf package, because the 
way protobuf compiles an import statement is

import "a/b/c.proto"->from a.b import c_pb2

and when the resultant package is included in a wider package, it gives an 
ImportError: no module named a.b.

At this point I'd be happy if anyone had a good workaround. I am going to 
have to resort to path-wrangling in each file that tries to import this 
package, which is ugly but works. The dream solution (as mentioned in the 
Github issue) is to have protoc generate nice relative imports and complete 
packages... And given that the "package" label is given and ignored for 
protobuf, this doesn't seem impossible: when compiling a package, use that 
to figure out how the path names should be changed to relative imports? 

-- 
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to protobuf+unsubscr...@googlegroups.com.
To post to this group, send email to protobuf@googlegroups.com.
Visit this group at https://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.

Re: [protobuf] Putting compiled protobufs into a python package

['John Admanski' via Protocol Buffers]

This has been far enough in the past that I can't actually remember what I
was doing that ran into this issue...

On Thu, Mar 3, 2016 at 11:36 AM, Adrian Wan  wrote:

> I believe I ran into this issue as well -
> https://github.com/google/protobuf/issues/881?
>
> On Monday, March 29, 2010 at 11:59:01 AM UTC-7, Kenton Varda wrote:
>>
>> Yeah, there's really no way to accomplish this without editing the .proto
>> files you are importing.  This is an issue in regular old Python as well
>> (or many other languages):  you can't really fix someone else's poor
>> package layout without editing their code.  I guess it's a bigger problem
>> in protobufs because people working in C++ may make decisions that aren't
>> so bad for C++ but are poor for Python.
>>
>> Instead of actually maintaining forks, you could write a sed script which
>> automatically fixes up the other project's .proto files and have your build
>> system automatically run this script.
>>
>> On Mon, Mar 29, 2010 at 10:00 AM, John Admanski 
>> wrote:
>>
>>> I have a situation where I want to make use of protocol buffers defined
>>> in a separate source tree; but when I compile the protobufs into python
>>> code all the compiled code assumes it's going to be exposed by putting it
>>> onto sys.path somehow (via one of the usual mechanisms). The problem is
>>> that due to the way the protobufs are defined in the original source tree
>>> this adds a bunch of new top-level packages and generally pollutes the
>>> global module namespace.
>>>
>>> So I wanted to work around this by putting all the compiled protobufs
>>> into a package, but this doesn't work with protoc; it assumes that if a
>>> protobuf references X/Y/Z.proto, then there's going to be an X.Y.Z_pb2
>>> module it can import. I can fix this by editing all the proto files to
>>> change all the references to be relative to this new package I'm defining
>>> but that basically means I have to fork the proto files from this other
>>> project.
>>>
>>> Is there any better mechanism when compiling protobufs that would allow
>>> me to put the compiled output into a package, rather than having to hang
>>> them off of sys.path or tweak all the proto files?
>>>
>>> -- John
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "Protocol Buffers" group.
>>> To post to this group, send email to prot...@googlegroups.com.
>>> To unsubscribe from this group, send email to
>>> protobuf+u...@googlegroups.com.
>>> For more options, visit this group at
>>> http://groups.google.com/group/protobuf?hl=en.
>>>
>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to protobuf+unsubscr...@googlegroups.com.
To post to this group, send email to protobuf@googlegroups.com.
Visit this group at https://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.

[protobuf] gcc float-equal error

[Seth Humphries]

I am using proto 3.0.0-beta-1 with a proto file that looks like this:

```
syntax = "proto3";

message SomeMsg {
string query = 1;
int32 integer = 2;
float nothernum = 3;
}
```

I am using gcc 4.9.2 with cmake 3.4.0. I use cmake to enable all gcc 
warnings and treat all warnings as errors. 

I get the follow errors
```
error: comparing floating point with == or != is unsafe 
[-Werror=float-equal]
if (this->nothernum() != 0) {
  ^
```
and
```
error: comparing floating point with == or != is unsafe 
[-Werror=float-equal]
if (from.nothernum() != 0) {
  ^
```

I have included the proto file in my hpp like ``` #include 
``` and still get the error. 

I can suppress the float-equal error but I would prefer to leave all 
warnings on. The error goes away also if I use proto2 and make everything 
optional.

Anyone have other ideas of how to resolve this without suppressing the 
warning and still use proto3?

-- 
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to protobuf+unsubscr...@googlegroups.com.
To post to this group, send email to protobuf@googlegroups.com.
Visit this group at https://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.

[protobuf] How does "_get_packed_size" work when a message member is "repeated"

[Sphoorthi Dayanand]

Hi All,

I have just started using Google Protocol Buffers and I am a little 
confused with the "get_packed_size" function.
For example, in the address book example, we see the following:

message AddressBook {
  repeated Person person = 1;
}

After compiling to generate C code, following function is defined:

size_t tutorial__address_book__get_packed_size
 (const Tutorial__AddressBook *message)
{
  PROTOBUF_C_ASSERT (message->base.descriptor == 
__address_book__descriptor);
  return protobuf_c_message_get_packed_size ((const 
ProtobufCMessage*)(message));
}

Since we dont know what will be the number of occurrences of the message 
"Person" within "AddressBook", how will the output of above function be 
generated?

Thanks and Regards,
Sphoorthi Dayanand

-- 
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to protobuf+unsubscr...@googlegroups.com.
To post to this group, send email to protobuf@googlegroups.com.
Visit this group at https://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.

Re: [protobuf] cannot generate cs file from addressbook.proto

[izzet izzet]

Oh! It is pre release package. So, you're right; I get the wrong nuget 
package then :).

Thank you.

2 Mart 2016 Çarşamba 21:19:29 UTC+2 tarihinde Feng Xiao yazdı:
>
>
> Protobuf is currently at v3.0.0-beta-2. The package you downloaded is not 
> maintained by the protobuf team. Try this one instead:
> https://www.nuget.org/packages/Google.Protobuf
>
> Please also download latest protoc binary from the release page:
> https://github.com/google/protobuf/releases/tag/v3.0.0-beta-2
>
> (try protoc-3.0.0-beta-2-win32.zip)
>  
>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Protocol Buffers" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to protobuf+u...@googlegroups.com .
>> To post to this group, send email to prot...@googlegroups.com 
>> .
>> Visit this group at https://groups.google.com/group/protobuf.
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to protobuf+unsubscr...@googlegroups.com.
To post to this group, send email to protobuf@googlegroups.com.
Visit this group at https://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.