Re: [protobuf] Default Values vs Missing Values

2016-05-17 Thread Teddy Zhang 
I'm really not happy to see that proto3 removed the ability in generate 
code for check whether a field exits or not.

For a message like this:
message Test1 {
  required int32 a = 1;
}
If field a is present, the encoded message will have field with id 1 and 
its value. If the field is not set, the encoded message will not have field 
id 1.
In proto2 generated code, it provides a has method to check whether the 
field exists or not.
In proto3, this is no such thing. During deserialization, if the field is 
not exists, default value is set. So you can't tell whether the field does 
not exist or have a default value. That doesn't match the underline 
encoding anymore.

This is a breaking change and will portentially impact a lot of people. 
Basically we're losing nullable support.
For our project, we heavily depends on that. There are workarounds (add 
a Boolean field) but it is ugly. I think that will stop us from moving from 
proto2 to proto3 (may need find alternatives).

Can we add the functionality back?

On Saturday, March 26, 2016 at 11:47:08 AM UTC-7, Ilia Mirkin wrote:

> Use proto2, which has the has_* checks per field. (Using get_* you 
> still get the default value, of course.) It's extremely unfortunate 
> that this functionality was removed in proto3, I see that making 
> proto3 unattractive for all but the simplest uses of protos. I know in 
> almost every protobuf use-case I've had, the presence accessors were 
> imperative to proper operation. 
>
> On Sat, Mar 26, 2016 at 2:43 PM, Yoav H  > wrote: 
> > Hi, 
> > 
> > I wanted ask regarding the decision to populate fields with default 
> values, 
> > even if they do not appear in the encoded message. 
> > If I want to send a "patch" message, where I want to update just the 
> > provided fields, how can I do that with protobuf (without adding 
> IsXXXSet 
> > for every field)? 
> > 
> > Why not add another type, representing a default value? 
> > So the schematics would be, if the field is missing, it is null, and if 
> the 
> > field exists, but with this "missing value" type, it will get the 
> default 
> > value? 
> > 
> > Thanks, 
> > Yoav. 
> > 
> > -- 
> > You received this message because you are subscribed to the Google 
> Groups 
> > "Protocol Buffers" group. 
> > To unsubscribe from this group and stop receiving emails from it, send 
> an 
> > email to protobuf+u...@googlegroups.com . 
> > To post to this group, send email to prot...@googlegroups.com 
> . 
> > Visit this group at https://groups.google.com/group/protobuf. 
> > For more options, visit https://groups.google.com/d/optout. 
>

-- 
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to protobuf+unsubscr...@googlegroups.com.
To post to this group, send email to protobuf@googlegroups.com.
Visit this group at https://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.

Re: [protobuf] protobuf-dt required password from eclipse marketplace

2016-05-17 Thread Yan Yan 
It worked. Thx.

On Wednesday, May 18, 2016 at 1:30:28 AM UTC+8, Alexander Rookey wrote:
>
> The marketplace has been updated to reflect the current update-site of 
> protobuf-dt.  It should work now.  The protobuf editor can also be 
> installed manually using the Eclipse Install dialog window "Help > Install 
> New Software..." and the update site "
> http://google.github.io/protobuf-dt/updates/latest/;.  Best of Luck!
>
> On Monday, May 16, 2016 at 1:30:19 PM UTC-4, workin...@gmail.com wrote:
>>
>>
>> Me and a collegue have encountered this exact problem too, looking 
>> forward to the fix!
>> On Tuesday, 10 May 2016 00:35:36 UTC+2, Adam Cozzette wrote:
>>>
>>> Thanks for reporting this, Yan. I've passed this along to the team that 
>>> maintains the plugin; they were able to reproduce the problem and are 
>>> working on a fix now.
>>>
>>> On Sun, May 8, 2016 at 6:00 AM, Yan Yan  wrote:
>>>

 


 On Saturday, May 7, 2016 at 8:00:02 AM UTC+8, Adam Cozzette wrote:
>
> Could you provide some more detail on where a password was required?
>
> On Sun, May 1, 2016 at 7:34 PM, Yan Yan  wrote:
>
>> It worked before. Now it required a password. Any idea? Thx.
>>
>> -- 
>> You received this message because you are subscribed to the Google 
>> Groups "Protocol Buffers" group.
>> To unsubscribe from this group and stop receiving emails from it, 
>> send an email to protobuf+u...@googlegroups.com.
>> To post to this group, send email to prot...@googlegroups.com.
>> Visit this group at https://groups.google.com/group/protobuf.
>> For more options, visit https://groups.google.com/d/optout.
>>
>
> -- 
 You received this message because you are subscribed to the Google 
 Groups "Protocol Buffers" group.
 To unsubscribe from this group and stop receiving emails from it, send 
 an email to protobuf+u...@googlegroups.com.
 To post to this group, send email to prot...@googlegroups.com.
 Visit this group at https://groups.google.com/group/protobuf.
 For more options, visit https://groups.google.com/d/optout.

>>>
>>>

-- 
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to protobuf+unsubscr...@googlegroups.com.
To post to this group, send email to protobuf@googlegroups.com.
Visit this group at https://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.

Re: [protobuf] protobuf-dt required password from eclipse marketplace

2016-05-17 Thread 'Alexander Rookey' via Protocol Buffers 
The marketplace has been updated to reflect the current update-site of 
protobuf-dt.  It should work now.  The protobuf editor can also be 
installed manually using the Eclipse Install dialog window "Help > Install 
New Software..." and the update site 
"http://google.github.io/protobuf-dt/updates/latest/;.  Best of Luck!

On Monday, May 16, 2016 at 1:30:19 PM UTC-4, workin...@gmail.com wrote:
>
>
> Me and a collegue have encountered this exact problem too, looking forward 
> to the fix!
> On Tuesday, 10 May 2016 00:35:36 UTC+2, Adam Cozzette wrote:
>>
>> Thanks for reporting this, Yan. I've passed this along to the team that 
>> maintains the plugin; they were able to reproduce the problem and are 
>> working on a fix now.
>>
>> On Sun, May 8, 2016 at 6:00 AM, Yan Yan  wrote:
>>
>>>
>>> 
>>>
>>>
>>> On Saturday, May 7, 2016 at 8:00:02 AM UTC+8, Adam Cozzette wrote:

 Could you provide some more detail on where a password was required?

 On Sun, May 1, 2016 at 7:34 PM, Yan Yan  wrote:

> It worked before. Now it required a password. Any idea? Thx.
>
> -- 
> You received this message because you are subscribed to the Google 
> Groups "Protocol Buffers" group.
> To unsubscribe from this group and stop receiving emails from it, send 
> an email to protobuf+u...@googlegroups.com.
> To post to this group, send email to prot...@googlegroups.com.
> Visit this group at https://groups.google.com/group/protobuf.
> For more options, visit https://groups.google.com/d/optout.
>

 -- 
>>> You received this message because you are subscribed to the Google 
>>> Groups "Protocol Buffers" group.
>>> To unsubscribe from this group and stop receiving emails from it, send 
>>> an email to protobuf+u...@googlegroups.com.
>>> To post to this group, send email to prot...@googlegroups.com.
>>> Visit this group at https://groups.google.com/group/protobuf.
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to protobuf+unsubscr...@googlegroups.com.
To post to this group, send email to protobuf@googlegroups.com.
Visit this group at https://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.

[protobuf] Re: XML to Protocol Buffers converter

2016-05-17 Thread Huimin 
Hello Simon,
Sorry! I forgot to mention that I am interested in XML to Protocol Buffers 
converter in java.
Thanks!
Huimin

On Monday, September 28, 2009 at 9:14:52 AM UTC-4, sim wrote:
>
> Hi all. Would anybody be interested in an XML to Protocol Buffers 
> converter if it were opened up to the community?  I have an XML 
> stylesheet that transforms an annotated XSD set into a Java class that 
> uses JAXB and the Protocol Buffers Java API to convert XML documents 
> into either text or binary mode Protocol Buffers messages.  The XSD 
> annotations define the mappings from XSD elements to Proto messages 
> (although at present staying close to a 1:1 mapping is probably 
> safest). Some fancy XSD features are not yet supported but the usual 
> complexTypes, simpleTypes, elements, and enumerations work. 
>
> Simon Weeks 
>

-- 
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to protobuf+unsubscr...@googlegroups.com.
To post to this group, send email to protobuf@googlegroups.com.
Visit this group at https://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.

[protobuf] Re: XML to Protocol Buffers converter

2016-05-17 Thread Huimin 
Hello Simon,
I am interested in the XML to Protocol Buffers converter. Is it opened up 
to the community? Where can I get it?
Thanks a lot for your support!
Huimin

On Monday, September 28, 2009 at 9:14:52 AM UTC-4, sim wrote:
>
> Hi all. Would anybody be interested in an XML to Protocol Buffers 
> converter if it were opened up to the community?  I have an XML 
> stylesheet that transforms an annotated XSD set into a Java class that 
> uses JAXB and the Protocol Buffers Java API to convert XML documents 
> into either text or binary mode Protocol Buffers messages.  The XSD 
> annotations define the mappings from XSD elements to Proto messages 
> (although at present staying close to a 1:1 mapping is probably 
> safest). Some fancy XSD features are not yet supported but the usual 
> complexTypes, simpleTypes, elements, and enumerations work. 
>
> Simon Weeks 
>

-- 
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to protobuf+unsubscr...@googlegroups.com.
To post to this group, send email to protobuf@googlegroups.com.
Visit this group at https://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.

Re: [protobuf] How to erase particular field with MergeFrom()

2016-05-17 Thread 'Feng Xiao' via Protocol Buffers 
On Tue, May 17, 2016 at 1:49 AM, Denis Bakhvalov 
wrote:

> Hi Feng Xiao,
>
> Thanks for your reply!
>
> Unfortunately I couldn't find good examples how to use it, so let me ask
> one more question:
>
> Should I put mask as a part of original message:
>
> message RepAndOpt
> {
> repeated string name = 1;
> string surname = 2;
>
> google.protobuf.FieldMask field_mask = 3;
> }
>
> or put it as standalone message?
>
> message RepAndOptMask
> {
> google.protobuf.FieldMask field_mask = 1;
> }
>
> I think there is nothing wrong if I will send them separately, right?
>
It's probably more clear to put it separately from the original message
it's applied to, but either one works.


>
> понедельник, 16 мая 2016 г., 22:14:33 UTC+2 пользователь Feng Xiao написал:
>>
>>
>>
>> On Wed, May 11, 2016 at 5:09 AM, Denis Bakhvalov 
>> wrote:
>>
>>> Hi,
>>>
>>> Suppose I have such message structure:
>>>
>>>
>>> package msg_RepAndOpt;
>>>
>>> message RepAndOpt{
>>> repeated string name = 1;
>>> optional string surname = 2;
>>> ...
>>> // there are lots of others.}
>>>
>>>
>>> And I have two components that have copies of this message:
>>>
>>>
>>> // component1:RepAndOpt A;
>>> A.add_name("Name");
>>> A.set_surname("Surname");
>>> // component2:RepAndOpt B;
>>>
>>>
>>> In my case components modify those messages via transaction mechanism.
>>> It means that if one component changes some field it also sends it to
>>> another component to propagate those changes. Component-receiver is doing
>>> merge:
>>>
>>>
>>> // Component2 modifies B and sends it to component1.// Component1 perfoms 
>>> merge:
>>> A.MergeFrom(B);
>>>
>>>
>>> Now, say, component2 wants to erase field "name". If it will send clear
>>> B message (default construction) than:
>>>
>>>- - MergeFrom() will not modify A;
>>>- - CopyFrom() will erase also other fields.
>>>
>>> Another way will be to fill B with the contents of A, clear name field
>>> and component1 will use CopyFrom(). But this is not acceptable because
>>> system is really high-loaded and there could be lots of other fields. So,
>>> desired solution to clean name field is:
>>>
>>>1. 1. Component2 create B message.
>>>2. 2. Explicitly stores information that it want to erase only name
>>>field.
>>>3. 3. Component1 perform A.MergeFrom(B).
>>>4. 4. Result: A::name is cleared but other fields are left untouched.
>>>
>>> As far as I tested this applies to repeated and optional fields. Is
>>> there any ready-to-use solution or I should modify protobuf implementation?
>>>
>> I think you can do this using FieldMaskUtil:
>>
>> https://github.com/google/protobuf/blob/beta-3/src/google/protobuf/util/field_mask_util.h#L111
>>
>> You create an empty B message along with a FieldMask containing only the
>> field you want to clear, and use FieldMaskUtil::MergeMessageTo(A,
>> field_mask, )  to replace the field in A with the value from B.
>>
>>
>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "Protocol Buffers" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to protobuf+u...@googlegroups.com.
>>> To post to this group, send email to prot...@googlegroups.com.
>>> Visit this group at https://groups.google.com/group/protobuf.
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>
>> --
> You received this message because you are subscribed to the Google Groups
> "Protocol Buffers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to protobuf+unsubscr...@googlegroups.com.
> To post to this group, send email to protobuf@googlegroups.com.
> Visit this group at https://groups.google.com/group/protobuf.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to protobuf+unsubscr...@googlegroups.com.
To post to this group, send email to protobuf@googlegroups.com.
Visit this group at https://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.

Re: [protobuf] protobuf-dt required password from eclipse marketplace

2016-05-17 Thread 'Adam Cozzette' via Protocol Buffers 
I've just now heard that the problem has been fixed. Yan and workingmeloen,
could you try again and let me know if you run into any more problems?

On Wed, May 11, 2016 at 5:21 AM,  wrote:

>
> Me and a collegue have encountered this exact problem too, looking forward
> to the fix!
> On Tuesday, 10 May 2016 00:35:36 UTC+2, Adam Cozzette wrote:
>>
>> Thanks for reporting this, Yan. I've passed this along to the team that
>> maintains the plugin; they were able to reproduce the problem and are
>> working on a fix now.
>>
>> On Sun, May 8, 2016 at 6:00 AM, Yan Yan  wrote:
>>
>>>
>>> 
>>>
>>>
>>> On Saturday, May 7, 2016 at 8:00:02 AM UTC+8, Adam Cozzette wrote:

 Could you provide some more detail on where a password was required?

 On Sun, May 1, 2016 at 7:34 PM, Yan Yan  wrote:

> It worked before. Now it required a password. Any idea? Thx.
>
> --
> You received this message because you are subscribed to the Google
> Groups "Protocol Buffers" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to protobuf+u...@googlegroups.com.
> To post to this group, send email to prot...@googlegroups.com.
> Visit this group at https://groups.google.com/group/protobuf.
> For more options, visit https://groups.google.com/d/optout.
>

 --
>>> You received this message because you are subscribed to the Google
>>> Groups "Protocol Buffers" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to protobuf+u...@googlegroups.com.
>>> To post to this group, send email to prot...@googlegroups.com.
>>> Visit this group at https://groups.google.com/group/protobuf.
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>
>> --
> You received this message because you are subscribed to the Google Groups
> "Protocol Buffers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to protobuf+unsubscr...@googlegroups.com.
> To post to this group, send email to protobuf@googlegroups.com.
> Visit this group at https://groups.google.com/group/protobuf.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to protobuf+unsubscr...@googlegroups.com.
To post to this group, send email to protobuf@googlegroups.com.
Visit this group at https://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.

Re: [protobuf] How to erase particular field with MergeFrom()

2016-05-17 Thread Denis Bakhvalov 
Hi Feng Xiao,

Thanks for your reply!

Unfortunately I couldn't find good examples how to use it, so let me ask 
one more question:

Should I put mask as a part of original message:

message RepAndOpt
{
repeated string name = 1;
string surname = 2;

google.protobuf.FieldMask field_mask = 3;
}

or put it as standalone message?

message RepAndOptMask
{
google.protobuf.FieldMask field_mask = 1;
}

I think there is nothing wrong if I will send them separately, right?

-- 
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to protobuf+unsubscr...@googlegroups.com.
To post to this group, send email to protobuf@googlegroups.com.
Visit this group at https://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.

Re: [protobuf] How to erase particular field with MergeFrom()

2016-05-17 Thread Denis Bakhvalov 
Hi Feng Xiao,

Thanks for your reply!

Unfortunately I couldn't find good examples how to use it, so let me ask 
one more question:

Should I put mask as a part of original message:

message RepAndOpt
{
repeated string name = 1;
string surname = 2;

google.protobuf.FieldMask field_mask = 3;
}

or put it as standalone message?

message RepAndOptMask
{
google.protobuf.FieldMask field_mask = 1;
}

I think there is nothing wrong if I will send them separately, right?

понедельник, 16 мая 2016 г., 22:14:33 UTC+2 пользователь Feng Xiao написал:
>
>
>
> On Wed, May 11, 2016 at 5:09 AM, Denis Bakhvalov  > wrote:
>
>> Hi,
>>
>> Suppose I have such message structure:
>>
>>
>> package msg_RepAndOpt;
>>
>> message RepAndOpt{
>> repeated string name = 1;
>> optional string surname = 2;
>> ...
>> // there are lots of others.}
>>
>>
>> And I have two components that have copies of this message:
>>
>>
>> // component1:RepAndOpt A;
>> A.add_name("Name");
>> A.set_surname("Surname");
>> // component2:RepAndOpt B;
>>
>>
>> In my case components modify those messages via transaction mechanism. It 
>> means that if one component changes some field it also sends it to another 
>> component to propagate those changes. Component-receiver is doing merge:
>>
>>
>> // Component2 modifies B and sends it to component1.// Component1 perfoms 
>> merge:
>> A.MergeFrom(B);
>>
>>
>> Now, say, component2 wants to erase field "name". If it will send clear B 
>> message (default construction) than:
>>
>>- - MergeFrom() will not modify A;
>>- - CopyFrom() will erase also other fields.
>>
>> Another way will be to fill B with the contents of A, clear name field 
>> and component1 will use CopyFrom(). But this is not acceptable because 
>> system is really high-loaded and there could be lots of other fields. So, 
>> desired solution to clean name field is:
>>
>>1. 1. Component2 create B message.
>>2. 2. Explicitly stores information that it want to erase only name 
>>field.
>>3. 3. Component1 perform A.MergeFrom(B).
>>4. 4. Result: A::name is cleared but other fields are left untouched.
>>
>> As far as I tested this applies to repeated and optional fields. Is there 
>> any ready-to-use solution or I should modify protobuf implementation?
>>
> I think you can do this using FieldMaskUtil:
>
> https://github.com/google/protobuf/blob/beta-3/src/google/protobuf/util/field_mask_util.h#L111
>
> You create an empty B message along with a FieldMask containing only the 
> field you want to clear, and use FieldMaskUtil::MergeMessageTo(A, 
> field_mask, )  to replace the field in A with the value from B.
>
>  
>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Protocol Buffers" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to protobuf+u...@googlegroups.com .
>> To post to this group, send email to prot...@googlegroups.com 
>> .
>> Visit this group at https://groups.google.com/group/protobuf.
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to protobuf+unsubscr...@googlegroups.com.
To post to this group, send email to protobuf@googlegroups.com.
Visit this group at https://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.