Re: [protobuf] How to construct a Builder from a FieldDescriptor with JavaType.MESSAGE

[Jon Emerson]

Ah, thanks for the response!  That wasn't it though - Calling
.getRepeatedFieldBuilder(FieldDescriptor, int) against a Builder I'm just
starting to populate will inevitably give only IndexOutOfBounds
exceptions... Since nothing's set in the repeated field yet!

But your response did give give me a huge hint: LOTS of new functionality
for working with repeated field builders was added in the November 2014
release to Github.  I was still using the release from July.  So I didn't
see .newBuilderForField(FieldDescriptor)... that was what I needed!  After
upgrading to the latest branch I was up and going in no time.

Here's my final (working!) code:

case MESSAGE:
  Message.Builder embeddedMessageBuilder =
  messageBuilder.newBuilderForField(fieldDescriptor);
  Class type = (Class)
embeddedMessageBuilder.getDefaultInstanceForType().getClass();
  for (T t : fromDBList((BasicBSONList) object.get(fieldName),
type)) {
messageBuilder.addRepeatedField(fieldDescriptor, t);
  }
  break;

It's still a bit janky that I construct a Builder to get its
DefaultInstance to then get its Class (I do this because I call my methods
recursively, and the top-most method needs a hint as to the top-most
Object's class)... But, hey, it works for now.  If there's more elegant
ways to get a Class from a repeated FieldDescriptor, I'd love to hear it.

Thanks!!

On Mon, Jan 26, 2015 at 5:55 PM, Feng Xiao  wrote:

> As it's a repeated field, you should use "getRepeatedFieldBuilder".
>
> On Mon Jan 26 2015 at 5:38:23 PM Jon Emerson  wrote:
>
>> Hi There,
>>
>> I'm writing a utility to convert data stored in MongoDB back to protocol
>> buffer format.  My strategy is to look for each field in the protocol
>> buffer definition, and if there's a matching field from my MongoDB object,
>> then pull that data and set it on my protocol buffer builder.  This works
>> great, except for embedded messages.  I can't figure out how to create a
>> Builder for an embedded message.
>>
>> Here's my first try:
>>
>>   switch (fieldDescriptor.getJavaType()) {
>> ...
>> case MESSAGE:
>>   Class type = (Class)
>> fieldDescriptor.getDefaultValue().getClass(); // getDefaultValue throws.
>>   messageBuilder.addRepeatedField(fieldDescriptor,
>>   fromDBObject((BasicBSONObject) object.get(fieldName),
>> type));
>>   break;
>>
>> But the noted line throws with the error,
>> "java.lang.UnsupportedOperationException: FieldDescriptor.getDefaultValue()
>> called on an embedded message field."
>>
>> Here's my second try:
>>   switch (fieldDescriptor.getJavaType()) {
>> ...
>> case MESSAGE:
>>   Message.Builder embeddedMessageBuilder =
>> messageBuilder.getFieldBuilder(fieldDescriptor); // getFieldBuilder throws.
>>   Class type = (Class)
>> embeddedMessageBuilder.getDefaultInstanceForType().getClass();
>>   messageBuilder.addRepeatedField(fieldDescriptor,
>>   fromDBObject((BasicBSONObject) object.get(fieldName),
>> type));
>>   break;
>>
>> Here, the noted line throws with the error,
>> "java.lang.UnsupportedOperationException: getFieldBuilder() called on a
>> non-Message type."
>>
>> I'm not sure what else to try.  My question boils down to a very simple
>> one: Given a FieldDescriptor that describes an embedded repeated MESSAGE
>> field, how can you create a Builder for it?
>>
>> Thanks!
>> jon
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Protocol Buffers" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to protobuf+unsubscr...@googlegroups.com.
>> To post to this group, send email to protobuf@googlegroups.com.
>> Visit this group at http://groups.google.com/group/protobuf.
>> For more options, visit https://groups.google.com/d/optout.
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to protobuf+unsubscr...@googlegroups.com.
To post to this group, send email to protobuf@googlegroups.com.
Visit this group at http://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.

Re: [protobuf] How to construct a Builder from a FieldDescriptor with JavaType.MESSAGE

['Feng Xiao' via Protocol Buffers]

As it's a repeated field, you should use "getRepeatedFieldBuilder".

On Mon Jan 26 2015 at 5:38:23 PM Jon Emerson  wrote:

> Hi There,
>
> I'm writing a utility to convert data stored in MongoDB back to protocol
> buffer format.  My strategy is to look for each field in the protocol
> buffer definition, and if there's a matching field from my MongoDB object,
> then pull that data and set it on my protocol buffer builder.  This works
> great, except for embedded messages.  I can't figure out how to create a
> Builder for an embedded message.
>
> Here's my first try:
>
>   switch (fieldDescriptor.getJavaType()) {
> ...
> case MESSAGE:
>   Class type = (Class)
> fieldDescriptor.getDefaultValue().getClass(); // getDefaultValue throws.
>   messageBuilder.addRepeatedField(fieldDescriptor,
>   fromDBObject((BasicBSONObject) object.get(fieldName),
> type));
>   break;
>
> But the noted line throws with the error,
> "java.lang.UnsupportedOperationException: FieldDescriptor.getDefaultValue()
> called on an embedded message field."
>
> Here's my second try:
>   switch (fieldDescriptor.getJavaType()) {
> ...
> case MESSAGE:
>   Message.Builder embeddedMessageBuilder =
> messageBuilder.getFieldBuilder(fieldDescriptor); // getFieldBuilder throws.
>   Class type = (Class)
> embeddedMessageBuilder.getDefaultInstanceForType().getClass();
>   messageBuilder.addRepeatedField(fieldDescriptor,
>   fromDBObject((BasicBSONObject) object.get(fieldName),
> type));
>   break;
>
> Here, the noted line throws with the error,
> "java.lang.UnsupportedOperationException: getFieldBuilder() called on a
> non-Message type."
>
> I'm not sure what else to try.  My question boils down to a very simple
> one: Given a FieldDescriptor that describes an embedded repeated MESSAGE
> field, how can you create a Builder for it?
>
> Thanks!
> jon
>
> --
> You received this message because you are subscribed to the Google Groups
> "Protocol Buffers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to protobuf+unsubscr...@googlegroups.com.
> To post to this group, send email to protobuf@googlegroups.com.
> Visit this group at http://groups.google.com/group/protobuf.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to protobuf+unsubscr...@googlegroups.com.
To post to this group, send email to protobuf@googlegroups.com.
Visit this group at http://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.

Re: [protobuf] Re: Protobuf Buffers v3.0.0-alpha-1

['Feng Xiao' via Protocol Buffers]

On Mon Jan 26 2015 at 5:38:22 PM Ryan Gaudon  wrote:

> As a new user it's encouraged to begin implementation with PB3.0+ opposed
> to 2.6. With that being said, since I'm beginning integration now and
> replacing XStream with this, where would you suggest I start? There is very
> little / no documentation available at this time, and there hasn't been an
> update since.
>
Most of the documentation on our developer guide site still applies to
proto3. We don't have any documentation for proto3 specific features though.


>
> Specifically I'm curious how to go about tackling serialization of objects
> that inherit properties and methods from a parent class. IIRC, this was
> handled with extensions and .setExtension() in older version, however I was
> unable to figure out how to use the new 'any' type and the compiler threw
> errors when attempting to work with it blindly.
>
Sorry, but "Any" is not added in this alpha version. In proto3, "Any" will
be a pre-defined message type. Its definition looks like this:
package google.protobuf;
message Any {
  string type_url = 1;
  bytes value = 2;
}
You could define such a message yourself and use it as a temporary
solution. In the next alpha version we'll provide these types and utility
functions to work with an Any message.


>
> Is there anything you have available to get those started off with
> protobuf 3?
>

>
> On Thursday, December 11, 2014 at 1:21:01 AM UTC-3:30, Feng Xiao wrote:
>>
>> Hi all,
>>
>> I just published protobuf v3.0.0-alpha-1 on our github site:
>> https://github.com/google/protobuf/releases/tag/v3.0.0-alpha-1
>>
>> This is the first alpha release of protobuf v3.0.0. In protobuf v3.0.0,
>> we will add a new protobuf language version (aka proto3) and support a
>> wider range of programming languages (to name a few: ruby, php, node.js,
>> objective-c). This alpha version contains C++ and Java implementation with
>> partial proto3 support (see below for details). In future releases we will
>> add support for more programming languages and implement the full proto3
>> feature set. Besides proto3, this alpha version also includes two other new
>> features: map fields and arena allocation. They are implemented for both
>> proto3 and the old protobuf language version (aka proto2).
>>
>> We are currently working on the documentation of these new features and
>> when it's ready it will be updated to our protobuf developer guide
>> . For the
>> time being if you have any questions regarding proto3 or other new
>> features, please post your question in the discussion group.
>>
>> CHANGS
>> ===
>> Version 3.0.0-alpha-1 (C++/Java):
>>
>>   General
>>   * Introduced Protocol Buffers language version 3 (aka proto3).
>>
>> When protobuf was initially opensourced it implemented Protocol
>> Buffers
>> language version 2 (aka proto2), which is why the version number
>> started from v2.0.0. From v3.0.0, a new language version (proto3) is
>> introduced while the old version (proto2) will continue to be
>> supported.
>>
>> The main intent of introducing proto3 is to clean up protobuf before
>> pushing the language as the foundation of Google's new API platform.
>> In proto3, the language is simplified, both for ease of use and  to
>> make it available in a wider range of programming languages. At the
>> same time a few features are added to better support common idioms
>> found in APIs.
>>
>> The following are the main new features in language version 3:
>>
>>   1. Removal of field presence logic for primitive value fields,
>> removal
>>  of required fields, and removal of default values. This makes
>> proto3
>>  significantly easier to implement with open struct
>> representations,
>>  as in languages like Android Java, Objective C, or Go.
>>   2. Removal of unknown fields.
>>   3. Removal of extensions, which are instead replaced by a new
>> standard
>>  type called Any.
>>   4. Fix semantics for unknown enum values.
>>   5. Addition of maps.
>>   6. Addition of a small set of standard types for representation of
>> time,
>>  dynamic data, etc.
>>   7. A well-defined encoding in JSON as an alternative to binary proto
>>  encoding.
>>
>> This release (v3.0.0-alpha-1) includes partial proto3 support for C++
>> and
>> Java. Items 6 (well-known types) and 7 (JSON format) in the above
>> feature
>> list are not implemented.
>>
>> A new notion "syntax" is introduced to specify whether a .proto file
>> uses proto2 or proto3:
>>
>>   // foo.proto
>>   syntax = "proto3";
>>   message Bar {...}
>>
>> If omitted, the protocol compiler will generate a warning and
>> "proto2" will
>> be used as the default. This warning will be turned into an error in a
>> future release.
>>
>> We recommend that new Protocol Buffers users use proto3. However, we
>>

[protobuf] Re: Protobuf Buffers v3.0.0-alpha-1

[Ryan Gaudon]

As a new user it's encouraged to begin implementation with PB3.0+ opposed 
to 2.6. With that being said, since I'm beginning integration now and 
replacing XStream with this, where would you suggest I start? There is very 
little / no documentation available at this time, and there hasn't been an 
update since. 

Specifically I'm curious how to go about tackling serialization of objects 
that inherit properties and methods from a parent class. IIRC, this was 
handled with extensions and .setExtension() in older version, however I was 
unable to figure out how to use the new 'any' type and the compiler threw 
errors when attempting to work with it blindly. 

Is there anything you have available to get those started off with protobuf 
3?

On Thursday, December 11, 2014 at 1:21:01 AM UTC-3:30, Feng Xiao wrote:
>
> Hi all,
>
> I just published protobuf v3.0.0-alpha-1 on our github site:
> https://github.com/google/protobuf/releases/tag/v3.0.0-alpha-1
>
> This is the first alpha release of protobuf v3.0.0. In protobuf v3.0.0, we 
> will add a new protobuf language version (aka proto3) and support a wider 
> range of programming languages (to name a few: ruby, php, node.js, 
> objective-c). This alpha version contains C++ and Java implementation with 
> partial proto3 support (see below for details). In future releases we will 
> add support for more programming languages and implement the full proto3 
> feature set. Besides proto3, this alpha version also includes two other new 
> features: map fields and arena allocation. They are implemented for both 
> proto3 and the old protobuf language version (aka proto2).
>
> We are currently working on the documentation of these new features and 
> when it's ready it will be updated to our protobuf developer guide 
> . For the 
> time being if you have any questions regarding proto3 or other new 
> features, please post your question in the discussion group.
>
> CHANGS
> ===
> Version 3.0.0-alpha-1 (C++/Java):
>
>   General
>   * Introduced Protocol Buffers language version 3 (aka proto3).
>
> When protobuf was initially opensourced it implemented Protocol Buffers
> language version 2 (aka proto2), which is why the version number
> started from v2.0.0. From v3.0.0, a new language version (proto3) is
> introduced while the old version (proto2) will continue to be 
> supported.
>
> The main intent of introducing proto3 is to clean up protobuf before
> pushing the language as the foundation of Google's new API platform.
> In proto3, the language is simplified, both for ease of use and  to
> make it available in a wider range of programming languages. At the
> same time a few features are added to better support common idioms
> found in APIs.
>
> The following are the main new features in language version 3:
>
>   1. Removal of field presence logic for primitive value fields, 
> removal
>  of required fields, and removal of default values. This makes 
> proto3
>  significantly easier to implement with open struct 
> representations,
>  as in languages like Android Java, Objective C, or Go.
>   2. Removal of unknown fields.
>   3. Removal of extensions, which are instead replaced by a new 
> standard
>  type called Any.
>   4. Fix semantics for unknown enum values.
>   5. Addition of maps.
>   6. Addition of a small set of standard types for representation of 
> time,
>  dynamic data, etc.
>   7. A well-defined encoding in JSON as an alternative to binary proto
>  encoding.
>
> This release (v3.0.0-alpha-1) includes partial proto3 support for C++ 
> and
> Java. Items 6 (well-known types) and 7 (JSON format) in the above 
> feature
> list are not implemented.
>
> A new notion "syntax" is introduced to specify whether a .proto file
> uses proto2 or proto3:
>
>   // foo.proto
>   syntax = "proto3";
>   message Bar {...}
>
> If omitted, the protocol compiler will generate a warning and "proto2" 
> will
> be used as the default. This warning will be turned into an error in a
> future release.
>
> We recommend that new Protocol Buffers users use proto3. However, we 
> do not
> generally recommend that existing users migrate from proto2 from 
> proto3 due
> to API incompatibility, and we will continue to support proto2 for a 
> long
> time.
>
>   * Added support for map fields (implemented in C++/Java for both proto2 
> and
> proto3).
>
> Map fields can be declared using the following syntax:
>
>   message Foo {
> map values = 1;
>   }
>
> Data of a map field will be stored in memory as an unordered map and it
> can be accessed through generated accessors.
>
>   C++
>   * Added arena allocation support (for both proto2 and proto3).
>
> Profiling shows memory allocation and deallocation constitutes a 
> s

[protobuf] How to construct a Builder from a FieldDescriptor with JavaType.MESSAGE

[Jon Emerson]

Hi There,

I'm writing a utility to convert data stored in MongoDB back to protocol 
buffer format.  My strategy is to look for each field in the protocol 
buffer definition, and if there's a matching field from my MongoDB object, 
then pull that data and set it on my protocol buffer builder.  This works 
great, except for embedded messages.  I can't figure out how to create a 
Builder for an embedded message.

Here's my first try:

  switch (fieldDescriptor.getJavaType()) {
...
case MESSAGE:
  Class type = (Class) 
fieldDescriptor.getDefaultValue().getClass(); // getDefaultValue throws.
  messageBuilder.addRepeatedField(fieldDescriptor,
  fromDBObject((BasicBSONObject) object.get(fieldName), 
type));
  break;

But the noted line throws with the error, 
"java.lang.UnsupportedOperationException: FieldDescriptor.getDefaultValue() 
called on an embedded message field."

Here's my second try:
  switch (fieldDescriptor.getJavaType()) {
...
case MESSAGE:
  Message.Builder embeddedMessageBuilder = 
messageBuilder.getFieldBuilder(fieldDescriptor); // getFieldBuilder throws.
  Class type = (Class) 
embeddedMessageBuilder.getDefaultInstanceForType().getClass();
  messageBuilder.addRepeatedField(fieldDescriptor,
  fromDBObject((BasicBSONObject) object.get(fieldName), 
type));
  break;

Here, the noted line throws with the error, 
"java.lang.UnsupportedOperationException: getFieldBuilder() called on a 
non-Message type."

I'm not sure what else to try.  My question boils down to a very simple 
one: Given a FieldDescriptor that describes an embedded repeated MESSAGE 
field, how can you create a Builder for it?

Thanks!
jon

-- 
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to protobuf+unsubscr...@googlegroups.com.
To post to this group, send email to protobuf@googlegroups.com.
Visit this group at http://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.