Re: [protobuf] [Java] Proper way to add an item to a List?

[Laird Nelson]

On Tuesday, July 11, 2017 at 5:09:30 PM UTC-7, Feng Xiao wrote:
>
> Actually you can't add a sub-builder to another builder. If you look at 
> the implementation of addDependencies(Builder), it just calls 
> builder.build() right away and add the resulted Chart object:
>
> https://microbean.github.io/microbean-helm/apidocs/src-html/hapi/chart/ChartOuterClass.Chart.Builder.html#line.1680
>

You're quite right of course; not sure how I missed that.
 

> You can instead get a sub-builder out from an existing builder, by calling 
> the getFieldBuilder() method, e.g., getDependenciesBuilder(int). This 
> sub-builder you get is "owned" by the parent builder and they are 
> cross-referenced: if you call build() on the parent builder, it will 
> recursively call build() on all its sub-builders; if you change the 
> sub-builder, its parent builder will get notified for the change.
>

And I add a sub-builder by...calling...addDependenciesBuilder(), with no 
arguments.  I guess.  This all feels very, very weird.

I wish I could use the actual top-level Chart.Builder to keep track of his 
tree of dependent Chart.Builders, but it looks like without some serious 
gymnastics this will be harder than just maintaining the hierarchy by hand 
out-of-band and then adding the immutable objects in the right places. 
 This feels cumbersome, which was why I was looking for a simpler way to do 
it.  Looks like there is no simpler way to do it.  Many thanks for your 
reply.

Best,
Laird

-- 
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to protobuf+unsubscr...@googlegroups.com.
To post to this group, send email to protobuf@googlegroups.com.
Visit this group at https://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.

Re: [protobuf] Need help with compilation of protobuf with a large enum for java

['Feng Xiao' via Protocol Buffers]

On Tue, Jul 11, 2017 at 5:05 PM, Minimol Thomas 
wrote:

> Thank you Feng for replying. Are there any plans to back port these
> changes to 2.6.1 ? Could you please point me to a commit id ?
>
There isn't any plan to port things back to 2.6.1 at the moment. The change
was implemented inside Google and is part of this enormous integration
commit:
https://github.com/google/protobuf/commit/cf14183bcd5485b4a71541599ddce0b35eb71352#diff-f6bd6179dc2afd0ccf0d9d113c95f2e1

I believe the change in java_enum.cc is what brings down the code size a
little bit:
https://github.com/google/protobuf/blob/cf14183bcd5485b4a71541599ddce0b35eb71352/src/google/protobuf/compiler/java/java_enum.cc#L95




>
> On Tuesday, July 11, 2017 at 1:58:38 PM UTC-7, Feng Xiao wrote:
>>
>>
>>
>> On Tue, Jul 11, 2017 at 1:06 PM, Minimol Thomas 
>> wrote:
>>
>>> Compilation of java code generated using protobuf compiler fails giving
>>> the error, "code too large".
>>> We have an enumeration with 6000 values. And the code generated for this
>>> enum is causing the problem.
>>> I would not be able to make any changes to the enumeration as the
>>> protofile is owned/maintained by another team.
>>> The java code generated using protobuf compiler 3 for same protofile
>>> compiles without any issues.
>>> Do we have any option to continue using protobuf compiler version 2.6.1
>>> and solve "code too large" problem.
>>> Note: I am already using "option java_multiple_files = true;"
>>>
>>
>>>
>> Please suggest
>>>
>> I remember we implemented something in protobuf to reduce the code size
>> for enums and that's probably why you don't get the error with newer
>> version of protocol compiler. If you have to use protobuf 2.6.1, I think
>> you will have to patch up protoc 2.6.1 yourself and adopt the same fix used
>> in latest version of protoc.
>>
>>
>>>
>>> Thanks,
>>> Minimol
>>>
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "Protocol Buffers" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to protobuf+u...@googlegroups.com.
>>> To post to this group, send email to prot...@googlegroups.com.
>>> Visit this group at https://groups.google.com/group/protobuf.
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>
>> --
> You received this message because you are subscribed to the Google Groups
> "Protocol Buffers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to protobuf+unsubscr...@googlegroups.com.
> To post to this group, send email to protobuf@googlegroups.com.
> Visit this group at https://groups.google.com/group/protobuf.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to protobuf+unsubscr...@googlegroups.com.
To post to this group, send email to protobuf@googlegroups.com.
Visit this group at https://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.

Re: [protobuf] [Java] Proper way to add an item to a List?

['Feng Xiao' via Protocol Buffers]

On Tue, Jul 11, 2017 at 4:53 PM, Laird Nelson  wrote:

> I hope this is the right group.
>
> I've generated a Java API for Helm's Tiller using gRPC and protocol
> buffers.  You can see the javadocs here: https://microbean.
> github.io/microbean-helm/apidocs/index.html  Look at the hapi.* packages.
>
> As I'm working with the generated API, I need to add one kind of thing to
> another thing's list of such things.
>
> Specifically (and arbitrarily) I need to add a (sub-) Chart.Builder to a
> (top-level) Chart.Builder's "dependencies", only if it isn't already
> present.
>
> I can do this via the following generated methods:
> https://microbean.github.io/microbean-helm/apidocs/hapi/
> chart/ChartOuterClass.Chart.Builder.html#addDependencies-
> hapi.chart.ChartOuterClass.Chart.Builder- (Adds a Chart.Builder)
> https://microbean.github.io/microbean-helm/apidocs/hapi/
> chart/ChartOuterClass.Chart.Builder.html#addDependenciesBuilder-- (I
> don't know what this does.)
>
> It seems from looking at the generated source code that if I call build()
> on the "top level" Chart.Builder, it will cascade to the Chart.Builders
> present in the return value of getDependenciesBuilderList() and call
> build() on them too.
>
Actually you can't add a sub-builder to another builder. If you look at the
implementation of addDependencies(Builder), it just calls builder.build()
right away and add the resulted Chart object:
https://microbean.github.io/microbean-helm/apidocs/src-html/hapi/chart/ChartOuterClass.Chart.Builder.html#line.1680

You can instead get a sub-builder out from an existing builder, by calling
the getFieldBuilder() method, e.g., getDependenciesBuilder(int). This
sub-builder you get is "owned" by the parent builder and they are
cross-referenced: if you call build() on the parent builder, it will
recursively call build() on all its sub-builders; if you change the
sub-builder, its parent builder will get notified for the change.


>
> I also need to add a Chart.Builder only if it isn't present already.
>
> When I tried to implement this logic, I found that no Chart.Builder added
> is equal to any other Chart.Builder.  That is, there doesn't seem to be the
> ability to check for a Chart.Builder "in" another Chart.Builder's list of
> dependencies.
>
I think you can have some kind of id field in your Chart message and use
that to check whether a Chart is already added in dependencies?


> Finally at the end of all this, I call build() on the top-level
> Chart.Builder and expect it to produce a Chart with some subcharts in its
> dependencies.  Should I in fact expect this?
>
> Now I feel like I have the semantics about these generated methods all
> wrong.  Is there a better walkthrough than https://developers.
> google.com/protocol-buffers/docs/javatutorial?  Is there a standard way
> for assembling an object graph out of connected builders?
>
> Thanks,
> Best,
> Laird
>
> --
> You received this message because you are subscribed to the Google Groups
> "Protocol Buffers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to protobuf+unsubscr...@googlegroups.com.
> To post to this group, send email to protobuf@googlegroups.com.
> Visit this group at https://groups.google.com/group/protobuf.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to protobuf+unsubscr...@googlegroups.com.
To post to this group, send email to protobuf@googlegroups.com.
Visit this group at https://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.

Re: [protobuf] Need help with compilation of protobuf with a large enum for java

[Minimol Thomas]

Thank you Feng for replying. Are there any plans to back port these changes 
to 2.6.1 ? Could you please point me to a commit id ?

On Tuesday, July 11, 2017 at 1:58:38 PM UTC-7, Feng Xiao wrote:
>
>
>
> On Tue, Jul 11, 2017 at 1:06 PM, Minimol Thomas  > wrote:
>
>> Compilation of java code generated using protobuf compiler fails giving 
>> the error, "code too large".
>> We have an enumeration with 6000 values. And the code generated for this 
>> enum is causing the problem.
>> I would not be able to make any changes to the enumeration as the 
>> protofile is owned/maintained by another team.
>> The java code generated using protobuf compiler 3 for same protofile 
>> compiles without any issues.
>> Do we have any option to continue using protobuf compiler version 2.6.1 
>> and solve "code too large" problem.
>> Note: I am already using "option java_multiple_files = true;" 
>>
>  
>>
> Please suggest
>>
> I remember we implemented something in protobuf to reduce the code size 
> for enums and that's probably why you don't get the error with newer 
> version of protocol compiler. If you have to use protobuf 2.6.1, I think 
> you will have to patch up protoc 2.6.1 yourself and adopt the same fix used 
> in latest version of protoc.
>  
>
>>
>> Thanks,
>> Minimol
>>
>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Protocol Buffers" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to protobuf+u...@googlegroups.com .
>> To post to this group, send email to prot...@googlegroups.com 
>> .
>> Visit this group at https://groups.google.com/group/protobuf.
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to protobuf+unsubscr...@googlegroups.com.
To post to this group, send email to protobuf@googlegroups.com.
Visit this group at https://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.

[protobuf] [Java] Proper way to add an item to a List?

[Laird Nelson]

I hope this is the right group.

I've generated a Java API for Helm's Tiller using gRPC and protocol 
buffers.  You can see the javadocs 
here: https://microbean.github.io/microbean-helm/apidocs/index.html  Look 
at the hapi.* packages.

As I'm working with the generated API, I need to add one kind of thing to 
another thing's list of such things.

Specifically (and arbitrarily) I need to add a (sub-) Chart.Builder to a 
(top-level) Chart.Builder's "dependencies", only if it isn't already 
present.

I can do this via the following generated methods:
https://microbean.github.io/microbean-helm/apidocs/hapi/chart/ChartOuterClass.Chart.Builder.html#addDependencies-hapi.chart.ChartOuterClass.Chart.Builder-
 
(Adds a Chart.Builder)
https://microbean.github.io/microbean-helm/apidocs/hapi/chart/ChartOuterClass.Chart.Builder.html#addDependenciesBuilder--
 
(I don't know what this does.)

It seems from looking at the generated source code that if I call build() 
on the "top level" Chart.Builder, it will cascade to the Chart.Builders 
present in the return value of getDependenciesBuilderList() and call 
build() on them too.

I also need to add a Chart.Builder only if it isn't present already.

When I tried to implement this logic, I found that no Chart.Builder added 
is equal to any other Chart.Builder.  That is, there doesn't seem to be the 
ability to check for a Chart.Builder "in" another Chart.Builder's list of 
dependencies.

Finally at the end of all this, I call build() on the top-level 
Chart.Builder and expect it to produce a Chart with some subcharts in its 
dependencies.  Should I in fact expect this?

Now I feel like I have the semantics about these generated methods all 
wrong.  Is there a better walkthrough 
than https://developers.google.com/protocol-buffers/docs/javatutorial?  Is 
there a standard way for assembling an object graph out of connected 
builders?

Thanks,
Best,
Laird

-- 
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to protobuf+unsubscr...@googlegroups.com.
To post to this group, send email to protobuf@googlegroups.com.
Visit this group at https://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.

[protobuf] Re: use of banned API functions on the ProtoBuf library

[Michael Muriuki]

Thank you Thomas.

This is clear now. 

On Monday, July 10, 2017 at 2:38:48 PM UTC, Thomas Van Lenten wrote:
>
> If you look at the calling code, they can't ever overflow; most of the 
> concern with these type of functions if when taking input from a third 
> party, and these are used between the generator code and runtime, so both 
> are known sources.  The page you linked to also talks about Microsoft only 
> replacements, so I'm not sure how much I'd take the advice of that pages as 
> the replacements don't exist on all platforms.
>
> TVL
>
>
> On Friday, July 7, 2017 at 1:57:28 PM UTC-4, Michael Muriuki wrote:
>>
>> Hi,
>>
>> Am new to the ProtoBuf library and only use it as part of the Google's 
>> libraries. Recently our security team indicated that the library in iOS 
>> uses some of the banned 
>>  API functions 
>> listed h ere. 
>> Does anyone know why these have not been replaced with the safer 
>> alternatives 
>> and what measures are in place to ensure that the code is not susceptible 
>> to buffer overflow injection?
>>
>> The functions *strlen, memcpy* and *memmove* are used in the following 
>> Protobuf code.
>>
>> GPBCodedOutputStream.h
>> GPBCodedOutputStream.h
>> GPBDescriptor.h
>> GPBDescriptor.m
>> GPBMessage.h
>> GPBMessage.m
>> GPBRootObject.h
>> GPBRootObject.h
>>
>
-- 


*Cellulant Group email disclaimer and confidentiality note*

Please go here 

 to 
read our email disclaimer and confidentiality note. 

-- 
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to protobuf+unsubscr...@googlegroups.com.
To post to this group, send email to protobuf@googlegroups.com.
Visit this group at https://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.

Re: [protobuf] Need help with compilation of protobuf with a large enum for java

['Feng Xiao' via Protocol Buffers]

On Tue, Jul 11, 2017 at 1:06 PM, Minimol Thomas 
wrote:

> Compilation of java code generated using protobuf compiler fails giving
> the error, "code too large".
> We have an enumeration with 6000 values. And the code generated for this
> enum is causing the problem.
> I would not be able to make any changes to the enumeration as the
> protofile is owned/maintained by another team.
> The java code generated using protobuf compiler 3 for same protofile
> compiles without any issues.
> Do we have any option to continue using protobuf compiler version 2.6.1
> and solve "code too large" problem.
> Note: I am already using "option java_multiple_files = true;"
>

>
Please suggest
>
I remember we implemented something in protobuf to reduce the code size for
enums and that's probably why you don't get the error with newer version of
protocol compiler. If you have to use protobuf 2.6.1, I think you will have
to patch up protoc 2.6.1 yourself and adopt the same fix used in latest
version of protoc.


>
> Thanks,
> Minimol
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Protocol Buffers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to protobuf+unsubscr...@googlegroups.com.
> To post to this group, send email to protobuf@googlegroups.com.
> Visit this group at https://groups.google.com/group/protobuf.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to protobuf+unsubscr...@googlegroups.com.
To post to this group, send email to protobuf@googlegroups.com.
Visit this group at https://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.

[protobuf] Need help with compilation of protobuf with a large enum for java

[Minimol Thomas]

Compilation of java code generated using protobuf compiler fails giving the 
error, "code too large".
We have an enumeration with 6000 values. And the code generated for this 
enum is causing the problem.
I would not be able to make any changes to the enumeration as the protofile 
is owned/maintained by another team.
The java code generated using protobuf compiler 3 for same protofile 
compiles without any issues.
Do we have any option to continue using protobuf compiler version 2.6.1 and 
solve "code too large" problem.
Note: I am already using "option java_multiple_files = true;"
Please suggest

Thanks,
Minimol


-- 
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to protobuf+unsubscr...@googlegroups.com.
To post to this group, send email to protobuf@googlegroups.com.
Visit this group at https://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.

Re: [protobuf] Re: MessageToJsonString output empty

['Feng Xiao' via Protocol Buffers]

On Sun, Jul 9, 2017 at 4:52 AM, hce h  wrote:

> Let me add more details, here is what I have a test program:
>
> test.proto:
>
> syntax = "proto3";
> package test;
>
> message TestMessage {
> string query = 1;
> int32 page_number = 2;  // Which page number do we want?
> int32 result_per_page = 3;  // Number of results to return per page.
> }
>
> test::TestMessage message;
>
If you don't set any value in the message, the JSON output is expected to
be empty.

If you want to print out values even not set, try the
always_print_primitive_fields flag:
https://github.com/google/protobuf/blob/master/src/google/protobuf/util/json_util.h#L63


> std::string output;
> google::protobuf::util::MessageToJsonString(message, );
> printf("Convert: %d %s\n", output.length(), output.c_str());
>
> Convert: 2 {}
>
>
>
>
>
>
> On Sunday, July 9, 2017 at 1:01:39 PM UTC+10, hce h wrote:
>>
>> Hi,
>>
>> I tried to call MessageToJsonString on Protobuf 3.3.0, but the output string 
>> was empty, output = "{}". Has anyone made it work? If so, could you please 
>> show me an example?
>>
>> Thank you.
>>
>>
>> --
> You received this message because you are subscribed to the Google Groups
> "Protocol Buffers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to protobuf+unsubscr...@googlegroups.com.
> To post to this group, send email to protobuf@googlegroups.com.
> Visit this group at https://groups.google.com/group/protobuf.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to protobuf+unsubscr...@googlegroups.com.
To post to this group, send email to protobuf@googlegroups.com.
Visit this group at https://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.

[protobuf] Re: Protobuf with JS

[Dylan Taft]

I keep answering my own questions.
https://developers.google.com/closure/library/docs/depswriter

Closure doesn't handle dependency walking in the browser, you either 
compile your scripts for prod or build a dependency file which depswriter 
can help with.
Perfect.

I am going to digest this a bit, type up brief instructions, and submit a 
pull request for ProtoBuf's readme documentation for JS.

-- 
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to protobuf+unsubscr...@googlegroups.com.
To post to this group, send email to protobuf@googlegroups.com.
Visit this group at https://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.

[protobuf] Re: Protobuf with JS

[Dylan Taft]

These files seem to be all that is required for closurebuilder.py to do 
it's thing.

arith.js  constants.js  decoder.js  encoder.js  map.js  message.js 
 reader.js  utils.js  writer.js

If there isn't a better way to do it - I can submit pull req 
to https://github.com/google/protobuf/blob/master/js/README.md
"The generated code will also goog.require() many types in the core 
library, and they will require many types in the Google Closure library. So 
make sure that your goog.provide() / goog.require() setup can find all of 
your generated code, the core library .js files in this directory, and the 
Google Closure library itself."
Specifically this verbiage is confusing.  The best I can come up with is 
using closurebuilder.py to find the dependencies and output code, or 
compile it with closure.

I can't find anywhere what Closure's "search path" is, or how it handles 
dependency walking for including files.  


-- 
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to protobuf+unsubscr...@googlegroups.com.
To post to this group, send email to protobuf@googlegroups.com.
Visit this group at https://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.