Re: [protobuf] How to model {key:value, key:value, ...} object in the ProtoBuf

suming code to check that all values are of an expected type (e.g. string). *Josh Humphries* jh...@bluegosling.com On Thu, Nov 24, 2022 at 12:28 PM MantasS wrote: > I'm trying to model this JSON into ProtoBuff reply: > { > "0xc02aaa39b223fe8d4a0e5c4f27ead9083c756cc2": "73185

Re: [protobuf] Re: How to: get the parent message

. Something like so: func handle(msg b, path []string) { currentPath := append(path, msg.name) // ... do something with msg and currentPath ... if msg.b != nil { handle(msg.b, currentPath) } } *Josh Humphries* jh...@bluegosling.com On Tue, Nov 22, 2022 at 8:46 AM

Re: [protobuf] proto3 pretty printer

complicated custom options -- like message options, where you put a long message literal as the value -- which can lead to less-than-stellar results, aesthetically speaking. But having a standard with some warts seemed better than no standard... *Josh Humphries* jh...@bluegosling.com On Fri

Re: [protobuf] Re: Google Cloud Security - Redact Keyword?

There is no built-in way to do this. However, you can create your own custom options and use those to annotate sensitive fields/messages whose contents should be redacted. Related: https://github.com/protocolbuffers/protobuf/issues/1160 *Josh Humphries* jh...@bluegosling.com On Thu, Oct

Re: [protobuf] Feature Request: make protocol buffer AST/parser SDK

and includes position information, including whitespace and comments, for everything lexed token. https://pkg.go.dev/github.com/jhump/protoreflect/desc/protoparse/ast *Josh Humphries* jh...@bluegosling.com On Tue, Feb 16, 2021 at 7:34 PM Gerardo Mora wrote: > I'm gonna use golang for my proj

Re: [protobuf] [Proposal]: special case fieldmask behaviour for struct.proto types

to handle this way (since the embedded type must be known by the masker and the embedded bytes must be unmarshaled, masked, then re-marshaled). But it would be strictly more useful if it could work this way. *Josh Humphries* jh...@bluegosling.com On Fri, Oct 23, 2020 at 6:42 AM Johan

Re: [protobuf] protoc generates value instead of pointer (Go)

wer and you must enable the feature via a flag: --experimental_allow_proto3_optional. You also need to be using a very recent protoc-gen-go (v1.22 or newer from the google.golang.org/protobuf import path, or v1.4.1 or newer from github.com/golang/protobuf). *Josh Humphries* jh...@bluegosling.com On Thu, Jun 18,

Re: [protobuf] Proposal: a mechanism to deal with sensitive/redacted fields in string output

sensitive field. (Admittedly, this is a manufactured hypothetical.) Anyhow, where do things stand with this enhancement request? Is there anything I can do to stoke the fire and get some attention on it? ---- *Josh Humphries* jh...@bluegosling.com On Wed, Aug 22, 2018 at 10:01 AM Zellyn wrote: &

Re: [protobuf] Guidance for proto linkage

On Tue, Mar 12, 2019 at 7:31 PM Michael Powell wrote: > > > On Monday, March 11, 2019 at 12:24:57 PM UTC-4, Josh Humphries wrote: >> >> Since I've implemented this before, I have a fairly lengthy list. >> >> Most of these constraints are in the docs, at least in

Re: [protobuf] Guidance for proto linkage

r an enum *must* have a numeric value of zero. (Similarly: every enum must have a value whose numeric value is zero.) - Field definitions may not use the "default" option (default field values in proto3 are always the zero value for the field's type). - If the file indicates sy

Re: [protobuf] Java Date object to be added in .proto file

r sources in your protobuf installation and also in the repo <https://github.com/protocolbuffers/protobuf/blob/master/src/google/protobuf/timestamp.proto> . *Josh Humphries* jh...@bluegosling.com On Fri, Jan 18, 2019 at 12:27 PM Shilpa Vittal wrote: > HI, > > I have a Date java object

Re: [protobuf] Re: Question about json_name and protoc

how one-ofs are handled (e.g. one-of names never appear in JSON; their constituents are flattened into the enclosing message). > > I'm probably way too late to get it fixed, though: it looks like 3.1.0 > introduced that behavior :-( > > Zellyn > > On Wednesday, January 16, 2019 at

Re: [protobuf] Re: Question about json_name and protoc

to protoc is that it would require all code-gen plugins (and/or runtime libraries that implement the JSON encoding) to have redundant logic for computing the canonical JSON name from the proto name. *Josh Humphries* jh...@bluegosling.com On Wed, Jan 16, 2019 at 10:10 AM Zellyn wrote: > A qu

Re: [protobuf] Re: packed = true encoding in protobuf

smaller since varint encoded length of 2 will be only one byte, where as extra varint encoded tag could possibly be >1 byte, depending on the tag value). ---- *Josh Humphries* jh...@bluegosling.com On Mon, Dec 10, 2018 at 4:03 AM wrote: > The code i am using is below: > > #include

Re: [protobuf] Re: Import statement semantics

/blob/2f28980f07eb0840854cb4338b2df6007a2c56bb/src/google/protobuf/compiler/cpp/cpp_file.cc#L1202>). So it's part of the grammar, but perhaps disallowed in practice (or, more likely, ignored and treated as a normal import). *Josh Humphries* jh...@bluegosling.com On Tue, De

Re: [protobuf] Proto v2 ambiguities in parsed identifiers

estedMsg { *// fqn = foo.bar.SomeMsg.NestedMsg* extends google.protobuf.FieldOption { uint64 some_other_option = ; *// fqn = foo.bar.SomeMsg.NestedMsg.some_other_option* } } } *Josh Humphries* jh...@bluegosling.com On Thu, Nov 29, 2018 at 9:14 AM Michael Powell wrote:

Re: [protobuf] Spec v2 int-lit snafu?

n attribute (i.e. '+' or '-'). This would potentially > > work, especially when there is base 16 (hex) or base 8 (octal) > > involved. > > > > 2. Otherwise, open to suggestions, but for Qi constraints; that I know > > of, fails to parse negative signed hexadecimal/oct

Re: [protobuf] Spec v2 int-lit snafu?

. *Josh Humphries* jh...@bluegosling.com On Sat, Nov 10, 2018 at 10:16 PM Michael Powell wrote: > Hello, > > I think 0 can be a decimal-lit, don't you think? However, the spec > reads as follows: > > intLit = decimalLit | octalLit | hexLit > decimalLit = ( "

Re: [protobuf] Proto2 possible field options

On Thu, Nov 1, 2018 at 11:36 AM Michael Powell wrote: > On Thu, Nov 1, 2018 at 11:08 AM Josh Humphries > wrote: > > > > Michael, > > They map to field names in the options messages defined in > descriptor.proto. > > This is pretty interesting. So they've used th

Re: [protobuf] Re: [Proto2] Language spec help

lent descriptors to protoc.) *Josh Humphries* jh...@bluegosling.com On Wed, Oct 31, 2018 at 12:23 PM Michael Powell wrote: > On Wed, Oct 31, 2018 at 12:22 PM Michael Powell > wrote: > > > > Concerning Constant, literally from the v2 spec: > > Rather, Syntax section, excu

Re: [protobuf] Proto2 possible field options

lified name enclosed in parentheses. That proto file above also serves as a sort of AST for proto source files, where FileDescriptorProto is the root of the AST. You can then look at UninterpretedOption for seeing the AST structure for parsing options. *Josh Humphries* jh...@bluegosling.com

Re: [protobuf] Embedding arbitrary JSON in a message

this way is guaranteed to be well-formed, whereas just using a string or bytes field could be garbage that is not proper JSON. *Josh Humphries* jh...@bluegosling.com On Wed, Sep 26, 2018 at 9:44 AM Jimit Modi wrote: > Hello @Siddharth, @John, > > Did you guys found any solution aroun

Re: [protobuf] Documentation Riddle

On Wed, Sep 26, 2018 at 3:42 AM omid pourhadi wrote: > I have some questions : > > 1. Is it a good idea to use Any for transferring a serialized/deserialized > java object from client to server for example consider this ? > A google.protobuf.Any encapsulates a *protobuf message *whose type can

Re: [protobuf] Reflection of well known types in C++

h between whether the original was nil or explicit zero). *Josh Humphries* jh...@bluegosling.com On Fri, Jul 27, 2018 at 2:08 PM Chris Buffett wrote: > I'm wondering if protobuf supports a way to determine if a type is a > well-known type via reflection in C++? I'm working on a marsh

Re: [protobuf] How to generate FileDescriptor Object which has custom options?

ld behave differently -- but it could be interesting to verify whether or not it behaves differently.) > > Thanks, > Ankit > > On Friday, July 20, 2018 at 5:56:35 PM UTC-7, Josh Humphries wrote: >> >> The issue is that when you access the descriptor that way, it doe

Re: [protobuf] How to generate FileDescriptor Object which has custom options?

, so my recollection of the Java protobuf runtime might be fading a little), to re-parse them you have to serialize it to bytes (in this case, the method descriptor proto) and then de-serialize again. The resulting de-serialized message will now have the options in a queryable form. *Josh

Re: [protobuf] Embedding arbitrary JSON in a message

for JS primitive types and null). *Josh Humphries* jh...@bluegosling.com On Thu, Jun 21, 2018 at 3:25 PM, Josh Humphries wrote: > Hi, John, > Take a look at the well-known type google.protobuf.Struct. It is > basically a JSON value, modeled as a proto. It's JSON repre

Re: [protobuf] ANN: 1.0 GA of ProtoPoet, Java API for generating Protobuf files

cement about it to the go-nuts mailing list. *Josh Humphries* jh...@bluegosling.com On Sat, Jun 2, 2018 at 3:08 PM, Derek Perez wrote: > Hey all, > > I've been working on this project in my spare time and its finally at what > I would call 1.0 level. The concept was inspired by Squ

Re: [protobuf] Tool for programmatically adding field options in .proto files

preserves formatting from the original file. The formatting produced isn't bad, but it doesn't quite look like hand-written code due to the extra whitespace you'll see (blank line between every element). To preserve formatting, you probably will need to write a custom parser. YMMV *Josh Humphries* jh

Re: [protobuf] read ahead from protobuf stream

length (or fixed 32-bit length would work, too). That is a common way to define a stream of protos, and there is some library support -- at least in Java -- for the varint-encoded delimited streams. *Josh Humphries* jh...@bluegosling.com On Sat, Apr 28, 2018 at 12:02 PM, Stefan Seefeld

[protobuf] Re: latest master: "make check" failing on OS X, can't find "google/protobuf/compiler/profile.pb.h"

Never mind. It must have been some artifacts from an older version sitting in my source tree. After running "make clean" and retrying, all is well. ---- *Josh Humphries* jh...@bluegosling.com On Sun, Feb 25, 2018 at 7:36 PM, Josh Humphries <jh...@bluegosling.com> wrote: > I

[protobuf] latest master: "make check" failing on OS X, can't find "google/protobuf/compiler/profile.pb.h"

an find no such file in the repo nor can I find any .proto source file from which it would be generated. Anyway have any tips on what's wrong with my environment? *Josh Humphries* jh...@bluegosling.com -- You received this message because you are subscribed to the Google Groups &quo

Re: [protobuf] How to convert protocol-buffer to a HashMap in java?

snake-case -> camel-case with initial lower-case). So if that's what you're after, it's probably is better to just rely on JSON serialization instead of trying to reproduce that logic. *Josh Humphries* jh...@bluegosling.com On Sun, Feb 18, 2018 at 10:30 PM, Debraj Manna <subharaj.ma...@

Re: [protobuf] Where is Protocol Buffers wire level specification?

length, to delimit multiple messages in a single stream. (Though other usages of proto, such as gRPC, have different framing mechanisms for concatenating multiple messages in a single stream.) *Josh Humphries* jh...@bluegosling.com On Tue, Jan 9, 2018 at 12:01 PM, Sergei Gnezdov <sergei.g

Re: [protobuf] Compile protobuf to use Java primitive classes instead of well-known types

insertion points). *Josh Humphries* jh...@bluegosling.com On Thu, Jan 4, 2018 at 7:13 AM, Renatas M <rm.laiki...@gmail.com> wrote: > Lets say I have test.proto file: > > syntax = "proto3"; > > option java_package = "testing"; > option java_

Re: [protobuf] Is protobuf3 a superset of protobuf2?

have support for proto2. Though I thought this was a short/medium-term issue. If all languages/runtimes eventually support both, it seems like a strange decision to continue supporting both indefinitely. *Josh Humphries* jh...@bluegosling.com On Tue, Dec 19, 2017 at 2:23 PM, Adam Cozzette

Re: [protobuf] Is protobuf3 a superset of protobuf2?

be changed atomically in order to compile (at least for languages/runtimes where generated code for the two syntaxes is materially different/incompatible). I would hope that this is something that will be addressed in future versions of protobuf, to facilitate migrations away from proto2. ---- *Josh

Re: [protobuf] Is protobuf3 a superset of protobuf2?

her features can still be used with proto2 source files (aside, possibly, from using JSON format with a message with extensions). *Josh Humphries* jh...@bluegosling.com On Fri, Dec 15, 2017 at 6:56 PM, 'Adam Cozzette' via Protocol Buffers < protobuf@googlegroups.com> wrote: > Neithe

Re: [protobuf] How to combine FieldOptions with Default Values in Messages

BTW, the default option in protos does not allow you to define a default value for aggregate fields (e.g. no defaults allowed for repeated/map fields or fields whose type is a nested message). *Josh Humphries* jh...@bluegosling.com On Sun, Nov 12, 2017 at 2:11 AM, <shiyw...@re

Re: [protobuf] Re: How to use custom options in official docs

golang/protobuf/proto" ) func TestABCMessage(t *testing.T) { var msg Bar _, md := descriptor.ForMessage() opts := md.Field[0].GetOptions() foo, err := proto.GetExtension(opts, E_FooOptions) if err != nil { t.Errorf("failed: %v", err) } else { fmt.Printl

Re: [protobuf] How to use custom options in official docs

, E_FooOptions) if err != nil { fmt.Println(foo.GetOpt1()) // 123 fmt.Println(foo.GetOpt2()) // baz } opts = md.Field[1].GetOptions() foo, err = proto.GetExtension(opts, E_FooOptions) if err != nil { fmt.Println(foo.GetOpt1()) // 456 fmt.Println(foo.GetOpt2()) // xaa } *Josh Humphries* jh

[protobuf] Golang: issues/discussions/pull requests

the backlog is not very large and the rate of incoming PRs looks pretty low). But I can't even get a reply to any question or remark, much less a review. Is there another communication channel I should be trying to use? Maybe Slack? *Josh Humphries* jh...@bluegosling.com -- You received

Re: [protobuf] Are wrapper.proto messages automatically packed / unpacked with

timestamps, and durations. ---- *Josh Humphries* jh...@bluegosling.com On Tue, Oct 24, 2017 at 5:54 AM, Jack Wootton <jackwoot...@gmail.com> wrote: > I'm using wrappers.proto in my own API that sits behind Google's > Extensible Service Proxy > <https://github.com/cloudendpoints/endpoin

Re: [protobuf] How to generate google.protobuf.Type object from a file

On Fri, Oct 20, 2017 at 9:05 PM, 'Adam Cozzette' via Protocol Buffers < protobuf@googlegroups.com> wrote: > I've CC'd Jisi who can confirm this, but I believe we actually went in a > different direction and did not implement the type server idea. In practice > we use the type_url field as an

Re: [protobuf] How to generate google.protobuf.Type object from a file

/dynamic/msgregistry *Josh Humphries* jh...@bluegosling.com On Thu, Oct 19, 2017 at 2:52 PM, <pradeep...@gmail.com> wrote: > Hi All, > > The documentation for google.protobuf.Any says that an HTTP GET on the URL > should return a Type object in binary. However, I can't find a

Re: [protobuf] Nested extensions generate invalid code

(in this case, descriptor.proto is still syntax = "proto2"), so it needs to special-case extension fields. ---- *Josh Humphries* jh...@bluegosling.com On Mon, Sep 25, 2017 at 5:05 AM, Tomoyuki Saito <aocch...@gmail.com> wrote: > When trying to use extensions with a parent message, a

Re: [protobuf] How does this .proto file result in this generated code?

right protoc plugin <https://github.com/grpc/grpc-java/tree/master/compiler> to generate GRPC-related Java stuff -- like service interfaces and methods for registering a server implementation. *Josh Humphries* jh...@bluegosling.com On Fri, Jul 14, 2017 at 3:26 PM, Laird Nelson <ljn

Re: [protobuf] pros and cons to upgrade protobuf v3

but since they rely on extensions they must be defined in a file with proto2 syntax. *Josh Humphries* jh...@bluegosling.com On Fri, Jul 14, 2017 at 7:33 AM, hce h <jupiter@gmail.com> wrote: > Hi, > > I had a debate with my colleagues regarding to upgrade pr

[protobuf] [golang] API provides no way to access unknown extensions

r something more efficient -- like being able to only do this for the unrecognized extensions. (I can already do this for other unrecognized fields thanks to the export XXX_unrecognized field that gets added to generated proto2 message structs). Thoughts? *Josh Humphries* jh...@bluegosl

Re: [protobuf] Replacing 'extensions' in proto3

On Mon, Jun 5, 2017 at 4:48 PM, R.C. wrote: > Hi Bo, > > Thanks for your response. Would using 'Any' in this fashion change the > format on the wire? How can I edit the proto file to maintain the format on > the wire (maintaining backward compatibility with older message

Re: [protobuf] new protoc keywords?

in protos. *Josh Humphries* jh...@bluegosling.com On Tue, Apr 25, 2017 at 6:53 PM, Scott Lewis <scottsle...@gmail.com> wrote: > I've been happily using protocol buffers to implement Java <-> Python > interaction using OSGi Remote Services [1,2]. > > Remote Services is

Re: [protobuf] Type URLs and Any types

On Thu, Apr 20, 2017 at 2:03 PM, Feng Xiao <xiaof...@google.com> wrote: > > > On Wed, Apr 19, 2017 at 5:25 PM, Josh Humphries <jh...@bluegosling.com> > wrote: > >> The protobuf docs for the Any type >> <https://developers.google.com/proto

[protobuf] Type URLs and Any types

issue that none of the documented examples actually work. *Josh Humphries* jh...@bluegosling.com -- You received this message because you are subscribed to the Google Groups "Protocol Buffers" group. To unsubscribe from this group and stop receiving emails from it, send an email to p

Re: [protobuf] Enum values and proto3 Any types

} Unfortunately, EnumValue is a name conflict with the existing type that describes an element in the enumeration. Maybe WrappedEnumValue? *Josh Humphries* jh...@bluegosling.com On Mon, Apr 17, 2017 at 12:54 PM, Adam Cozzette <acozze...@google.com> wrote: > I think the right solution fo

[protobuf] Enum values and proto3 Any types

pe into an Any field and indicate that same URL. I then get into trouble because I'd be looking to parse a google.protobuf.Type, in order to understand the Any message contents, but actually get back the bytes for a google.protobuf.Enum. That doesn't seem sound. *Josh Humphries* jh...@bluegosling.c

Re: [protobuf] Re: why protobuf optional field does not take null

) { b.setFrobnitz(frobnitz) } return b.build(); // Better: return MyMessage.newBuilder() .setFoo(foo) .setBar(bar) .setBaz(baz) .setOrClearFrobnitz(frobnitz) .build(); *Josh Humphries* jh...@bluegosling.com On Mon, Mar 20, 2017 at 7:53 AM, Subin Sebastian <subinsebast...@gmail.

Re: [protobuf] serializing new proto message with old client (Java)

Are you using syntax = "proto3" in your file? I believe the docs state that proto3 does not require unknown fields to be preserved when a message is de-serialized and re-serialized. So this behavior, IIRC, only works for messages generated from proto2 files. ---- *Josh Hum

[protobuf] go protobuf: "rich" descriptors

without a better/more usable form of the descriptors). Is this something that would be welcome in the Go protobuf implementation? I am happy to open a pull request and contribute it. But I wasn't sure if the maintainers would prefer it remain a third-party library instead of integrating it into the core

Re: [protobuf] extending field and message options

in the generated code. *Josh Humphries* jh...@bluegosling.com On Mon, Mar 13, 2017 at 4:15 PM, Arpit Baldeva <abald...@gmail.com> wrote: > Hi, > > I have my proto file like following. > > package example; > > message ExFieldOptions > { > map<string, s

Re: [protobuf] How to resolve a custom option in a protocol buffer FileDescriptor

When you parse the serialized descriptor proto, you must supply an ExtensionRegistry and make sure that custom option is registered therein. *Josh Humphries* jh...@bluegosling.com On Tue, Mar 7, 2017 at 11:02 AM, Bill Smith <william.m.sm...@gmail.com> wrote: > I'm using protoco

Re: [protobuf] What are well-known types?

that the proto2 behavior was useful, but at the expense of on-the-wire bloat for present values. *Josh Humphries* Manager, Shared Systems | Platform Engineering Atlanta, GA | 678-400-4867 *Square* (www.squareup.com) On Tue, Feb 23, 2016 at 6:34 PM, 'Feng Xiao' via Protocol Buffers

[protobuf] Unhelpful compiler warning when a file has no syntax declaration

a discussion on this group first, to give visibility to it? Or is there some time window I should allow (few days?) before expecting any attention/comments on a pull request? Thanks in advance. *Josh Humphries* Manager, Shared Systems | Platform Engineering Atlanta, GA | 678-400-4867 *Square

[protobuf] Re: [grpc-io] Re: GRPC and Ruby, Proto language level 2

I think that would be great. Thanks, Tim! *Josh Humphries* Manager, Shared Systems | Platform Engineering Atlanta, GA | 678-400-4867 *Square* (www.squareup.com) On Fri, Jan 15, 2016 at 11:49 AM, Tim Emiola <temi...@google.com> wrote: > Hi Eric, > > We never actually publ

[protobuf] GRPC and Ruby, Proto language level 2

-protoc generated protos onto GRPC. *Josh Humphries* Manager, Shared Systems | Platform Engineering Atlanta, GA | 678-400-4867 *Square* (www.squareup.com) -- You received this message because you are subscribed to the Google Groups "Protocol Buffers" group. To unsubscribe from

Re: [protobuf] Re: Issue 491 in protobuf: Optional field throws NullPointerException when not set values.

the field when null is passed in). *Josh Humphries* Manager, Shared Systems | Platform Engineering Atlanta, GA | 678-400-4867 *Square* (www.squareup.com) On Mon, Jun 8, 2015 at 4:04 AM, Andreas V andreas.v...@gmail.com wrote: I know its an old thread but i use protobuf3 with Java

[protobuf] Proto 3... release date?

The latest version I saw, IIRC, was still an alpha (alpha 2). What is the planned release date for protobuf v3.0? *Josh Humphries* Manager, Shared Systems | Platform Engineering Atlanta, GA | 678-400-4867 *Square* (www.squareup.com) -- You received this message because you