Re: [protobuf] Embedding arbitrary JSON in a message
If you are using JavaScript, there may not be a good solution. Ironically,
the JavaScript implementation of protos does not support the canonical JSON
representation for protobuf types (which would be needed to convert
arbitrary JSON into a google.protobuf.Value).
Most other languages, though, should work fine. Here's a sketch in Go:
*example.proto*
syntax = "proto3";
import "google/protobuf/struct.proto";
message SomeMessage {
google.protobuf.Value arbitrary_json = 1;
}
*example.go*
package example
import "github.com/golang/protobuf/ptypes/struct"
import "github.com/golang/protobuf/jsonpb"
func example(json string) (*SomeMessage, error) {
msg := {ArbitraryJson: {}}
jsm := jsonpb.Marshaler{}
if err := jsm.Unmarshal(json, msg.ArbitraryJson); err != nil {
return nil, err
}
return msg, nil
}
You could also use the "encoding/json" package to then convert any kind of
Go value into a google.protobuf.Value: you just have to marshal to JSON
first, and from there into the protobuf. While using JSON as intermediate
format is not efficient, you do get the benefit that a
google.protobuf.Value created this way is guaranteed to be well-formed,
whereas just using a string or bytes field could be garbage that is not
proper JSON.
*Josh Humphries*
jh...@bluegosling.com
On Wed, Sep 26, 2018 at 9:44 AM Jimit Modi wrote:
> Hello @Siddharth, @John,
>
> Did you guys found any solution around it. We are also stuck at very same
> thing.
>
> On Friday, August 17, 2018 at 3:36:29 PM UTC+5:30, Siddharth Kherada wrote:
>>
>> Can I please get an example of how to do this is any language with proto
>> file?
>>
>> I desperately need help with this. I am stuck for a week now on this
>> issue.
>>
>> Thanks,
>> Sid
>>
>> On Thursday, 21 June 2018 15:32:46 UTC-4, Josh Humphries wrote:
>>>
>>> Oops, I meant to point you to google.protobuf.Value:
>>> https://github.com/google/protobuf/blob/master/src/google/protobuf/struct.proto#L63
>>>
>>> It can represent *any* kind of JSON value. The Struct type is what is
>>> used to represent JSON *objects* (there is also ListValue, for arrays,
>>> as well as support for JS primitive types and null).
>>>
>>>
>>>
>>> *Josh Humphries*
>>> jh...@bluegosling.com
>>>
>>> On Thu, Jun 21, 2018 at 3:25 PM, Josh Humphries
>>> wrote:
>>>
Hi, John,
Take a look at the well-known type google.protobuf.Struct. It is
basically a JSON value, modeled as a proto. It's JSON representation is
exactly what you want, too:
https://github.com/google/protobuf/blob/master/src/google/protobuf/struct.proto#L52
*Josh Humphries*
jh...@bluegosling.com
On Thu, Jun 21, 2018 at 3:20 PM, John Lilley wrote:
> Disclaimer: I am totally new to protobuf, engaged in an exploratory
> POC. Please forgive dumb questions :-)
>
> We are looking at migrating an existing, JSON-based protocol in which
> hand-coded C++ is written to perform serdes between objects and JSON. We
> want to replace the hand-coding with an automated approach that can be
> shared between C++ and Java. However, a stumbling block I see is that
> some
> messages have an arbitrary field full of JSON like:
>
> {
>"name":"john",
>"address":"123 main st",
>"attributes":{ any JSON can go here }
> }
>
> While I realize that we could stringify the JSON, this breaks our
> published API. Is there any way I can use protobuf to perform serdes
> between message like this and some struct like:
>
> {
>string name;
>string address;
>json attributes;
> }
>
> I'm even OK if the internal data is stringified JSON:
>
> {
>string name;
>string address;
>string attributes;
> }
>
> So long as the exchanged JSON isn't stringified. In other words, this
> is bad:
> {
>"name":"john",
>"address":"123 main st",
>"attributes":"{ \"attr1\":\"value1\", \"attr2\":[\"elem1\",
> \"elem2\"] }"
> }
>
> It needs to be exchanged like
> {
>"name":"john",
>"address":"123 main st",
>"attributes":{ "attr1":"value1", "attr2":["elem1", "elem2"] }"
> }
>
> Is this possible?
>
> Thanks
> john
>
> --
> You received this message because you are subscribed to the Google
> Groups "Protocol Buffers" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to protobuf+u...@googlegroups.com.
> To post to this group, send email to prot...@googlegroups.com.
> Visit this group at https://groups.google.com/group/protobuf.
> For more options, visit https://groups.google.com/d/optout.
>
>>> --
> You received this message because you are subscribed to the Google Groups
> "Protocol Buffers" group.
> To unsubscribe from this group and stop
Re: [protobuf] Embedding arbitrary JSON in a message
Hello @Siddharth, @John,
Did you guys found any solution around it. We are also stuck at very same
thing.
On Friday, August 17, 2018 at 3:36:29 PM UTC+5:30, Siddharth Kherada wrote:
>
> Can I please get an example of how to do this is any language with proto
> file?
>
> I desperately need help with this. I am stuck for a week now on this issue.
>
> Thanks,
> Sid
>
> On Thursday, 21 June 2018 15:32:46 UTC-4, Josh Humphries wrote:
>>
>> Oops, I meant to point you to google.protobuf.Value:
>> https://github.com/google/protobuf/blob/master/src/google/protobuf/struct.proto#L63
>>
>> It can represent *any* kind of JSON value. The Struct type is what is
>> used to represent JSON *objects* (there is also ListValue, for arrays,
>> as well as support for JS primitive types and null).
>>
>>
>>
>> *Josh Humphries*
>> jh...@bluegosling.com
>>
>> On Thu, Jun 21, 2018 at 3:25 PM, Josh Humphries > > wrote:
>>
>>> Hi, John,
>>> Take a look at the well-known type google.protobuf.Struct. It is
>>> basically a JSON value, modeled as a proto. It's JSON representation is
>>> exactly what you want, too:
>>>
>>>
>>> https://github.com/google/protobuf/blob/master/src/google/protobuf/struct.proto#L52
>>>
>>>
>>>
>>>
>>> *Josh Humphries*
>>> jh...@bluegosling.com
>>>
>>> On Thu, Jun 21, 2018 at 3:20 PM, John Lilley >> > wrote:
>>>
Disclaimer: I am totally new to protobuf, engaged in an exploratory
POC. Please forgive dumb questions :-)
We are looking at migrating an existing, JSON-based protocol in which
hand-coded C++ is written to perform serdes between objects and JSON. We
want to replace the hand-coding with an automated approach that can be
shared between C++ and Java. However, a stumbling block I see is that
some
messages have an arbitrary field full of JSON like:
{
"name":"john",
"address":"123 main st",
"attributes":{ any JSON can go here }
}
While I realize that we could stringify the JSON, this breaks our
published API. Is there any way I can use protobuf to perform serdes
between message like this and some struct like:
{
string name;
string address;
json attributes;
}
I'm even OK if the internal data is stringified JSON:
{
string name;
string address;
string attributes;
}
So long as the exchanged JSON isn't stringified. In other words, this
is bad:
{
"name":"john",
"address":"123 main st",
"attributes":"{ \"attr1\":\"value1\", \"attr2\":[\"elem1\",
\"elem2\"] }"
}
It needs to be exchanged like
{
"name":"john",
"address":"123 main st",
"attributes":{ "attr1":"value1", "attr2":["elem1", "elem2"] }"
}
Is this possible?
Thanks
john
--
You received this message because you are subscribed to the Google
Groups "Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to protobuf+u...@googlegroups.com .
To post to this group, send email to prot...@googlegroups.com
.
Visit this group at https://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.
>>>
>>>
>>
--
You received this message because you are subscribed to the Google Groups
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to protobuf+unsubscr...@googlegroups.com.
To post to this group, send email to protobuf@googlegroups.com.
Visit this group at https://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.
Re: [protobuf] Embedding arbitrary JSON in a message
Can I please get an example of how to do this is any language with proto
file?
I desperately need help with this. I am stuck for a week now on this issue.
Thanks,
Sid
On Thursday, 21 June 2018 15:32:46 UTC-4, Josh Humphries wrote:
>
> Oops, I meant to point you to google.protobuf.Value:
> https://github.com/google/protobuf/blob/master/src/google/protobuf/struct.proto#L63
>
> It can represent *any* kind of JSON value. The Struct type is what is
> used to represent JSON *objects* (there is also ListValue, for arrays, as
> well as support for JS primitive types and null).
>
>
>
> *Josh Humphries*
> jh...@bluegosling.com
>
> On Thu, Jun 21, 2018 at 3:25 PM, Josh Humphries
> wrote:
>
>> Hi, John,
>> Take a look at the well-known type google.protobuf.Struct. It is
>> basically a JSON value, modeled as a proto. It's JSON representation is
>> exactly what you want, too:
>>
>>
>> https://github.com/google/protobuf/blob/master/src/google/protobuf/struct.proto#L52
>>
>>
>>
>>
>> *Josh Humphries*
>> jh...@bluegosling.com
>>
>> On Thu, Jun 21, 2018 at 3:20 PM, John Lilley wrote:
>>
>>> Disclaimer: I am totally new to protobuf, engaged in an exploratory
>>> POC. Please forgive dumb questions :-)
>>>
>>> We are looking at migrating an existing, JSON-based protocol in which
>>> hand-coded C++ is written to perform serdes between objects and JSON. We
>>> want to replace the hand-coding with an automated approach that can be
>>> shared between C++ and Java. However, a stumbling block I see is that some
>>> messages have an arbitrary field full of JSON like:
>>>
>>> {
>>>"name":"john",
>>>"address":"123 main st",
>>>"attributes":{ any JSON can go here }
>>> }
>>>
>>> While I realize that we could stringify the JSON, this breaks our
>>> published API. Is there any way I can use protobuf to perform serdes
>>> between message like this and some struct like:
>>>
>>> {
>>>string name;
>>>string address;
>>>json attributes;
>>> }
>>>
>>> I'm even OK if the internal data is stringified JSON:
>>>
>>> {
>>>string name;
>>>string address;
>>>string attributes;
>>> }
>>>
>>> So long as the exchanged JSON isn't stringified. In other words, this
>>> is bad:
>>> {
>>>"name":"john",
>>>"address":"123 main st",
>>>"attributes":"{ \"attr1\":\"value1\", \"attr2\":[\"elem1\",
>>> \"elem2\"] }"
>>> }
>>>
>>> It needs to be exchanged like
>>> {
>>>"name":"john",
>>>"address":"123 main st",
>>>"attributes":{ "attr1":"value1", "attr2":["elem1", "elem2"] }"
>>> }
>>>
>>> Is this possible?
>>>
>>> Thanks
>>> john
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "Protocol Buffers" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to protobuf+unsubscr...@googlegroups.com.
>>> To post to this group, send email to protobuf@googlegroups.com.
>>> Visit this group at https://groups.google.com/group/protobuf.
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>
>>
>
--
You received this message because you are subscribed to the Google Groups
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to protobuf+unsubscr...@googlegroups.com.
To post to this group, send email to protobuf@googlegroups.com.
Visit this group at https://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.
Re: [protobuf] Embedding arbitrary JSON in a message
Oops, I meant to point you to google.protobuf.Value: https:/
/github.com/google/protobuf/blob/master/src/google/protobuf/struct.proto#L63
It can represent *any* kind of JSON value. The Struct type is what is used
to represent JSON *objects* (there is also ListValue, for arrays, as well
as support for JS primitive types and null).
*Josh Humphries*
jh...@bluegosling.com
On Thu, Jun 21, 2018 at 3:25 PM, Josh Humphries
wrote:
> Hi, John,
> Take a look at the well-known type google.protobuf.Struct. It is
> basically a JSON value, modeled as a proto. It's JSON representation is
> exactly what you want, too:
>
> https://github.com/google/protobuf/blob/master/src/google/
> protobuf/struct.proto#L52
>
>
>
>
> *Josh Humphries*
> jh...@bluegosling.com
>
> On Thu, Jun 21, 2018 at 3:20 PM, John Lilley wrote:
>
>> Disclaimer: I am totally new to protobuf, engaged in an exploratory POC.
>> Please forgive dumb questions :-)
>>
>> We are looking at migrating an existing, JSON-based protocol in which
>> hand-coded C++ is written to perform serdes between objects and JSON. We
>> want to replace the hand-coding with an automated approach that can be
>> shared between C++ and Java. However, a stumbling block I see is that some
>> messages have an arbitrary field full of JSON like:
>>
>> {
>>"name":"john",
>>"address":"123 main st",
>>"attributes":{ any JSON can go here }
>> }
>>
>> While I realize that we could stringify the JSON, this breaks our
>> published API. Is there any way I can use protobuf to perform serdes
>> between message like this and some struct like:
>>
>> {
>>string name;
>>string address;
>>json attributes;
>> }
>>
>> I'm even OK if the internal data is stringified JSON:
>>
>> {
>>string name;
>>string address;
>>string attributes;
>> }
>>
>> So long as the exchanged JSON isn't stringified. In other words, this is
>> bad:
>> {
>>"name":"john",
>>"address":"123 main st",
>>"attributes":"{ \"attr1\":\"value1\", \"attr2\":[\"elem1\", \"elem2\"]
>> }"
>> }
>>
>> It needs to be exchanged like
>> {
>>"name":"john",
>>"address":"123 main st",
>>"attributes":{ "attr1":"value1", "attr2":["elem1", "elem2"] }"
>> }
>>
>> Is this possible?
>>
>> Thanks
>> john
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Protocol Buffers" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to protobuf+unsubscr...@googlegroups.com.
>> To post to this group, send email to protobuf@googlegroups.com.
>> Visit this group at https://groups.google.com/group/protobuf.
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>
--
You received this message because you are subscribed to the Google Groups
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to protobuf+unsubscr...@googlegroups.com.
To post to this group, send email to protobuf@googlegroups.com.
Visit this group at https://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.
[protobuf] Embedding arbitrary JSON in a message
Disclaimer: I am totally new to protobuf, engaged in an exploratory POC.
Please forgive dumb questions :-)
We are looking at migrating an existing, JSON-based protocol in which
hand-coded C++ is written to perform serdes between objects and JSON. We
want to replace the hand-coding with an automated approach that can be
shared between C++ and Java. However, a stumbling block I see is that some
messages have an arbitrary field full of JSON like:
{
"name":"john",
"address":"123 main st",
"attributes":{ any JSON can go here }
}
While I realize that we could stringify the JSON, this breaks our published
API. Is there any way I can use protobuf to perform serdes between message
like this and some struct like:
{
string name;
string address;
json attributes;
}
I'm even OK if the internal data is stringified JSON:
{
string name;
string address;
string attributes;
}
So long as the exchanged JSON isn't stringified. In other words, this is
bad:
{
"name":"john",
"address":"123 main st",
"attributes":"{ \"attr1\":\"value1\", \"attr2\":[\"elem1\", \"elem2\"] }"
}
It needs to be exchanged like
{
"name":"john",
"address":"123 main st",
"attributes":{ "attr1":"value1", "attr2":["elem1", "elem2"] }"
}
Is this possible?
Thanks
john
--
You received this message because you are subscribed to the Google Groups
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to protobuf+unsubscr...@googlegroups.com.
To post to this group, send email to protobuf@googlegroups.com.
Visit this group at https://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.
