Re: [protobuf] How to generate FileDescriptor Object which has custom options?
Hi Josh,
I tried your suggested approach, the result is still the same.
helloD.findServiceByName("HelloWorldService").findMethodByName("sayHello").getOptions()
still populates them as unknown fields instead of matching them up to its
descriptors from Annotations FileDescriptor.
Ankit
On Monday, July 23, 2018 at 11:52:56 AM UTC-7, Josh Humphries wrote:
>
> On Mon, Jul 23, 2018 at 12:56 PM Ankit Patel > wrote:
>
>> Hi Josh,
>>
>> Thanks for responding. I do not have access to the codegen classes from
>> the proto compiler. My requirement is to use the proto files and create
>> descriptors from it and rely on that. I have the proto file where the
>> extension is defined and I am using that to create a registry and then feed
>> it to the File descriptor of my proto as shown in the above code. But It is
>> still not reading that registry and updating itself. What am I doing wrong?
>>
>
> Hi, Ankit,
> I apologize that my first response was incorrect. I clearly didn't ingest
> your entire code example. After reviewing more closely, I'll admit, it
> looks like you are doing everything correctly. Hopefully someone else on
> the list can spot the issue.
>
> In the meantime, I'd probably use a debugger to step through it,
> particularly the call to internalUpdateFileDescriptor, to see what's
> going on internally. An alternative you might try is just parsing
> "Hello.pb" with your initialized extension registry, instead of parsing it
> up front without and then having to call the internal update method later.
> (Not that it should behave differently -- but it could be interesting to
> verify whether or not it behaves differently.)
>
>
>>
>> Thanks,
>> Ankit
>>
>> On Friday, July 20, 2018 at 5:56:35 PM UTC-7, Josh Humphries wrote:
>>>
>>> The issue is that when you access the descriptor that way, it does not
>>> know about the custom options.
>>>
>>> You need to create an ExtensionRegistry. You'll also need the Java code
>>> produced from the annotations.proto file: that generated code will have a
>>> way to access the extensions (to add them to your registry), and IIRC even
>>> includes a way to register all extensions in the file.
>>>
>>> Then you'll have to re-parse the descriptor proto with the
>>> ExtensionRegistry -- that way, the field will be recognized and parsed
>>> correctly instead of considered an unknown field. IIRC (I've been doing Go
>>> for a while, so my recollection of the Java protobuf runtime might be
>>> fading a little), to re-parse them you have to serialize it to bytes (in
>>> this case, the method descriptor proto) and then de-serialize again. The
>>> resulting de-serialized message will now have the options in a queryable
>>> form.
>>>
>>>
>>> *Josh Humphries*
>>> jh...@bluegosling.com
>>>
>>>
>>>
>>> On Fri, Jul 20, 2018 at 8:30 PM Ankit Patel
>>> wrote:
>>>
Hi All,
I have a simple .proto file
syntax = "proto3";
package helloworld.v1;
option java_package = "helloworld.v1";
option java_multiple_files = true;
import "google/api/annotations.proto";
service HelloWorldService {
rpc sayHello (SayHelloRequest) returns (SayHelloResponse) {
option (google.api.http) = {
post: "/v1/example/echo"
body: "*"
};
}
}
message SayHelloRequest {
string favourite_thing = 1;
}
message SayHelloResponse {
string reply = 1;
}
I am generating a proto descriptor set from it using the proto compiler.
After that I am trying to read the proto descriptor programatically in
the java code like this
FileDescriptorSet Hello = FileDescriptorSet.parseFrom(new
FileInputStream(new File("D:\\temp\\descriptor\\Hello.pb")));
FileDescriptorSet Annotations = FileDescriptorSet.parseFrom(new
FileInputStream(new File("D:\\temp\\descriptor\\annotations.pb")));
FileDescriptorSet Http = FileDescriptorSet.parseFrom(new
FileInputStream(new File("D:\\temp\\descriptor\\http.pb")));
FileDescriptorSet Descriptor = FileDescriptorSet.parseFrom(new
FileInputStream(new File("D:\\temp\\descriptor\\descriptor.pb")));
FileDescriptor httpD = FileDescriptor.buildFrom(Http.getFile(0), new
FileDescriptor[] {}, true);
FileDescriptor descriptorD =
FileDescriptor.buildFrom(Descriptor.getFile(0), new FileDescriptor[] {},
true);
FileDescriptor annotationsD =
FileDescriptor.buildFrom(Annotations.getFile(0), new FileDescriptor[]
{httpD, descriptorD});
ExtensionRegistry registry = ExtensionRegistry.newInstance();
FieldDescriptor httpFD = annotationsD.findExtensionByName("http");
Message httpRuleM =
DynamicMessage.newBuilder(httpD.findMessageTypeByName("HttpRule")).build();
registry.add(httpFD, httpRuleM);
FileDescriptor helloD = FileDescriptor.buildFrom(Hello.getFile(0), new
FileDescriptor[]
Re: [protobuf] How to generate FileDescriptor Object which has custom options?
On Mon, Jul 23, 2018 at 12:56 PM Ankit Patel
wrote:
> Hi Josh,
>
> Thanks for responding. I do not have access to the codegen classes from
> the proto compiler. My requirement is to use the proto files and create
> descriptors from it and rely on that. I have the proto file where the
> extension is defined and I am using that to create a registry and then feed
> it to the File descriptor of my proto as shown in the above code. But It is
> still not reading that registry and updating itself. What am I doing wrong?
>
Hi, Ankit,
I apologize that my first response was incorrect. I clearly didn't ingest
your entire code example. After reviewing more closely, I'll admit, it
looks like you are doing everything correctly. Hopefully someone else on
the list can spot the issue.
In the meantime, I'd probably use a debugger to step through it,
particularly the call to internalUpdateFileDescriptor, to see what's going
on internally. An alternative you might try is just parsing "Hello.pb" with
your initialized extension registry, instead of parsing it up front without
and then having to call the internal update method later. (Not that it
should behave differently -- but it could be interesting to verify whether
or not it behaves differently.)
>
> Thanks,
> Ankit
>
> On Friday, July 20, 2018 at 5:56:35 PM UTC-7, Josh Humphries wrote:
>>
>> The issue is that when you access the descriptor that way, it does not
>> know about the custom options.
>>
>> You need to create an ExtensionRegistry. You'll also need the Java code
>> produced from the annotations.proto file: that generated code will have a
>> way to access the extensions (to add them to your registry), and IIRC even
>> includes a way to register all extensions in the file.
>>
>> Then you'll have to re-parse the descriptor proto with the
>> ExtensionRegistry -- that way, the field will be recognized and parsed
>> correctly instead of considered an unknown field. IIRC (I've been doing Go
>> for a while, so my recollection of the Java protobuf runtime might be
>> fading a little), to re-parse them you have to serialize it to bytes (in
>> this case, the method descriptor proto) and then de-serialize again. The
>> resulting de-serialized message will now have the options in a queryable
>> form.
>>
>>
>> *Josh Humphries*
>> jh...@bluegosling.com
>>
>>
>>
>> On Fri, Jul 20, 2018 at 8:30 PM Ankit Patel wrote:
>>
>>> Hi All,
>>>
>>> I have a simple .proto file
>>>
>>> syntax = "proto3";
>>>
>>> package helloworld.v1;
>>>
>>> option java_package = "helloworld.v1";
>>> option java_multiple_files = true;
>>>
>>> import "google/api/annotations.proto";
>>>
>>> service HelloWorldService {
>>> rpc sayHello (SayHelloRequest) returns (SayHelloResponse) {
>>> option (google.api.http) = {
>>> post: "/v1/example/echo"
>>> body: "*"
>>> };
>>> }
>>> }
>>>
>>> message SayHelloRequest {
>>> string favourite_thing = 1;
>>> }
>>>
>>> message SayHelloResponse {
>>> string reply = 1;
>>> }
>>>
>>> I am generating a proto descriptor set from it using the proto compiler.
>>>
>>> After that I am trying to read the proto descriptor programatically in
>>> the java code like this
>>> FileDescriptorSet Hello = FileDescriptorSet.parseFrom(new
>>> FileInputStream(new File("D:\\temp\\descriptor\\Hello.pb")));
>>> FileDescriptorSet Annotations = FileDescriptorSet.parseFrom(new
>>> FileInputStream(new File("D:\\temp\\descriptor\\annotations.pb")));
>>> FileDescriptorSet Http = FileDescriptorSet.parseFrom(new
>>> FileInputStream(new File("D:\\temp\\descriptor\\http.pb")));
>>> FileDescriptorSet Descriptor = FileDescriptorSet.parseFrom(new
>>> FileInputStream(new File("D:\\temp\\descriptor\\descriptor.pb")));
>>>
>>> FileDescriptor httpD = FileDescriptor.buildFrom(Http.getFile(0), new
>>> FileDescriptor[] {}, true);
>>> FileDescriptor descriptorD =
>>> FileDescriptor.buildFrom(Descriptor.getFile(0), new FileDescriptor[] {},
>>> true);
>>> FileDescriptor annotationsD =
>>> FileDescriptor.buildFrom(Annotations.getFile(0), new FileDescriptor[]
>>> {httpD, descriptorD});
>>>
>>> ExtensionRegistry registry = ExtensionRegistry.newInstance();
>>> FieldDescriptor httpFD = annotationsD.findExtensionByName("http");
>>> Message httpRuleM =
>>> DynamicMessage.newBuilder(httpD.findMessageTypeByName("HttpRule")).build();
>>> registry.add(httpFD, httpRuleM);
>>>
>>> FileDescriptor helloD = FileDescriptor.buildFrom(Hello.getFile(0), new
>>> FileDescriptor[] {annotationsD});
>>> FileDescriptor.internalUpdateFileDescriptor(helloD, registry);
>>>
>>> helloD.findServiceByName("HelloWorldService").findMethodByName("sayHello").getOptions();
>>>
>>> The api call in the last line puts the "http" in the unknown field, and
>>> the reason is because its a custom option and not registered. But as you
>>> can see I am doing that.
>>>
>>> Can someone please guide me on what am I doing wrong?
>>>
>>> Thanks!
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "Protocol
Re: [protobuf] How to generate FileDescriptor Object which has custom options?
Hi Josh,
Thanks for responding. I do not have access to the codegen classes from the
proto compiler. My requirement is to use the proto files and create
descriptors from it and rely on that. I have the proto file where the
extension is defined and I am using that to create a registry and then feed
it to the File descriptor of my proto as shown in the above code. But It is
still not reading that registry and updating itself. What am I doing wrong?
Thanks,
Ankit
On Friday, July 20, 2018 at 5:56:35 PM UTC-7, Josh Humphries wrote:
>
> The issue is that when you access the descriptor that way, it does not
> know about the custom options.
>
> You need to create an ExtensionRegistry. You'll also need the Java code
> produced from the annotations.proto file: that generated code will have a
> way to access the extensions (to add them to your registry), and IIRC even
> includes a way to register all extensions in the file.
>
> Then you'll have to re-parse the descriptor proto with the
> ExtensionRegistry -- that way, the field will be recognized and parsed
> correctly instead of considered an unknown field. IIRC (I've been doing Go
> for a while, so my recollection of the Java protobuf runtime might be
> fading a little), to re-parse them you have to serialize it to bytes (in
> this case, the method descriptor proto) and then de-serialize again. The
> resulting de-serialized message will now have the options in a queryable
> form.
>
>
> *Josh Humphries*
> jh...@bluegosling.com
>
>
>
> On Fri, Jul 20, 2018 at 8:30 PM Ankit Patel > wrote:
>
>> Hi All,
>>
>> I have a simple .proto file
>>
>> syntax = "proto3";
>>
>> package helloworld.v1;
>>
>> option java_package = "helloworld.v1";
>> option java_multiple_files = true;
>>
>> import "google/api/annotations.proto";
>>
>> service HelloWorldService {
>> rpc sayHello (SayHelloRequest) returns (SayHelloResponse) {
>> option (google.api.http) = {
>> post: "/v1/example/echo"
>> body: "*"
>> };
>> }
>> }
>>
>> message SayHelloRequest {
>> string favourite_thing = 1;
>> }
>>
>> message SayHelloResponse {
>> string reply = 1;
>> }
>>
>> I am generating a proto descriptor set from it using the proto compiler.
>>
>> After that I am trying to read the proto descriptor programatically in
>> the java code like this
>> FileDescriptorSet Hello = FileDescriptorSet.parseFrom(new
>> FileInputStream(new File("D:\\temp\\descriptor\\Hello.pb")));
>> FileDescriptorSet Annotations = FileDescriptorSet.parseFrom(new
>> FileInputStream(new File("D:\\temp\\descriptor\\annotations.pb")));
>> FileDescriptorSet Http = FileDescriptorSet.parseFrom(new
>> FileInputStream(new File("D:\\temp\\descriptor\\http.pb")));
>> FileDescriptorSet Descriptor = FileDescriptorSet.parseFrom(new
>> FileInputStream(new File("D:\\temp\\descriptor\\descriptor.pb")));
>>
>> FileDescriptor httpD = FileDescriptor.buildFrom(Http.getFile(0), new
>> FileDescriptor[] {}, true);
>> FileDescriptor descriptorD =
>> FileDescriptor.buildFrom(Descriptor.getFile(0), new FileDescriptor[] {},
>> true);
>> FileDescriptor annotationsD =
>> FileDescriptor.buildFrom(Annotations.getFile(0), new FileDescriptor[]
>> {httpD, descriptorD});
>>
>> ExtensionRegistry registry = ExtensionRegistry.newInstance();
>> FieldDescriptor httpFD = annotationsD.findExtensionByName("http");
>> Message httpRuleM =
>> DynamicMessage.newBuilder(httpD.findMessageTypeByName("HttpRule")).build();
>> registry.add(httpFD, httpRuleM);
>>
>> FileDescriptor helloD = FileDescriptor.buildFrom(Hello.getFile(0), new
>> FileDescriptor[] {annotationsD});
>> FileDescriptor.internalUpdateFileDescriptor(helloD, registry);
>>
>> helloD.findServiceByName("HelloWorldService").findMethodByName("sayHello").getOptions();
>>
>> The api call in the last line puts the "http" in the unknown field, and
>> the reason is because its a custom option and not registered. But as you
>> can see I am doing that.
>>
>> Can someone please guide me on what am I doing wrong?
>>
>> Thanks!
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Protocol Buffers" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to protobuf+u...@googlegroups.com .
>> To post to this group, send email to prot...@googlegroups.com
>> .
>> Visit this group at https://groups.google.com/group/protobuf.
>> For more options, visit https://groups.google.com/d/optout.
>>
>
--
You received this message because you are subscribed to the Google Groups
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to protobuf+unsubscr...@googlegroups.com.
To post to this group, send email to protobuf@googlegroups.com.
Visit this group at https://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.
Re: [protobuf] How to generate FileDescriptor Object which has custom options?
The issue is that when you access the descriptor that way, it does not know
about the custom options.
You need to create an ExtensionRegistry. You'll also need the Java code
produced from the annotations.proto file: that generated code will have a
way to access the extensions (to add them to your registry), and IIRC even
includes a way to register all extensions in the file.
Then you'll have to re-parse the descriptor proto with the
ExtensionRegistry -- that way, the field will be recognized and parsed
correctly instead of considered an unknown field. IIRC (I've been doing Go
for a while, so my recollection of the Java protobuf runtime might be
fading a little), to re-parse them you have to serialize it to bytes (in
this case, the method descriptor proto) and then de-serialize again. The
resulting de-serialized message will now have the options in a queryable
form.
*Josh Humphries*
jh...@bluegosling.com
On Fri, Jul 20, 2018 at 8:30 PM Ankit Patel
wrote:
> Hi All,
>
> I have a simple .proto file
>
> syntax = "proto3";
>
> package helloworld.v1;
>
> option java_package = "helloworld.v1";
> option java_multiple_files = true;
>
> import "google/api/annotations.proto";
>
> service HelloWorldService {
> rpc sayHello (SayHelloRequest) returns (SayHelloResponse) {
> option (google.api.http) = {
> post: "/v1/example/echo"
> body: "*"
> };
> }
> }
>
> message SayHelloRequest {
> string favourite_thing = 1;
> }
>
> message SayHelloResponse {
> string reply = 1;
> }
>
> I am generating a proto descriptor set from it using the proto compiler.
>
> After that I am trying to read the proto descriptor programatically in the
> java code like this
> FileDescriptorSet Hello = FileDescriptorSet.parseFrom(new
> FileInputStream(new File("D:\\temp\\descriptor\\Hello.pb")));
> FileDescriptorSet Annotations = FileDescriptorSet.parseFrom(new
> FileInputStream(new File("D:\\temp\\descriptor\\annotations.pb")));
> FileDescriptorSet Http = FileDescriptorSet.parseFrom(new
> FileInputStream(new File("D:\\temp\\descriptor\\http.pb")));
> FileDescriptorSet Descriptor = FileDescriptorSet.parseFrom(new
> FileInputStream(new File("D:\\temp\\descriptor\\descriptor.pb")));
>
> FileDescriptor httpD = FileDescriptor.buildFrom(Http.getFile(0), new
> FileDescriptor[] {}, true);
> FileDescriptor descriptorD =
> FileDescriptor.buildFrom(Descriptor.getFile(0), new FileDescriptor[] {},
> true);
> FileDescriptor annotationsD =
> FileDescriptor.buildFrom(Annotations.getFile(0), new FileDescriptor[]
> {httpD, descriptorD});
>
> ExtensionRegistry registry = ExtensionRegistry.newInstance();
> FieldDescriptor httpFD = annotationsD.findExtensionByName("http");
> Message httpRuleM =
> DynamicMessage.newBuilder(httpD.findMessageTypeByName("HttpRule")).build();
> registry.add(httpFD, httpRuleM);
>
> FileDescriptor helloD = FileDescriptor.buildFrom(Hello.getFile(0), new
> FileDescriptor[] {annotationsD});
> FileDescriptor.internalUpdateFileDescriptor(helloD, registry);
>
> helloD.findServiceByName("HelloWorldService").findMethodByName("sayHello").getOptions();
>
> The api call in the last line puts the "http" in the unknown field, and
> the reason is because its a custom option and not registered. But as you
> can see I am doing that.
>
> Can someone please guide me on what am I doing wrong?
>
> Thanks!
>
> --
> You received this message because you are subscribed to the Google Groups
> "Protocol Buffers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to protobuf+unsubscr...@googlegroups.com.
> To post to this group, send email to protobuf@googlegroups.com.
> Visit this group at https://groups.google.com/group/protobuf.
> For more options, visit https://groups.google.com/d/optout.
>
--
You received this message because you are subscribed to the Google Groups
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to protobuf+unsubscr...@googlegroups.com.
To post to this group, send email to protobuf@googlegroups.com.
Visit this group at https://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.
[protobuf] How to generate FileDescriptor Object which has custom options?
Hi All,
I have a simple .proto file
syntax = "proto3";
package helloworld.v1;
option java_package = "helloworld.v1";
option java_multiple_files = true;
import "google/api/annotations.proto";
service HelloWorldService {
rpc sayHello (SayHelloRequest) returns (SayHelloResponse) {
option (google.api.http) = {
post: "/v1/example/echo"
body: "*"
};
}
}
message SayHelloRequest {
string favourite_thing = 1;
}
message SayHelloResponse {
string reply = 1;
}
I am generating a proto descriptor set from it using the proto compiler.
After that I am trying to read the proto descriptor programatically in the
java code like this
FileDescriptorSet Hello = FileDescriptorSet.parseFrom(new
FileInputStream(new File("D:\\temp\\descriptor\\Hello.pb")));
FileDescriptorSet Annotations = FileDescriptorSet.parseFrom(new
FileInputStream(new File("D:\\temp\\descriptor\\annotations.pb")));
FileDescriptorSet Http = FileDescriptorSet.parseFrom(new
FileInputStream(new File("D:\\temp\\descriptor\\http.pb")));
FileDescriptorSet Descriptor = FileDescriptorSet.parseFrom(new
FileInputStream(new File("D:\\temp\\descriptor\\descriptor.pb")));
FileDescriptor httpD = FileDescriptor.buildFrom(Http.getFile(0), new
FileDescriptor[] {}, true);
FileDescriptor descriptorD =
FileDescriptor.buildFrom(Descriptor.getFile(0), new FileDescriptor[] {},
true);
FileDescriptor annotationsD =
FileDescriptor.buildFrom(Annotations.getFile(0), new FileDescriptor[]
{httpD, descriptorD});
ExtensionRegistry registry = ExtensionRegistry.newInstance();
FieldDescriptor httpFD = annotationsD.findExtensionByName("http");
Message httpRuleM =
DynamicMessage.newBuilder(httpD.findMessageTypeByName("HttpRule")).build();
registry.add(httpFD, httpRuleM);
FileDescriptor helloD = FileDescriptor.buildFrom(Hello.getFile(0), new
FileDescriptor[] {annotationsD});
FileDescriptor.internalUpdateFileDescriptor(helloD, registry);
helloD.findServiceByName("HelloWorldService").findMethodByName("sayHello").getOptions();
The api call in the last line puts the "http" in the unknown field, and the
reason is because its a custom option and not registered. But as you can
see I am doing that.
Can someone please guide me on what am I doing wrong?
Thanks!
--
You received this message because you are subscribed to the Google Groups
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to protobuf+unsubscr...@googlegroups.com.
To post to this group, send email to protobuf@googlegroups.com.
Visit this group at https://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.
