On Thu, Sep 23, 2010 at 9:52 AM, Himanshu Srivastava <him.s...@gmail.com>wrote:
> Hi Jason,
>
> In code:
> //********************************************//
> DescriptorPool descriptor_pool;
>
> for (int i = 0; i < f.file_size(); ++i) {
>
> descriptor_pool.BuildFile(f.file(i));
> //********************************************//
> descriptor_pool is of course NULL.
>
I meant that you should check the return value of BuildFile(). If it returns
NULL, then it could not build the file. The ErrorCollector would tell you
why.
> And for the Importer approach my code does SEG-FAULT.
>
> Further I do need conversion from proto file directly and also a static
> FileDescriptor so that it can be passed around for construction of my new
> messages in a recursive tree algorithm in another distinct class object.
>
That's up to you: you'll need to ensure that the DescriptorPool has the
lifetime you want.
Previously, you mentioned that the importer approach returned NULL. That
would be indicative of a problem in the .proto file: the file could not be
built. Even if you were to successfully build the file, your code would
segfault, because the FileDescriptor is no longer valid, as the Importer,
and thus the DescriptorPool containing the FileDescriptor, have been
deleted.
>
> I do not want the descriptor set approach, since that is only available
> when protoc exe is present.
> ---------------------
> >>I doubt that your code will work, as Importer should be looking for a
> file named "dynamicProtoMessages.proto", which probably >>does not exist.
> ------------------------
> I create the proto file "dynamicProtoMessages.proto" parmanently on disk
> and read it from disk to Parser after it is generated from my
> CXMLToProtoFileGenerator class.
>
Gotcha. If you run protoc on the file produced by your generator class, does
it succeed?
>
> Thanks & Regards,
> Himanshu
>
>
> On Thu, Sep 23, 2010 at 9:39 PM, Jason Hsueh <jas...@google.com> wrote:
>
>>
>>
>> On Thu, Sep 23, 2010 at 2:40 AM, Himanshu Srivastava <him.s...@gmail.com
>> > wrote:
>>
>>> Kenton:
>>>
>>>
>>> I really sent you a only a part of the complete RecordHeader message,
>>> since I was focusing on "nested message not getting parsed" issue for this
>>> particular message. Indeed my complete proto file is a large one. I have
>>> attached the truncated proto file containing this complete message only
>>> for reference with this mail (plese see attachment
>>> 'dynamicProtoMessages.proto '.
>>>
>>> I had used Parser and Importer earlier and generated the FileDescriptor
>>> in a class “CMessageParser” member function as below:
>>>
>>>
>>>
>>> //***********************************************************//
>>>
>>> class PacketTracerErrorFileErrorCollector:
>>> publicgoogle::protobuf::compiler::MultiFileErrorCollector
>>>
>>> {
>>>
>>> ...
>>>
>>> };
>>>
>>>
>>>
>>>
>>>
>>> const google::protobuf::FileDescriptor* CMessageParser
>>> ::generateMessageFileDescriptor()
>>>
>>> {
>>>
>>> std::pair<std::string, std::string> filePair;
>>>
>>> google::protobuf::compiler::DiskSourceTree source_tree;
>>>
>>> source_tree.MapPath(/*filePair.first*/"", "."/*filePair.second*/);
>>>
>>> PacketTracerErrorFileErrorCollector error_collector;//Defined above
>>>
>>> google::protobuf::compiler::Importer importer(&source_tree,
>>> &error_collector);
>>>
>>> std::string outFileNameString = “dynamicProtoMessages.proto”;
>>> const google::protobuf::FileDescriptor* file_desc =
>>> importer.Import(outFileNameString);
>>> return file_desc;
>>>
>>> }
>>>
>>> //***********************************************************//
>>>
>>>
>>> However, the problem with that approach was, as soon as I returned the
>>> pointer from the FileDescriptor to a client class object of
>>> CMessageParser, the FileDescriptor* becomes NULL or invalid once out
>>> of the class and it becomes useless.
>>>
>> //***********************************************************//
>>>
>>> CMessageParser myParser;
>>>
>>> const google::protobuf::FileDescriptor *file_desc =
>>> myParser.generateMessageFileDescriptor();
>>> //file_desc is NULL
>>>
>>> //***********************************************************//
>>>
>>>
>>> So, I thought, I was following some wrong approach and to retain the
>>> File Descriptors for a long enough period across scopes, I may need the
>>> DescriptorPool, which might have the permanent lifetime for the period of
>>> the application.
>>>
>>
>> Importer just wraps DescriptorPool and a database of proto files. Neither
>> of these guarantee static duration for the descriptors - any descriptors
>> that you build will only remain alive while the DescriptorPool/Importer
>> remains alive.
>>
>> Of course, that means your code should seg-fault, not return NULL,
>> assuming that you successfully built the file. You should look at the error
>> collector to see what went wrong. I doubt that your code will work, as
>> Importer should be looking for a file named "dynamicProtoMessages.proto",
>> which probably does not exist.
>>
>> But it sounds like you don't actually need to read .proto files, and
>> instead you are generating the *DescriptorProto representations directly. If
>> that's the case, you don't need Importer, you should directly use a
>> DescriptorPool as you did in the code snippet below.
>>
>>
>>>
>>>
>>> Hi Jason:
>>>
>>> I am always able to generate the FileDescriptorSet easily, although the
>>> FileDescriptor was never showing completely populated fields when debugging
>>> with Visual studio 2008 Express edition.
>>>
>>> I have attached the generated FileDescriptorSet 'dynProtoMessage.dsc'
>>> file along with the proto file containing the above messages with this mail.
>>> You can see even this dsc file (made with simple messages as in the attached
>>> proto file) does not load through DescriptorPool::BuildFile().
>>>
>>>
>>>
>>> This is the function I used to load DescriptorSet:
>>>
>>>
>>>
>>> void CMessageParser
>>> ::getMessagesFromDescriptorSet(constgoogle::protobuf::FileDescriptor**
>>> new_file_desc)
>>>
>>> {
>>>
>>> using namespace google::protobuf;
>>>
>>> using namespace google::protobuf::io;
>>>
>>> using namespace google::protobuf::compiler;
>>>
>>> ifstream desc_file("dynProtoMessage.dsc",ios::in);
>>>
>>> if (desc_file.bad()) {
>>>
>>> std::cout << "desc file not read...exiting" <<
>>> std::endl;
>>>
>>> return;
>>>
>>> }
>>>
>>>
>>>
>>> FileDescriptorSet f;
>>>
>>> f.ParseFromIstream(&desc_file);
>>>
>>>
>>>
>>> // DescriptorPoolErrorCollector errorCol;
>>> //google::protobuf::SimpleDescriptorDatabase desc_database;
>>> //DescriptorPool descriptor_pool(&errorCol, &desc_database);
>>>
>>> DescriptorPool descriptor_pool;
>>>
>>> for (int i = 0; i < f.file_size(); ++i) {
>>>
>>> descriptor_pool.BuildFile(f.file(i));
>>>
>>
>> Did you check that these return non-null and don't have any errors?
>>
>>
>>> }
>>>
>>>
>>> const Descriptor* descriptor =
>>> descriptor_pool.FindMessageTypeByName("RecordHeader");
>>> // descriptor =>NULL here
>>>
>>
>> You need to use the fully-qualified name to find the message type:
>> descriptor_pool.FindMessageTypeByName("traceFileMessagesProto.RecordHeader");
>>
>>
>>> //const FileDescriptor* new_file_desc =
>>> descriptor_pool.FindFileByName("dynMessage.dsc");
>>> /*
>>> DynamicMessageFactory factory;
>>>
>>> Message* my_message = factory.GetPrototype(descriptor)->New();
>>> delete my_message;
>>> */
>>>
>>>
>>>
>>> }
>>>
>>>
>>> Finally, the point raised by Kenton and you that the proto file did not
>>> parse correctly to yield the valid FileDescriptor might be correct, since, I
>>> am generating the .proto file containing the messages on the fly by
>>> converting corresponding tags from my XML file.
>>> When earlier I used a handmade proto file for the messages, which parsed
>>> correctly to provide the descriptors, I was able to stream data into
>>> protocol buffers and use the buffers correctly. I will be inspecting this
>>> file at my end for any typing errors.
>>>
>>> Thanks for the answer and for opening this protobuf library on public
>>> domain, though.
>>>
>>>
>>> Thanks and Regards,
>>>
>>> Himanshu
>>>
>>>
>>>
>>>
>>>
>>>
>>> On Thu, Sep 23, 2010 at 7:53 AM, Jason Hsueh <jas...@google.com> wrote:
>>>
>>>> It occurred to me that protoc shouldn't accept your file either. Did
>>>> it really compile successfully? What do you actually get in your
>>>> FileDescriptorSet?
>>>>
>>>> On Wednesday, September 22, 2010, Jason Hsueh <jas...@google.com>
>>>> wrote:
>>>> > 1) I would guess the problem is that the "action" message type refers
>>>> to undefined types eaction_type and ereason. There are probably error
>>>> messages farther up indicating that action could not be built. Because
>>>> action isn't a valid type, RecordHeader can't refer to it, and thinks that
>>>> action is an undefined_symbol as well. The DescriptorBuilder will read in
>>>> all inputs, and cross-link the types once everything has been processed. It
>>>> doesn't matter what order they are defined in.
>>>> > 2) You can manually build the FileDescriptorProto representation
>>>> yourself, or use the classes used by the compiler: see
>>>> google::protobuf::compiler::Parser or google::protobuf::compiler::Importer.
>>>> >
>>>> > On Wed, Sep 22, 2010 at 4:42 AM, Himanshu Srivastava <
>>>> him.s...@gmail.com> wrote:
>>>> >
>>>> > Hi Kenton,
>>>> > I have the following message in my .proto file:
>>>> > //************************************************/
>>>> > package traceFileMessagesProto;
>>>> > message RecordHeader {
>>>> > message myqueue{
>>>> > optional uint32 id = 1;
>>>> > optional uint32 Priority = 2;
>>>> > }
>>>> >
>>>> > message action{
>>>> > optional eactiontype eactiontype_instance = 1;
>>>> > optional ereason ereason_instance = 2;
>>>> > optional myqueue myqueue_instance = 3;
>>>> > }
>>>> > optional action action_instance =1;
>>>> > } //end RecordHeader
>>>> > /***************************************************/
>>>> > Thus, "action" is nested message in "RecordHeader" and has "queue"
>>>> submessage in it.
>>>> > I used the protoc compiler -o <desc-file> option to generate the
>>>> descriptor set file and loaded it through istream in c++ application,
>>>> followed by streaming FileDescriptorSet object with this descriptor set
>>>> file:
>>>> >
>>>> >
>>>> > //***************************************************************//
>>>> > ifstream desc_file("mydescriptor.dsc",ios::in);
>>>> > FileDescriptorSet f;
>>>> > f.ParseFromIstream(&desc_file);
>>>> > //***************************************************************//
>>>> >
>>>> >
>>>> > However, while parsing through DescriptorPool::BuildFile(f.file(0));
>>>> > at the line corresponding to the message file line:
>>>> > => optional action action_instance =1;
>>>> > the BuildFile() function exits by giving error:
>>>> > undefined_symbol "action"
>>>> > The code returns through the following function in protoc library:
>>>> > //***************************************************************//
>>>> > void DescriptorBuilder::AddNotDefinedError(
>>>> > const string& element_name,
>>>> > const Message& descriptor,
>>>> > DescriptorPool::ErrorCollector::ErrorLocation location,
>>>> > const string& undefined_symbol) {
>>>> > if (possible_undeclared_dependency_ == NULL) {
>>>> > AddError(element_name, descriptor, location,
>>>> > "\"" + undefined_symbol + "\" is not defined.");
>>>> > //***************************************************************//
>>>> > The only possible explanation could be that the message "action" is
>>>> not parsed before its instance declaration from my .proto/descriptor_set
>>>> file.
>>>> > Probably all nested messages should be parsed on their instance
>>>> declarations in a recursive manner in the DescriptorPool::Buildfile()
>>>> function.
>>>> >
>>>> >
>>>> > Qustion1:
>>>> > How do I solve the above issue?
>>>> > Question 2:
>>>> > Can not I simply load my messages in DescriptorPool through the .proto
>>>> file instead of calling the service of protoc compiler to generate a
>>>> descriptor file? I have to generate .proto file at runtime and I don't want
>>>> to call another exe (protoc.exe) from my application for dependency
>>>> reasons.
>>>> >
>>>> >
>>>> > Thanks in advance for any solutions /directions.
>>>> > Regards,
>>>> > Himanshu
>>>> >
>>>> >
>>>> >
>>>> >
>>>> > --
>>>> > You received this message because you are subscribed to the Google
>>>> Groups "Protocol Buffers" group.
>>>> > To post to this group, send email to proto...@googlegroups.com.
>>>> > To unsubscribe from this group, send email to
>>>> protobuf+unsubscr...@googlegroups.com<protobuf%2bunsubscr...@googlegroups.com>
>>>> .
>>>> > For more options, visit this group at
>>>> http://groups.google.com/group/protobuf?hl=en.
>>>> >
>>>> >
>>>> >
>>>>
>>>
>>>
>>
> --
> You received this message because you are subscribed to the Google Groups
> "Protocol Buffers" group.
> To post to this group, send email to proto...@googlegroups.com.
> To unsubscribe from this group, send email to
> protobuf+unsubscr...@googlegroups.com<protobuf%2bunsubscr...@googlegroups.com>
> .
> For more options, visit this group at
> http://groups.google.com/group/protobuf?hl=en.
>
--
You received this message because you are subscribed to the Google Groups
"Protocol Buffers" group.
To post to this group, send email to proto...@googlegroups.com.
To unsubscribe from this group, send email to
protobuf+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/protobuf?hl=en.
