>>Gotcha. If you run protoc on the file produced by your generator class, does it succeed?
protoc.exe succeeds parsing my proto file. I have emailed the file to you and you can verify that. >>>>descriptor_pool is of course NULL. >>I meant that you should check the return value of BuildFile(). If it returns NULL, then it could not build the file. The ErrorCollector >>would tell you why I meant the same, Build_File() returns Null. However, I will get back to this issue. It will take some time to verify issues in my implementation. Thanks & Regards, Himanshu On Thu, Sep 23, 2010 at 10:37 PM, Jason Hsueh <[email protected]> wrote: > > > On Thu, Sep 23, 2010 at 9:52 AM, Himanshu Srivastava > <[email protected]>wrote: > >> Hi Jason, >> >> In code: >> //********************************************// >> DescriptorPool descriptor_pool; >> >> for (int i = 0; i < f.file_size(); ++i) { >> >> descriptor_pool.BuildFile(f.file(i)); >> //********************************************// >> descriptor_pool is of course NULL. >> > > I meant that you should check the return value of BuildFile(). If it > returns NULL, then it could not build the file. The ErrorCollector would > tell you why. > > >> And for the Importer approach my code does SEG-FAULT. >> >> Further I do need conversion from proto file directly and also a static >> FileDescriptor so that it can be passed around for construction of my new >> messages in a recursive tree algorithm in another distinct class object. >> > > That's up to you: you'll need to ensure that the DescriptorPool has the > lifetime you want. > > Previously, you mentioned that the importer approach returned NULL. That > would be indicative of a problem in the .proto file: the file could not be > built. Even if you were to successfully build the file, your code would > segfault, because the FileDescriptor is no longer valid, as the Importer, > and thus the DescriptorPool containing the FileDescriptor, have been > deleted. > > >> >> I do not want the descriptor set approach, since that is only available >> when protoc exe is present. >> --------------------- >> >>I doubt that your code will work, as Importer should be looking for a >> file named "dynamicProtoMessages.proto", which probably >>does not exist. >> ------------------------ >> I create the proto file "dynamicProtoMessages.proto" parmanently on disk >> and read it from disk to Parser after it is generated from my >> CXMLToProtoFileGenerator class. >> > > Gotcha. If you run protoc on the file produced by your generator class, > does it succeed? > > >> >> Thanks & Regards, >> Himanshu >> >> >> On Thu, Sep 23, 2010 at 9:39 PM, Jason Hsueh <[email protected]> wrote: >> >>> >>> >>> On Thu, Sep 23, 2010 at 2:40 AM, Himanshu Srivastava < >>> [email protected]> wrote: >>> >>>> Kenton: >>>> >>>> >>>> I really sent you a only a part of the complete RecordHeader message, >>>> since I was focusing on "nested message not getting parsed" issue for this >>>> particular message. Indeed my complete proto file is a large one. I >>>> have attached the truncated proto file containing this complete message >>>> only >>>> for reference with this mail (plese see attachment >>>> 'dynamicProtoMessages.proto '. >>>> >>>> I had used Parser and Importer earlier and generated the FileDescriptor >>>> in a class “CMessageParser” member function as below: >>>> >>>> >>>> >>>> //***********************************************************// >>>> >>>> class PacketTracerErrorFileErrorCollector: >>>> publicgoogle::protobuf::compiler::MultiFileErrorCollector >>>> >>>> { >>>> >>>> ... >>>> >>>> }; >>>> >>>> >>>> >>>> >>>> >>>> const google::protobuf::FileDescriptor* CMessageParser >>>> ::generateMessageFileDescriptor() >>>> >>>> { >>>> >>>> std::pair<std::string, std::string> filePair; >>>> >>>> google::protobuf::compiler::DiskSourceTree source_tree; >>>> >>>> source_tree.MapPath(/*filePair.first*/"", "."/*filePair.second*/); >>>> >>>> PacketTracerErrorFileErrorCollector error_collector;//Defined above >>>> >>>> google::protobuf::compiler::Importer importer(&source_tree, >>>> &error_collector); >>>> >>>> std::string outFileNameString = “dynamicProtoMessages.proto”; >>>> const google::protobuf::FileDescriptor* file_desc = >>>> importer.Import(outFileNameString); >>>> return file_desc; >>>> >>>> } >>>> >>>> //***********************************************************// >>>> >>>> >>>> However, the problem with that approach was, as soon as I returned the >>>> pointer from the FileDescriptor to a client class object of >>>> CMessageParser, the FileDescriptor* becomes NULL or invalid once out >>>> of the class and it becomes useless. >>>> >>> //***********************************************************// >>>> >>>> CMessageParser myParser; >>>> >>>> const google::protobuf::FileDescriptor *file_desc = >>>> myParser.generateMessageFileDescriptor(); >>>> //file_desc is NULL >>>> >>>> //***********************************************************// >>>> >>>> >>>> So, I thought, I was following some wrong approach and to retain the >>>> File Descriptors for a long enough period across scopes, I may need the >>>> DescriptorPool, which might have the permanent lifetime for the period of >>>> the application. >>>> >>> >>> Importer just wraps DescriptorPool and a database of proto files. Neither >>> of these guarantee static duration for the descriptors - any descriptors >>> that you build will only remain alive while the DescriptorPool/Importer >>> remains alive. >>> >>> Of course, that means your code should seg-fault, not return NULL, >>> assuming that you successfully built the file. You should look at the error >>> collector to see what went wrong. I doubt that your code will work, as >>> Importer should be looking for a file named "dynamicProtoMessages.proto", >>> which probably does not exist. >>> >>> But it sounds like you don't actually need to read .proto files, and >>> instead you are generating the *DescriptorProto representations directly. If >>> that's the case, you don't need Importer, you should directly use a >>> DescriptorPool as you did in the code snippet below. >>> >>> >>>> >>>> >>>> Hi Jason: >>>> >>>> I am always able to generate the FileDescriptorSet easily, although the >>>> FileDescriptor was never showing completely populated fields when debugging >>>> with Visual studio 2008 Express edition. >>>> >>>> I have attached the generated FileDescriptorSet 'dynProtoMessage.dsc' >>>> file along with the proto file containing the above messages with this >>>> mail. >>>> You can see even this dsc file (made with simple messages as in the >>>> attached >>>> proto file) does not load through DescriptorPool::BuildFile(). >>>> >>>> >>>> >>>> This is the function I used to load DescriptorSet: >>>> >>>> >>>> >>>> void CMessageParser >>>> ::getMessagesFromDescriptorSet(constgoogle::protobuf::FileDescriptor** >>>> new_file_desc) >>>> >>>> { >>>> >>>> using namespace google::protobuf; >>>> >>>> using namespace google::protobuf::io; >>>> >>>> using namespace google::protobuf::compiler; >>>> >>>> ifstream desc_file("dynProtoMessage.dsc",ios::in); >>>> >>>> if (desc_file.bad()) { >>>> >>>> std::cout << "desc file not read...exiting" << >>>> std::endl; >>>> >>>> return; >>>> >>>> } >>>> >>>> >>>> >>>> FileDescriptorSet f; >>>> >>>> f.ParseFromIstream(&desc_file); >>>> >>>> >>>> >>>> // DescriptorPoolErrorCollector errorCol; >>>> //google::protobuf::SimpleDescriptorDatabase desc_database; >>>> //DescriptorPool descriptor_pool(&errorCol, &desc_database); >>>> >>>> DescriptorPool descriptor_pool; >>>> >>>> for (int i = 0; i < f.file_size(); ++i) { >>>> >>>> descriptor_pool.BuildFile(f.file(i)); >>>> >>> >>> Did you check that these return non-null and don't have any errors? >>> >>> >>>> } >>>> >>>> >>>> const Descriptor* descriptor = >>>> descriptor_pool.FindMessageTypeByName("RecordHeader"); >>>> // descriptor =>NULL here >>>> >>> >>> You need to use the fully-qualified name to find the message type: >>> descriptor_pool.FindMessageTypeByName("traceFileMessagesProto.RecordHeader"); >>> >>> >>>> //const FileDescriptor* new_file_desc = >>>> descriptor_pool.FindFileByName("dynMessage.dsc"); >>>> /* >>>> DynamicMessageFactory factory; >>>> >>>> Message* my_message = factory.GetPrototype(descriptor)->New(); >>>> delete my_message; >>>> */ >>>> >>>> >>>> >>>> } >>>> >>>> >>>> Finally, the point raised by Kenton and you that the proto file did >>>> not parse correctly to yield the valid FileDescriptor might be correct, >>>> since, I am generating the .proto file containing the messages on the fly >>>> by >>>> converting corresponding tags from my XML file. >>>> When earlier I used a handmade proto file for the messages, which parsed >>>> correctly to provide the descriptors, I was able to stream data into >>>> protocol buffers and use the buffers correctly. I will be inspecting this >>>> file at my end for any typing errors. >>>> >>>> Thanks for the answer and for opening this protobuf library on public >>>> domain, though. >>>> >>>> >>>> Thanks and Regards, >>>> >>>> Himanshu >>>> >>>> >>>> >>>> >>>> >>>> >>>> On Thu, Sep 23, 2010 at 7:53 AM, Jason Hsueh <[email protected]> wrote: >>>> >>>>> It occurred to me that protoc shouldn't accept your file either. Did >>>>> it really compile successfully? What do you actually get in your >>>>> FileDescriptorSet? >>>>> >>>>> On Wednesday, September 22, 2010, Jason Hsueh <[email protected]> >>>>> wrote: >>>>> > 1) I would guess the problem is that the "action" message type refers >>>>> to undefined types eaction_type and ereason. There are probably error >>>>> messages farther up indicating that action could not be built. Because >>>>> action isn't a valid type, RecordHeader can't refer to it, and thinks that >>>>> action is an undefined_symbol as well. The DescriptorBuilder will read in >>>>> all inputs, and cross-link the types once everything has been processed. >>>>> It >>>>> doesn't matter what order they are defined in. >>>>> > 2) You can manually build the FileDescriptorProto representation >>>>> yourself, or use the classes used by the compiler: see >>>>> google::protobuf::compiler::Parser or >>>>> google::protobuf::compiler::Importer. >>>>> > >>>>> > On Wed, Sep 22, 2010 at 4:42 AM, Himanshu Srivastava < >>>>> [email protected]> wrote: >>>>> > >>>>> > Hi Kenton, >>>>> > I have the following message in my .proto file: >>>>> > //************************************************/ >>>>> > package traceFileMessagesProto; >>>>> > message RecordHeader { >>>>> > message myqueue{ >>>>> > optional uint32 id = 1; >>>>> > optional uint32 Priority = 2; >>>>> > } >>>>> > >>>>> > message action{ >>>>> > optional eactiontype eactiontype_instance = 1; >>>>> > optional ereason ereason_instance = 2; >>>>> > optional myqueue myqueue_instance = 3; >>>>> > } >>>>> > optional action action_instance =1; >>>>> > } //end RecordHeader >>>>> > /***************************************************/ >>>>> > Thus, "action" is nested message in "RecordHeader" and has "queue" >>>>> submessage in it. >>>>> > I used the protoc compiler -o <desc-file> option to generate the >>>>> descriptor set file and loaded it through istream in c++ application, >>>>> followed by streaming FileDescriptorSet object with this descriptor set >>>>> file: >>>>> > >>>>> > >>>>> > //***************************************************************// >>>>> > ifstream desc_file("mydescriptor.dsc",ios::in); >>>>> > FileDescriptorSet f; >>>>> > f.ParseFromIstream(&desc_file); >>>>> > //***************************************************************// >>>>> > >>>>> > >>>>> > However, while parsing through DescriptorPool::BuildFile(f.file(0)); >>>>> > at the line corresponding to the message file line: >>>>> > => optional action action_instance =1; >>>>> > the BuildFile() function exits by giving error: >>>>> > undefined_symbol "action" >>>>> > The code returns through the following function in protoc library: >>>>> > //***************************************************************// >>>>> > void DescriptorBuilder::AddNotDefinedError( >>>>> > const string& element_name, >>>>> > const Message& descriptor, >>>>> > DescriptorPool::ErrorCollector::ErrorLocation location, >>>>> > const string& undefined_symbol) { >>>>> > if (possible_undeclared_dependency_ == NULL) { >>>>> > AddError(element_name, descriptor, location, >>>>> > "\"" + undefined_symbol + "\" is not defined."); >>>>> > //***************************************************************// >>>>> > The only possible explanation could be that the message "action" is >>>>> not parsed before its instance declaration from my .proto/descriptor_set >>>>> file. >>>>> > Probably all nested messages should be parsed on their instance >>>>> declarations in a recursive manner in the DescriptorPool::Buildfile() >>>>> function. >>>>> > >>>>> > >>>>> > Qustion1: >>>>> > How do I solve the above issue? >>>>> > Question 2: >>>>> > Can not I simply load my messages in DescriptorPool through the >>>>> .proto file instead of calling the service of protoc compiler to generate >>>>> a >>>>> descriptor file? I have to generate .proto file at runtime and I don't >>>>> want >>>>> to call another exe (protoc.exe) from my application for dependency >>>>> reasons. >>>>> > >>>>> > >>>>> > Thanks in advance for any solutions /directions. >>>>> > Regards, >>>>> > Himanshu >>>>> > >>>>> > >>>>> > >>>>> > >>>>> > -- >>>>> > You received this message because you are subscribed to the Google >>>>> Groups "Protocol Buffers" group. >>>>> > To post to this group, send email to [email protected]. >>>>> > To unsubscribe from this group, send email to >>>>> [email protected]<protobuf%[email protected]> >>>>> . >>>>> > For more options, visit this group at >>>>> http://groups.google.com/group/protobuf?hl=en. >>>>> > >>>>> > >>>>> > >>>>> >>>> >>>> >>> >> -- >> You received this message because you are subscribed to the Google >> Groups "Protocol Buffers" group. >> To post to this group, send email to [email protected]. >> To unsubscribe from this group, send email to >> [email protected]<protobuf%[email protected]> >> . >> For more options, visit this group at >> http://groups.google.com/group/protobuf?hl=en. >> > > -- You received this message because you are subscribed to the Google Groups "Protocol Buffers" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/protobuf?hl=en.
