Hi Jason, The modules all seem to work now as soon as I created the Importer on the heap and the FileDescriptor is also available across scopes. I need to follow the algorithm to see all messages are getting instanciated.
Thanks for your quick help. Thanks & Regards, Himanshu On Thu, Sep 23, 2010 at 10:22 PM, Himanshu Srivastava <[email protected]>wrote: > Hi Jason, > > In code: > //********************************************// > DescriptorPool descriptor_pool; > > for (int i = 0; i < f.file_size(); ++i) { > > descriptor_pool.BuildFile(f.file(i)); > //********************************************// > descriptor_pool is of course NULL. > > And for the Importer approach my code does SEG-FAULT. > > Further I do need conversion from proto file directly and also a static > FileDescriptor so that it can be passed around for construction of my new > messages in a recursive tree algorithm in another distinct class object. > > I do not want the descriptor set approach, since that is only available > when protoc exe is present. > --------------------- > >>I doubt that your code will work, as Importer should be looking for a > file named "dynamicProtoMessages.proto", which probably >>does not exist. > ------------------------ > I create the proto file "dynamicProtoMessages.proto" parmanently on disk > and read it from disk to Parser after it is generated from my > CXMLToProtoFileGenerator class. > > > Thanks & Regards, > Himanshu > > > On Thu, Sep 23, 2010 at 9:39 PM, Jason Hsueh <[email protected]> wrote: > >> >> >> On Thu, Sep 23, 2010 at 2:40 AM, Himanshu Srivastava <[email protected] >> > wrote: >> >>> Kenton: >>> >>> >>> I really sent you a only a part of the complete RecordHeader message, >>> since I was focusing on "nested message not getting parsed" issue for this >>> particular message. Indeed my complete proto file is a large one. I have >>> attached the truncated proto file containing this complete message only >>> for reference with this mail (plese see attachment >>> 'dynamicProtoMessages.proto '. >>> >>> I had used Parser and Importer earlier and generated the FileDescriptor >>> in a class “CMessageParser” member function as below: >>> >>> >>> >>> //***********************************************************// >>> >>> class PacketTracerErrorFileErrorCollector: >>> publicgoogle::protobuf::compiler::MultiFileErrorCollector >>> >>> { >>> >>> ... >>> >>> }; >>> >>> >>> >>> >>> >>> const google::protobuf::FileDescriptor* CMessageParser >>> ::generateMessageFileDescriptor() >>> >>> { >>> >>> std::pair<std::string, std::string> filePair; >>> >>> google::protobuf::compiler::DiskSourceTree source_tree; >>> >>> source_tree.MapPath(/*filePair.first*/"", "."/*filePair.second*/); >>> >>> PacketTracerErrorFileErrorCollector error_collector;//Defined above >>> >>> google::protobuf::compiler::Importer importer(&source_tree, >>> &error_collector); >>> >>> std::string outFileNameString = “dynamicProtoMessages.proto”; >>> const google::protobuf::FileDescriptor* file_desc = >>> importer.Import(outFileNameString); >>> return file_desc; >>> >>> } >>> >>> //***********************************************************// >>> >>> >>> However, the problem with that approach was, as soon as I returned the >>> pointer from the FileDescriptor to a client class object of >>> CMessageParser, the FileDescriptor* becomes NULL or invalid once out >>> of the class and it becomes useless. >>> >> //***********************************************************// >>> >>> CMessageParser myParser; >>> >>> const google::protobuf::FileDescriptor *file_desc = >>> myParser.generateMessageFileDescriptor(); >>> //file_desc is NULL >>> >>> //***********************************************************// >>> >>> >>> So, I thought, I was following some wrong approach and to retain the >>> File Descriptors for a long enough period across scopes, I may need the >>> DescriptorPool, which might have the permanent lifetime for the period of >>> the application. >>> >> >> Importer just wraps DescriptorPool and a database of proto files. Neither >> of these guarantee static duration for the descriptors - any descriptors >> that you build will only remain alive while the DescriptorPool/Importer >> remains alive. >> >> Of course, that means your code should seg-fault, not return NULL, >> assuming that you successfully built the file. You should look at the error >> collector to see what went wrong. I doubt that your code will work, as >> Importer should be looking for a file named "dynamicProtoMessages.proto", >> which probably does not exist. >> >> But it sounds like you don't actually need to read .proto files, and >> instead you are generating the *DescriptorProto representations directly. If >> that's the case, you don't need Importer, you should directly use a >> DescriptorPool as you did in the code snippet below. >> >> >>> >>> >>> Hi Jason: >>> >>> I am always able to generate the FileDescriptorSet easily, although the >>> FileDescriptor was never showing completely populated fields when debugging >>> with Visual studio 2008 Express edition. >>> >>> I have attached the generated FileDescriptorSet 'dynProtoMessage.dsc' >>> file along with the proto file containing the above messages with this mail. >>> You can see even this dsc file (made with simple messages as in the attached >>> proto file) does not load through DescriptorPool::BuildFile(). >>> >>> >>> >>> This is the function I used to load DescriptorSet: >>> >>> >>> >>> void CMessageParser >>> ::getMessagesFromDescriptorSet(constgoogle::protobuf::FileDescriptor** >>> new_file_desc) >>> >>> { >>> >>> using namespace google::protobuf; >>> >>> using namespace google::protobuf::io; >>> >>> using namespace google::protobuf::compiler; >>> >>> ifstream desc_file("dynProtoMessage.dsc",ios::in); >>> >>> if (desc_file.bad()) { >>> >>> std::cout << "desc file not read...exiting" << >>> std::endl; >>> >>> return; >>> >>> } >>> >>> >>> >>> FileDescriptorSet f; >>> >>> f.ParseFromIstream(&desc_file); >>> >>> >>> >>> // DescriptorPoolErrorCollector errorCol; >>> //google::protobuf::SimpleDescriptorDatabase desc_database; >>> //DescriptorPool descriptor_pool(&errorCol, &desc_database); >>> >>> DescriptorPool descriptor_pool; >>> >>> for (int i = 0; i < f.file_size(); ++i) { >>> >>> descriptor_pool.BuildFile(f.file(i)); >>> >> >> Did you check that these return non-null and don't have any errors? >> >> >>> } >>> >>> >>> const Descriptor* descriptor = >>> descriptor_pool.FindMessageTypeByName("RecordHeader"); >>> // descriptor =>NULL here >>> >> >> You need to use the fully-qualified name to find the message type: >> descriptor_pool.FindMessageTypeByName("traceFileMessagesProto.RecordHeader"); >> >> >>> //const FileDescriptor* new_file_desc = >>> descriptor_pool.FindFileByName("dynMessage.dsc"); >>> /* >>> DynamicMessageFactory factory; >>> >>> Message* my_message = factory.GetPrototype(descriptor)->New(); >>> delete my_message; >>> */ >>> >>> >>> >>> } >>> >>> >>> Finally, the point raised by Kenton and you that the proto file did not >>> parse correctly to yield the valid FileDescriptor might be correct, since, I >>> am generating the .proto file containing the messages on the fly by >>> converting corresponding tags from my XML file. >>> When earlier I used a handmade proto file for the messages, which parsed >>> correctly to provide the descriptors, I was able to stream data into >>> protocol buffers and use the buffers correctly. I will be inspecting this >>> file at my end for any typing errors. >>> >>> Thanks for the answer and for opening this protobuf library on public >>> domain, though. >>> >>> >>> Thanks and Regards, >>> >>> Himanshu >>> >>> >>> >>> >>> >>> >>> On Thu, Sep 23, 2010 at 7:53 AM, Jason Hsueh <[email protected]> wrote: >>> >>>> It occurred to me that protoc shouldn't accept your file either. Did >>>> it really compile successfully? What do you actually get in your >>>> FileDescriptorSet? >>>> >>>> On Wednesday, September 22, 2010, Jason Hsueh <[email protected]> >>>> wrote: >>>> > 1) I would guess the problem is that the "action" message type refers >>>> to undefined types eaction_type and ereason. There are probably error >>>> messages farther up indicating that action could not be built. Because >>>> action isn't a valid type, RecordHeader can't refer to it, and thinks that >>>> action is an undefined_symbol as well. The DescriptorBuilder will read in >>>> all inputs, and cross-link the types once everything has been processed. It >>>> doesn't matter what order they are defined in. >>>> > 2) You can manually build the FileDescriptorProto representation >>>> yourself, or use the classes used by the compiler: see >>>> google::protobuf::compiler::Parser or google::protobuf::compiler::Importer. >>>> > >>>> > On Wed, Sep 22, 2010 at 4:42 AM, Himanshu Srivastava < >>>> [email protected]> wrote: >>>> > >>>> > Hi Kenton, >>>> > I have the following message in my .proto file: >>>> > //************************************************/ >>>> > package traceFileMessagesProto; >>>> > message RecordHeader { >>>> > message myqueue{ >>>> > optional uint32 id = 1; >>>> > optional uint32 Priority = 2; >>>> > } >>>> > >>>> > message action{ >>>> > optional eactiontype eactiontype_instance = 1; >>>> > optional ereason ereason_instance = 2; >>>> > optional myqueue myqueue_instance = 3; >>>> > } >>>> > optional action action_instance =1; >>>> > } //end RecordHeader >>>> > /***************************************************/ >>>> > Thus, "action" is nested message in "RecordHeader" and has "queue" >>>> submessage in it. >>>> > I used the protoc compiler -o <desc-file> option to generate the >>>> descriptor set file and loaded it through istream in c++ application, >>>> followed by streaming FileDescriptorSet object with this descriptor set >>>> file: >>>> > >>>> > >>>> > //***************************************************************// >>>> > ifstream desc_file("mydescriptor.dsc",ios::in); >>>> > FileDescriptorSet f; >>>> > f.ParseFromIstream(&desc_file); >>>> > //***************************************************************// >>>> > >>>> > >>>> > However, while parsing through DescriptorPool::BuildFile(f.file(0)); >>>> > at the line corresponding to the message file line: >>>> > => optional action action_instance =1; >>>> > the BuildFile() function exits by giving error: >>>> > undefined_symbol "action" >>>> > The code returns through the following function in protoc library: >>>> > //***************************************************************// >>>> > void DescriptorBuilder::AddNotDefinedError( >>>> > const string& element_name, >>>> > const Message& descriptor, >>>> > DescriptorPool::ErrorCollector::ErrorLocation location, >>>> > const string& undefined_symbol) { >>>> > if (possible_undeclared_dependency_ == NULL) { >>>> > AddError(element_name, descriptor, location, >>>> > "\"" + undefined_symbol + "\" is not defined."); >>>> > //***************************************************************// >>>> > The only possible explanation could be that the message "action" is >>>> not parsed before its instance declaration from my .proto/descriptor_set >>>> file. >>>> > Probably all nested messages should be parsed on their instance >>>> declarations in a recursive manner in the DescriptorPool::Buildfile() >>>> function. >>>> > >>>> > >>>> > Qustion1: >>>> > How do I solve the above issue? >>>> > Question 2: >>>> > Can not I simply load my messages in DescriptorPool through the .proto >>>> file instead of calling the service of protoc compiler to generate a >>>> descriptor file? I have to generate .proto file at runtime and I don't want >>>> to call another exe (protoc.exe) from my application for dependency >>>> reasons. >>>> > >>>> > >>>> > Thanks in advance for any solutions /directions. >>>> > Regards, >>>> > Himanshu >>>> > >>>> > >>>> > >>>> > >>>> > -- >>>> > You received this message because you are subscribed to the Google >>>> Groups "Protocol Buffers" group. >>>> > To post to this group, send email to [email protected]. >>>> > To unsubscribe from this group, send email to >>>> [email protected]<protobuf%[email protected]> >>>> . >>>> > For more options, visit this group at >>>> http://groups.google.com/group/protobuf?hl=en. >>>> > >>>> > >>>> > >>>> >>> >>> >> > -- You received this message because you are subscribed to the Google Groups "Protocol Buffers" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/protobuf?hl=en.
