If I understand correctly, you are talking about two separate issue. One is that I use of ParseFromString() is unreliable. The other one is that I should switch to self describing messages. The latter is not feasible for now; I receive serialized PB messages that do not contain their type name, and need to figure out what type they are. In my particular setup, I only have one message type, and still cannot figure out the messages that come from the Java sender. Is there any difference between the way things get serialized in Java vs. C++, or is it a bug in my code?
Nader On 8/27/2011 13:11 Jason Hsueh writes: > It appears that the C++ listener code supports multiple types, but assumes > that the correct type can be determined by checking if parsing the serialized > string returns successfully. This is not a reliable way to detect message > types: in fact ParseFromString() only fails if the serialized data is not a > valid protocol buffer encoding (not type-specific), or if the message > definition contains required fields and those fields were not seen in the > serialized data. You should also transmit the message type name: http:// > code.google.com/apis/protocolbuffers/docs/techniques.html#self-description > > On Fri, Aug 26, 2011 at 10:27 PM, Nader Salehi <[email protected]> wrote: > > I am having difficulty understanding the problem I am having with > parsing of some of the messages that I produce in my test environment. > There seems to be some difference between how things are constructed > and serialized in Java vs. C++. That is, unless there is a bug in my > code :) > > Here is how I have set things up: > > * There is one Java sender, which periodically sends messages of type > A. It also sends the FileDescriptorSet that is required for > parsing A. Both events are serialized and sent over UDP packets > > * There is also a C++ sender, which sends messages of the same type A > as the Java sender. It does NOT, however, send the > FileDescriptorSet. Like the Java version, it serializes the > message and sends it over UDP. > > * There is a C++ listener, which listens to both senders. It > successfully parse the FileDescriptorSet that the Java app sends, > creates SimpleDescriptorDatabase from it, and creates a > DescriptorPool using the SimpleDescriptorDatabase. > > The listener receives both message and the descriptor from the Java > sender. It is able to dynamically parse the message, but when it > tries to display the message, using DebugString(), it fails to show > field names. Instead, it only shows field number. But, it has > absolutely no problem showing field names for messages it receives > from the C++ sender. Note that the file descriptor set comes from the > Java sender. > > I am adding the Java module where it creates the file descriptor set. > I also add the C++ code where it a) creates the descriptor pool and b) > uses message factory to parse the message. I would appreciate it if > someone could give me a hint as to what might be in play. > > Thanks, > Nader > > Java > -------------------------------------------------------------------------- > public static FileDescriptorSet getDescriptorSetFor(final Message > aMessage) { > final FileDescriptor file = aMessage.getDescriptorForType > ().getFile(); > final List<FileDescriptor> fileDescriptors = new ArrayList > <FileDescriptor>(); > > Stack<FileDescriptorDependency> dependencyStack = > new Stack<FileDescriptorDependency>(); > FileDescriptorDependency fDescObj = > new FileDescriptorDependency(); > fDescObj.setFDesc(file); > fDescObj.setDependencyCnt(0); > dependencyStack.push(fDescObj); > while (!dependencyStack.empty()) { > fDescObj = dependencyStack.pop(); > if (fDescObj.getDependencyCnt() == > fDescObj.getFDesc().getDependencies().size()) { > > // Time to add the file descriptor to the file > // descriptor set > fileDescriptors.add(fDescObj.getFDesc()); > } else { > int idx = fDescObj.getDependencyCnt(); > fDescObj.setDependencyCnt(idx + 1); > dependencyStack.push(fDescObj); > List<FileDescriptor> depList = > fDescObj.getFDesc().getDependencies(); > FileDescriptorDependency fddObj = > new FileDescriptorDependency(); > fddObj.setFDesc(depList.get(idx)); > fddObj.setDependencyCnt(0); > dependencyStack.push(fddObj); > } > } > > final FileDescriptorSet.Builder builder = > FileDescriptorSet.newBuilder(); > for (FileDescriptor descriptor : fileDescriptors) { > builder.addFile(descriptor.toProto()); > } > return builder.build(); > } > > C++ > > ----------------------------------------------------------------------------- > <snip> > struct DBContainer { > DBContainer(const std::string & name, > boost::shared_ptr<pb::SimpleDescriptorDatabase> db) : > typeName(name), > descDB(db), > pool(db.get()) > { > } > > const std::string & operator()() const { return typeName; } > > std::string typeName; > boost::shared_ptr<pb::SimpleDescriptorDatabase> descDB; > pb::DescriptorPool pool; > }; > > <snip> > > void > MessageDetector::insert (const pb::FileDescriptorSet & fDescSet, > const string & typeName) > { > vector<DBContainerPtr>::iterator iter = > std::find_if(dbContainerVec.begin(), > dbContainerVec.end(), > boost::bind(&DBContainer::operator(), _1) == > typeName); > if (iter == dbContainerVec.end()) { > // Need to add the file descriptor set to the data base > cout << "Adding " << typeName << endl; > cout << fDescSet.DebugString() << endl; > shared_ptr<pb::SimpleDescriptorDatabase> > db(new pb::SimpleDescriptorDatabase); > for (int i = 0; i < fDescSet.file_size(); ++i) { > assert(db->Add(fDescSet.file(i))); > for (int j = 0; j < fDescSet.file(i).message_type_size(); ++j) > { > string formalName; > formalName += fDescSet.file(i).has_package() ? > fDescSet.file(i).package() + "." : ""; > formalName += fDescSet.file(i).message_type(j).name(); > DBContainerPtr dbContainerElem(new DBContainer(formalName, > db)); > dbContainerVec.push_back(dbContainerElem); > } > } > } > } > > void > MessageDetector::discoverAndDump (const string & msg) > { > bool found = false; > for (int i = dbContainerVec.size() - 1; !found && i >= 0; --i) { > pb::DynamicMessageFactory factory; > const pb::Descriptor * desc = > dbContainerVec[i]->pool.FindMessageTypeByName > (dbContainerVec[i]->typeName); > std::auto_ptr<pb::Message> > revEngMessage(factory.GetPrototype(desc)->New()); > revEngMessage->ParsePartialFromString(msg); > if (revEngMessage->IsInitialized()) { > cout << revEngMessage->DebugString() << endl; > found = true; > } > } > > if (!found) { > cout << (dbContainerVec.size() == 0 ? > "No valid announcement has been received" : > "No message found for the message") > << endl; > } > > -- > > -- > You received this message because you are subscribed to the Google Groups > "Protocol Buffers" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]. > For more options, visit this group at http://groups.google.com/group/ > protobuf?hl=en. > > -- > You received this message because you are subscribed to the Google Groups > "Protocol Buffers" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]. > For more options, visit this group at http://groups.google.com/group/protobuf? > hl=en. > -- -- You received this message because you are subscribed to the Google Groups "Protocol Buffers" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/protobuf?hl=en.
