I am kind of new to Protocol Buffers and really like it at beginning. Later
(now) I found out it has powerful framework for generate data structure
class from definition file but very limited entrance, only its own binary
format. It looks it have huge mansion but only one small window. It looks
you climbed 999 steps, one step away from the door to a new world but you
stopped right there.
My suggested solution will
· Support all kinds of data format like XML, JSon, Text, Binary.
· Support customized schema like <Person><name>John Doe</name> or
<Person f1="John Doe" or <class type="Person" name="John Doe"
· The freedom how you want to generate it or parse it, user can
choose XmlStreamWriter or DOM or string concatenation or anything you want
to make XML output.
· This solution does not use reflection. It uses OO only. No tricky
stuff. Good Performance. So pretty much across all supported languages.
· The solution is very simple, no plugin or add-on. One standard
library, one standard compiler and one standard generated output code. But
it is not limited to one output format. You can create one object in memory
and output same object as XmlFormat1, XmlFormat2, Binary1,… at same time
· Only Serialization? Think broader! How about Save / Load from DB,
output to excel / PDF, generate html table? Think about the whole universe
of unified data handling.
The bad news is that the solution will be very hard to implement, like
“9999 steps” to the door for everything I mentioned…
Just kidding, it is a piece of coke to smart Google. But definitely not
just “one step”, Let me make “two” for now.
Let me talk about output first. So what you need to do is
A: define a simple interface in you library, for example, in java code
public interface IObjectWriter {
public void writeStartObject(String classname) throws
Exception;
public void writeEndObject() throws Exception;
public void writeStartField(int index, String fieldname)
throws Exception;
public void writeEndField() throws Exception;
public void writeData(String data) throws
Exception;
public void writeData(int data) throws Exception;
public void writeRepeatData(String data) throws
Exception;
public void writeRepeatData(int data) throws Exception;
}
B. When generated java class definition, add following function piece. It
is very similar the function what you generated code to output your binary
format (“public void writeTo(com.google.protobuf.CodedOutputStream
output)”). The framework is already there. Take me about one hour to add
this piece in libprotoc projects.
public void writeTo(IObjectWriter writer)
throws java.lang.Exception {
writer.writeStartObject("Person");
if (((bitField0_ & 0x00000001) == 0x00000001)) {
writer.writeStartField(1, "name");
writer.writeData(getName());
writer.writeEndField();
}
if (((bitField0_ & 0x00000002) == 0x00000002)) {
writer.writeStartField(2, "id");
writer.writeData(id_);
writer.writeEndField();
}
if (((bitField0_ & 0x00000004) == 0x00000004)) {
writer.writeStartField(3, "email");
writer.writeData(getEmail());
writer.writeEndField();
}
writer.writeStartField(4, "relatedPersonIds");
for (int i = 0; i < relatedPersonIds_.size(); i++) {
writer.writeRepeatData(relatedPersonIds_.get(i));
}
writer.writeEndField();
writer.writeStartField(5, "othernames");
for (int i = 0; i < othernames_.size(); i++) {
writer.writeRepeatData(othernames_.get(i));
}
writer.writeEndField();
if (((bitField0_ & 0x00000008) == 0x00000008)) {
writer.writeStartField(10, "myphone");
myphone_.writeTo(writer);
writer.writeEndField();
}
writer.writeStartField(11, "phone");
for (int i = 0; i < phone_.size(); i++) {
phone_.get(i).writeTo(writer);
}
writer.writeEndField();
writer.writeEndObject();
}
C. You are done. Finished. Yes, you are “one step” closer to the door now.
Let me describe how it works: protocol buffer generated class focus on data
structure and left input / output implementation to user. It is the user
who creates the customized class implements IObjectWriter to do whatever
they want. Person object can writes like
writer.writeStartObject("Person"); writer.writeStartField(1, "name");
writer.writeData("John
Doe"); writer.writeEndField(); writer.writeEndObject(); xmlwriter using
index, attribute <Person f1="John Doe" /> xmlwriter using name, element
<Person> <name> John Doe </name> </Person> jsonwriter using name {
"name":"John
Doe", } DBWriter Set "Add" + "Person" store procedure Add SQL Store
procedure Parameter Execute the store procedure Htmlwriter <tr> <td>John
Doe</td> </tr>
Got it? Super Easy, right?
Next talk about reader. I did not really code it yet but the idea is clear.
Same thing, protocol buffer generated class focus on data structure, in
another word, how to set value according to index or name. So it should
have something like SetValue(string name, object value), SetValue(int
index, object value), SetMessage(string name, IDataReader reader),
SetMessage(int index, IDataReader reader). There are nothing about input
type / schema. On the other hand, user defined class which implements
IObjectReader for how to parse the content and set object value
accordingly. This is a little complicate than Writer but it is easy too.
OK, now you are really at the door. Think about it. Google Protocol Buffer
binary format is only one vertex vs. the whole multiple dimensions universe
I mentioned above. (I assume Protocol Buffer binary format can be
transferred to subclass of IDataWriter and IDataReader as well, of course,
more information or functions will be added).
In general, I appreciated the design and hard work of Google on Message
definition and generated beautiful data structure. But mix binary format
into it really mess up the design. Maybe binary format is the major reason
Google developed this project, but at same time, the whole world is closed
with only one window left. Frankly, I think Google make a mistake here and
need take action now.
If you have any questions or feedback, please let me know. Also I do have
some code / sample there if you are interested. My email address is
[email protected].
Thanks
Li Chen
--
You received this message because you are subscribed to the Google Groups
"Protocol Buffers" group.
To view this discussion on the web visit
https://groups.google.com/d/msg/protobuf/-/H5Buld8VONIJ.
To post to this group, send email to [email protected].
To unsubscribe from this group, send email to
[email protected].
For more options, visit this group at
http://groups.google.com/group/protobuf?hl=en.
