Howdy,
I'm working on a project that recently needed to insert data represented by
protobufs into elasticsearch. Using the built in JSON serialization we
were able to quickly get data into elasticsearch, however, the JSON
serialization seems to be rather slow when compared to generating with a
library like rapidjson. Is this expected or is a likely we're doing
something wrong? Below is info on what we're using, and relative
serialization performance results. Surprisingly, rapidjson serialization
was faster than protobufs binary serialization in some cases, which leads
me to believe I'm doing something wrong.
Ubuntu 16.04
GCC 7.3, std=c++17, libstdc++11 string api
Protobuf 3.5.1.1 compiled with -O3, proto3 syntax
I've measure the performance of 3 cases, serializing the protobuf to
binary, serializing the protobuf to JSON via MessageToJSONString, and
building a rapidjson::Document from the protobuf and then serializing that
to JSON. All tests use the same message with different portions of the
message populated, 100,000 iterations. The json generated from the
protobuf and rapidjson match exactly.
Test 1, a single string field populated.
proto binary: 0.01s
proto json: 0.50s
rapidjson: 0.02s
Test 2, 1 top level string field, 1 nested object with 3 more string fields.
proto binary: 0.02s
proto json: 1.06s
rapidjson: 0.05s
Test 3, 2 string fields, and 1 ::google::protobuf::ListValue containing
doubles of the format, [[[double, double], [double, double], ...]], 36
pairs of doubles total.
*proto binary: 1.50s*
*proto json: 8.87s*
*rapidjson: 0.41s*
Protobuf binary serialization code:
std::string toJSON(Message const& msg) { return
msg.SerializeAsString(); }
Protobuf json serialization code:
std::string toJSON(Message const& msg) { return
msg.SerializeAsString(); }
std::string json;
::google::protobuf::util::MessageToJsonString(msg,
std::addressof(json));
return json;
}
Rapidjson serialization code:
// It's a lengthy section of code manually populating the document. Of
note, empty strings and numbers set to 0 are omitted from the JSON as the
protobuf does. The resulting JSON is exactly the same as the protobuf json.
Any info on how to improve the protobuf to JSON serialization would be
greatly appreciated!
Thanks,
Ed
--
You received this message because you are subscribed to the Google Groups
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
Visit this group at https://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.
