Status: New
Owner: liuj...@google.com
Labels: Type-Defect Priority-Medium
New issue 341 by eyal.far...@gmail.com: proposed optimization for parsing
of packed repeated fields
http://code.google.com/p/protobuf/issues/detail?id=341
What steps will reproduce the problem?
1.Create a message type with a packed repeated field of int32
2.Compile with the protoc compiler...
3.Parse a message with about 300-400 integers (in the packed field)
What is the expected output? What do you see instead?
It works, but could be faster
What version of the product are you using? On what operating system?
2.4.1 on win7 (x64)
Please provide any additional information below.
I've already posted this in the GPB group
(https://groups.google.com/forum/#!topic/protobuf/P0GBgD74v08), and was
suggested to open an issue.
basically I have a process that spends about half of its time parsing a
protobuf message with a large repeated field of integers (300-400 items).
profiling this process showed that ‘ReadPackedPrimitive’ dominated the
parsing, a closer look at the code revealed that it calls
‘BytesUntilLimit()’ proportionally to the number of items in the repeated
field, this method evaluates a condition and several arithmetic operations
which seems harmless but become annoying when executed repeatedly.
I initially patched my generated code to use a local function instead of
‘ReadPackedPrimitive’, the basic idea was to check the size of the
available buffer before resorting to ‘BytesUntilLimit()’, this buffer is
usually large enough to hold the next item (actually, in my case the entire
message feats in buffer). The result of this patch was a 10-15% increase in
throughput of my process, and 20% increase when the process was 'crippled'
to do only protobuf parsing (no farther processing).
I'm attaching a patch to wire_format_lite_inl.h, this patch is made on top
of V2.4.1
I'd appreciate any feed-back, and be more than happy if it's accepted.
Eyal.
Attachments:
protobuff.packed_rfield.diff 1.6 KB
--
You received this message because you are subscribed to the Google Groups "Protocol
Buffers" group.
To post to this group, send email to protobuf@googlegroups.com.
To unsubscribe from this group, send email to
protobuf+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/protobuf?hl=en.
