Add this to your .proto file: option optimize_for = SPEED; Does it help?
On Thu, Mar 5, 2009 at 3:23 PM, Zachary Turner <[email protected]>wrote: > > I'll give it a try. I haven't built the protobuf libraries with > instrumenting support or else I'd already know, but I should be able > to get it working. > > On Mar 5, 5:20 pm, Kenton Varda <[email protected]> wrote: > > Wow, that's interesting. I don't know why it would do that. Can you > look > > deeper into your profiles and see what part of Clear() is taking so long? > > For example, is it spending the time clearing STL strings? > > > > On Thu, Mar 5, 2009 at 3:11 PM, Zachary Turner <[email protected] > >wrote: > > > > > > > > > I have a fairly old version of the protobuf library, so if this has > > > been changed let me know, but I have a situation where Message::Clear > > > () is causing my cpu to go to like 70% for an extended period of time. > > > > > It's also possible this is user error, so please correct me if that's > > > the case. > > > > > Basically what I have is a top level message with a bunch of optional > > > messages, which I send across the wire. > > > > > One of these optional messages is defined as follows: > > > > > message DataChunkList { > > > required bool is_end_of_list = 1; > > > repeated DataChunk data = 2; > > > }; > > > > > message DataChunk { > > > optional bytes data = 1; > > > //Other fields here > > > }; > > > > > The "data" field will almost always be exactly 4k, and I will usually > > > not want to send 1 chunk at a time, but a list of around 32 at a > > > time. > > > > > So I save an instance of the top level message in the class containing > > > my sending code, and right before I'm about to send data I do the > > > following: > > > > > net::DataChunkList* pChunks = m_CachedTopLevel.mutable_data_chunk_list > > > (); > > > > > //Should already be clear, but just in case > > > pChunks->Clear(); > > > prevCount = pChunks->mutable_data()->ClearedCount(); > > > > > for (int i=prevCount; i < num_chunks; ++i) > > > { > > > net::DataChunk* pChunk = new net::DataChunk(); > > > pChunk->mutable_data()->reserve(4096); > > > pChunkList->mutable_data()->AddCleared(pChunk); > > > } > > > > > for (int i=0; i < num_chunks; ++i) > > > { > > > net::DataChunk* pChunk = pChunks->mutable_data()->ReleaseCleared(); > > > pChunk->mutable_data()->assign(global_4k_buffer, 4096); > > > pChunks->mutable_data()->AddAllocated(pChunks); > > > } > > > > > send(m_CachedTopLevel); > > > > > m_CachedTopLevel.Clear(); > > > > > I ran a profiler on my code, and the very last line (the Clear()) > > > takes up almost 95% of the CPU usage for the function, and the > > > function takes up about about 30% of the CPU usage of the entire app. > > > So obviously this is a big problem. > > > > > The comment on the code says that clear "does not free any memory" > > > however. So why could it be using so much CPU? Am I misunderstanding > > > the purpose / usage of these methods? What I'm trying to do is just > > > re-use a pool of 4k buffers for all of these sends. > > > --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Protocol Buffers" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~----------~----~----~----~------~----~------~--~---
