I'll give it a try. I haven't built the protobuf libraries with instrumenting support or else I'd already know, but I should be able to get it working.
On Mar 5, 5:20 pm, Kenton Varda <[email protected]> wrote: > Wow, that's interesting. I don't know why it would do that. Can you look > deeper into your profiles and see what part of Clear() is taking so long? > For example, is it spending the time clearing STL strings? > > On Thu, Mar 5, 2009 at 3:11 PM, Zachary Turner <[email protected]>wrote: > > > > > I have a fairly old version of the protobuf library, so if this has > > been changed let me know, but I have a situation where Message::Clear > > () is causing my cpu to go to like 70% for an extended period of time. > > > It's also possible this is user error, so please correct me if that's > > the case. > > > Basically what I have is a top level message with a bunch of optional > > messages, which I send across the wire. > > > One of these optional messages is defined as follows: > > > message DataChunkList { > > required bool is_end_of_list = 1; > > repeated DataChunk data = 2; > > }; > > > message DataChunk { > > optional bytes data = 1; > > //Other fields here > > }; > > > The "data" field will almost always be exactly 4k, and I will usually > > not want to send 1 chunk at a time, but a list of around 32 at a > > time. > > > So I save an instance of the top level message in the class containing > > my sending code, and right before I'm about to send data I do the > > following: > > > net::DataChunkList* pChunks = m_CachedTopLevel.mutable_data_chunk_list > > (); > > > //Should already be clear, but just in case > > pChunks->Clear(); > > prevCount = pChunks->mutable_data()->ClearedCount(); > > > for (int i=prevCount; i < num_chunks; ++i) > > { > > net::DataChunk* pChunk = new net::DataChunk(); > > pChunk->mutable_data()->reserve(4096); > > pChunkList->mutable_data()->AddCleared(pChunk); > > } > > > for (int i=0; i < num_chunks; ++i) > > { > > net::DataChunk* pChunk = pChunks->mutable_data()->ReleaseCleared(); > > pChunk->mutable_data()->assign(global_4k_buffer, 4096); > > pChunks->mutable_data()->AddAllocated(pChunks); > > } > > > send(m_CachedTopLevel); > > > m_CachedTopLevel.Clear(); > > > I ran a profiler on my code, and the very last line (the Clear()) > > takes up almost 95% of the CPU usage for the function, and the > > function takes up about about 30% of the CPU usage of the entire app. > > So obviously this is a big problem. > > > The comment on the code says that clear "does not free any memory" > > however. So why could it be using so much CPU? Am I misunderstanding > > the purpose / usage of these methods? What I'm trying to do is just > > re-use a pool of 4k buffers for all of these sends. --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Protocol Buffers" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~----------~----~----~----~------~----~------~--~---
