I get somewhat better results with that flag. I built protobuf with profiling enabled and I'm a little suspicious that the information is 100% accurate, but it seems like std::string::clear() takes up the most time. But the percentages don't match up to what I calculate, so I'm not sure where the inconsistency is.
Just out of curiosity, is there even any need for me to call Clear()? I'm filling out every single field every single time, and always using mutable_data()->assign() to copy the data into the message, so is it fine to just leave it "uncleared" but still stick it back into the cleared list? On Thu, Mar 5, 2009 at 5:25 PM, Kenton Varda <[email protected]> wrote: > Add this to your .proto file: > option optimize_for = SPEED; > > Does it help? > > On Thu, Mar 5, 2009 at 3:23 PM, Zachary Turner <[email protected]>wrote: > >> >> I'll give it a try. I haven't built the protobuf libraries with >> instrumenting support or else I'd already know, but I should be able >> to get it working. >> >> On Mar 5, 5:20 pm, Kenton Varda <[email protected]> wrote: >> > Wow, that's interesting. I don't know why it would do that. Can you >> look >> > deeper into your profiles and see what part of Clear() is taking so >> long? >> > For example, is it spending the time clearing STL strings? >> > >> > On Thu, Mar 5, 2009 at 3:11 PM, Zachary Turner <[email protected] >> >wrote: >> >> > >> > >> > >> > > I have a fairly old version of the protobuf library, so if this has >> > > been changed let me know, but I have a situation where Message::Clear >> > > () is causing my cpu to go to like 70% for an extended period of time. >> > >> > > It's also possible this is user error, so please correct me if that's >> > > the case. >> > >> > > Basically what I have is a top level message with a bunch of optional >> > > messages, which I send across the wire. >> > >> > > One of these optional messages is defined as follows: >> > >> > > message DataChunkList { >> > > required bool is_end_of_list = 1; >> > > repeated DataChunk data = 2; >> > > }; >> > >> > > message DataChunk { >> > > optional bytes data = 1; >> > > //Other fields here >> > > }; >> > >> > > The "data" field will almost always be exactly 4k, and I will usually >> > > not want to send 1 chunk at a time, but a list of around 32 at a >> > > time. >> > >> > > So I save an instance of the top level message in the class containing >> > > my sending code, and right before I'm about to send data I do the >> > > following: >> > >> > > net::DataChunkList* pChunks = m_CachedTopLevel.mutable_data_chunk_list >> > > (); >> > >> > > //Should already be clear, but just in case >> > > pChunks->Clear(); >> > > prevCount = pChunks->mutable_data()->ClearedCount(); >> > >> > > for (int i=prevCount; i < num_chunks; ++i) >> > > { >> > > net::DataChunk* pChunk = new net::DataChunk(); >> > > pChunk->mutable_data()->reserve(4096); >> > > pChunkList->mutable_data()->AddCleared(pChunk); >> > > } >> > >> > > for (int i=0; i < num_chunks; ++i) >> > > { >> > > net::DataChunk* pChunk = pChunks->mutable_data()->ReleaseCleared(); >> > > pChunk->mutable_data()->assign(global_4k_buffer, 4096); >> > > pChunks->mutable_data()->AddAllocated(pChunks); >> > > } >> > >> > > send(m_CachedTopLevel); >> > >> > > m_CachedTopLevel.Clear(); >> > >> > > I ran a profiler on my code, and the very last line (the Clear()) >> > > takes up almost 95% of the CPU usage for the function, and the >> > > function takes up about about 30% of the CPU usage of the entire app. >> > > So obviously this is a big problem. >> > >> > > The comment on the code says that clear "does not free any memory" >> > > however. So why could it be using so much CPU? Am I misunderstanding >> > > the purpose / usage of these methods? What I'm trying to do is just >> > > re-use a pool of 4k buffers for all of these sends. >> >> >> > --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Protocol Buffers" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~----------~----~----~----~------~----~------~--~---
