Hi, I have a large set of files with a number of the same type messages saved. My code reads messages in a sequence from these files one after another. I've measured time (with terminal "time" command) of running the code, and get something like:
READING ======= Read ProtoBuf Processed events: 50000000 real 7m2.146s user 5m25.545s sys 0m31.959s Then I've adjusted the code to read the files in threads (8 threads on 8 cores machine). The reading procedure is independent and put into separate class. Therefore each thread is really independent of the others. Nevertheless, the time measurement is: READING (MULTITHREADS) ===================== Thread read 6000000 events Thread read 6000000 events Thread read 6000000 events Thread read 6000000 events Thread read 6000000 events Thread read 6000000 events Thread read 7000000 events Thread read 7000000 events real 5m3.808s user 5m42.301s sys 0m35.221s As you may see, the "user" as well as "real" time is pretty much the same. So, it seems that there is some internal locks done somewhere. I only use locks between threads and class, that creates and manages threads. The locks are used only when thread finishes reading the file(s). Does ProtoBuf use some sort of generic static/singleton functions/objects that are used to de-serialize messages and therefore lock when accessed form different threads? If so, is there a way to suppress this and get truly independent messages parsing? thanks. P.S. My code can be browsed on github: http://goo.gl/DXCCF . The reading of messages is done by: http://goo.gl/OsHV9 The code uses ROOT framework (root.cern.ch) if one wants to compile it. -- You received this message because you are subscribed to the Google Groups "Protocol Buffers" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/protobuf?hl=en.
