Than you Jason. Yes I profiled it and found > 50% of execution time was going into invokation of this method :
public static ByteString copyFromUtf8(final String text) Digging further it appeared this was called again and again while LogFile was trying to find the size of message. What I decide to do was , instead of calling logfile.add(record) and then finally calling a single writeTo(), I instead called 'recordB.build().writeDelimitedTo(output)' in each iteration of loop. Execution time dropped by 50%. What do you think Jason? Am I making things up or this makes sense? On Apr 11, 1:37 pm, Jason Hsueh <[email protected]> wrote: > Have you profiled this program? I would be surprised if protobuf code > consumes a significant amount of time: all you are doing is setting the > field values. > > If you want to minimize memory usage, you can probably flush your LogFile > periodically to the stream: as long as you don't write anything else to the > stream, the concatenated data will be properly parsed into a single message > with all the LogRecords. > > On Tue, Apr 10, 2012 at 5:51 PM, Shrijeet Paliwal < > > > > > > > > [email protected]> wrote: > > Can one of more experienced members comment on this attempt > >https://gist.github.com/2355991 > > It took 25 seconds to convert a 261M file using this. The heap usage > > was > 512M > > I am hoping for tips to improve performance. > > > On Apr 6, 3:14 pm, Shrijeet Paliwal <[email protected]> > > wrote: > > > Could some one me direct on how to parse and map a delimited string to > > > a protobuf Message, example a tab delimited string with 100 fields > > > needs to be mapped to respective protobuf defined message. I am trying > > > to convert a text log file to a protobuf serialized file. > > > > -Shrijeet > > > -- > > You received this message because you are subscribed to the Google Groups > > "Protocol Buffers" group. > > To post to this group, send email to [email protected]. > > To unsubscribe from this group, send email to > > [email protected]. > > For more options, visit this group at > >http://groups.google.com/group/protobuf?hl=en. -- You received this message because you are subscribed to the Google Groups "Protocol Buffers" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/protobuf?hl=en.
