Jason,
Thanks for replying. Java API has parseDelimitedFrom method. I used it to
parse it back.

  while ((record = LogRecord.parseDelimitedFrom(bufferIn)) != null) {
            StringBuffer sb = new StringBuffer();
            for (Object value : record.getAllFields().values()) {
                sb.append(value.toString());
                sb.append("\t");
            }
            sb.replace(sb.length() - 1, sb.length(), "\n");
            bufferOut.write(sb.toString()
                    .getBytes());
        }

On Thu, Apr 12, 2012 at 4:20 PM, Jason Hsueh <jas...@google.com> wrote:

> You won't be able to parse the data as a LogFile with the format you
> write, but if you are ok with manually parsing each delimited message, that
> approach will be fine.
>
> There are probably other folks who are more familiar with the Java
> internals and how to optimize for builder reuse etc.
>
> On Wed, Apr 11, 2012 at 2:06 PM, Shrijeet Paliwal <
> shrij...@rocketfuelinc.com> wrote:
>
>> Than you Jason. Yes I profiled it and found > 50% of execution time
>> was going into invokation of this method :
>>
>> public static ByteString copyFromUtf8(final String text)
>>
>> Digging further it appeared this was called again and again while
>> LogFile was trying to find the size of message.
>> What I decide to do was , instead of calling logfile.add(record) and
>> then finally calling a single writeTo(), I instead
>> called 'recordB.build().writeDelimitedTo(output)' in each iteration of
>> loop. Execution time dropped by 50%.
>>
>> What do you think Jason? Am I making things up or this makes sense?
>>
>>
>> On Apr 11, 1:37 pm, Jason Hsueh <jas...@google.com> wrote:
>> > Have you profiled this program? I would be surprised if protobuf code
>> > consumes a significant amount of time: all you are doing is setting the
>> > field values.
>> >
>> > If you want to minimize memory usage, you can probably flush your
>> LogFile
>> > periodically to the stream: as long as you don't write anything else to
>> the
>> > stream, the concatenated data will be properly parsed into a single
>> message
>> > with all the LogRecords.
>> >
>> > On Tue, Apr 10, 2012 at 5:51 PM, Shrijeet Paliwal <
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> > shrij...@rocketfuelinc.com> wrote:
>> > > Can one of more experienced members comment on this attempt
>> > >https://gist.github.com/2355991
>> > > It took 25 seconds to convert a 261M file using this. The heap usage
>> > > was > 512M
>> > > I am hoping for tips to improve performance.
>> >
>> > > On Apr 6, 3:14 pm, Shrijeet Paliwal <shrij...@rocketfuelinc.com>
>> > > wrote:
>> > > > Could some one me direct on how to parse and map a delimited string
>> to
>> > > > a protobuf Message, example a tab delimited string with 100 fields
>> > > > needs to be mapped to respective protobuf defined message. I am
>> trying
>> > > > to convert a text log file to a protobuf serialized file.
>> >
>> > > > -Shrijeet
>> >
>> > > --
>> > > You received this message because you are subscribed to the Google
>> Groups
>> > > "Protocol Buffers" group.
>> > > To post to this group, send email to protobuf@googlegroups.com.
>> > > To unsubscribe from this group, send email to
>> > > protobuf+unsubscr...@googlegroups.com.
>> > > For more options, visit this group at
>> > >http://groups.google.com/group/protobuf?hl=en.
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Protocol Buffers" group.
>> To post to this group, send email to protobuf@googlegroups.com.
>> To unsubscribe from this group, send email to
>> protobuf+unsubscr...@googlegroups.com.
>> For more options, visit this group at
>> http://groups.google.com/group/protobuf?hl=en.
>>
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To post to this group, send email to protobuf@googlegroups.com.
To unsubscribe from this group, send email to 
protobuf+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en.

Reply via email to