Hey Christopher,

I wanted to ask that do you use RPC calling for your applications? You
serialized the data into the files and send those files to the
distributed processing. If I am getting this right then RPC can be one
way for the client server architecture. So what your feed on that. Is
protocol buffer performs good in the distributed environment.

Stuti

On Apr 20, 9:37 am, Christopher Smith <cbsm...@gmail.com> wrote:
> We do stuff very much like this. Generally we try to encode the PB's as 
> sequence files so that they are splittable. If that doesn't make sense, then 
> we use non-splittable coded streams.
>
> --Chris
>
> On Apr 19, 2010, at 9:22 PM, stuti awasthi <stutic...@gmail.com> wrote:
>
>
>
>
>
> > Thanks Jason,
>
> > Yes map reduce works with the raw bytes of data. I was planning to
> > have log analysis over hadoop using map reduce. Suppose I have ever
> > increasing log files and I want to serialize its data and then pass
> > over the Hadoop map reduce to get it analyzed. If this is the use case
> > then do we have any mechanism through which we can feed the serialized
> > data into the map reduce?
>
> > Bye :)
>
> > On Apr 19, 10:32 pm, Jason Hsueh <jas...@google.com> wrote:
> >> As far as I recall, mapreduce just works with raw bytes. If you want to use
> >> protocol buffers within map reduce, you just need to use the serialization
> >> routines to convert from the raw bytes to your proto.
>
> >> On Mon, Apr 19, 2010 at 4:06 AM, stuti awasthi <stutic...@gmail.com> wrote:
> >>> Hi all,
>
> >>> I just started looking at the Protocol Buffer for serialization of
> >>> data structure. I wanted to use this with the map reduce framework. I
> >>> searched on web but do not get any specific pointers on how to do it.
>
> >>> If any body can please tell me how can we use PB with the map reduce
> >>> framework. Im new to this concept of distributed serialization.
>
> >>> Any pointers will be great.. Thanks in advance.
>
> >>> Bye
>
> >>> --
> >>> You received this message because you are subscribed to the Google Groups
> >>> "Protocol Buffers" group.
> >>> To post to this group, send email to proto...@googlegroups.com.
> >>> To unsubscribe from this group, send email to
> >>> protobuf+unsubscr...@googlegroups.com<protobuf%2bunsubscr...@googlegroups.com>
> >>> .
> >>> For more options, visit this group at
> >>>http://groups.google.com/group/protobuf?hl=en.
>
> >> --
> >> You received this message because you are subscribed to the Google Groups 
> >> "Protocol Buffers" group.
> >> To post to this group, send email to proto...@googlegroups.com.
> >> To unsubscribe from this group, send email to 
> >> protobuf+unsubscr...@googlegroups.com.
> >> For more options, visit this group 
> >> athttp://groups.google.com/group/protobuf?hl=en.
>
> > --
> > You received this message because you are subscribed to the Google Groups 
> > "Protocol Buffers" group.
> > To post to this group, send email to proto...@googlegroups.com.
> > To unsubscribe from this group, send email to 
> > protobuf+unsubscr...@googlegroups.com.
> > For more options, visit this group 
> > athttp://groups.google.com/group/protobuf?hl=en.
>
> --
> You received this message because you are subscribed to the Google Groups 
> "Protocol Buffers" group.
> To post to this group, send email to proto...@googlegroups.com.
> To unsubscribe from this group, send email to 
> protobuf+unsubscr...@googlegroups.com.
> For more options, visit this group 
> athttp://groups.google.com/group/protobuf?hl=en.

-- 
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To post to this group, send email to proto...@googlegroups.com.
To unsubscribe from this group, send email to 
protobuf+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en.

Reply via email to