Re: [protobuf] Re: Serializing nested messages

2010-03-29 Thread Henner Zeller
HI,
On Mon, Mar 29, 2010 at 23:20, Nader Salehi  wrote:
>
>
> On Mar 29, 10:59 pm, Henner Zeller 
> wrote:
>> On Mon, Mar 29, 2010 at 22:53, Nader Salehi  wrote:
>> > These are valid points, but I need to work within the constraints that
>> > I have; I need to write serialized buffers into different files.  I
>> > might end up serializing the second time, but still would like to know
>> > if there is a way which is NOT a hack to avoid this.
>>
>> mmh, but what is wrong in having two files open, wrap them in streams
>> and serialize a() and b() into these individually ? No way to hack
>> around, two files you write to, minimal copying of data from a buffer
>> to a file.
>> So I guess you need to specify exactly what your constraints are.
>>
>
> Let's say that you are dealing with a spaghetti code and want to deal
> minimal changes :)
>
> For that reason, I want to avoid wrapping the fds in streams.  Also,
> what I'm reading from your comment is that there is no explicit API to
> allow me to grab the enclosed message and that I'm better off
> serializing a second time.  If true, then it's a good enough take-away
> for me also.

Or to put it this way: I never had the need to :) I'd just go ahead
and serialize the messages individually. You will probably not have
any measurable performance impact compared to serializing everything
at once and then splitting things apart.
The added benefit is as well that you only need to allocate a smaller
buffer for your serialization.

-h

>
> Thanks,
> Nader
>
>> -h
>>
>>
>>
>> > Nader
>>
>> > On Mar 29, 2:41 pm, Henner Zeller 
>> > wrote:
>> >> On Mon, Mar 29, 2010 at 10:31, Nader Salehi  
>> >> wrote:
>> >> > Hi,
>>
>> >> > In my code, I have a PB message which encompass other PB messages.
>> >> > For instance,
>>
>> >> > Protocol Buffer File
>> >> > ===
>> >> > message A {
>> >> >  required int32 a = 1;
>> >> > }
>>
>> >> > message B {
>> >> >  required int32 b = 1;
>> >> > }
>>
>> >> > message C {
>> >> >  required A a = 1;
>> >> >  required B b = 2;
>> >> > }
>>
>> >> > C++ File
>> >> > 
>> >> > #include "PB File.pb.h"
>>
>> >> > using namespace 
>>
>> >> > int main ()
>> >> > {
>> >> >  C c;
>> >> >   ...
>> >> >  char buff[MAX_SIZE]
>> >> >  int len = c.SerializeToArray(buff, MAX_SIZE);
>> >> >  write(fd, buff, len);
>> >> >  ...
>> >> > }
>>
>> >> > As part of the logic, I need to write into separate files the
>> >> > serialized equivalent of c.a and c.b.  Of course you can always call
>> >> > c.a().SerializeToArray() but that requires additional CPU time which I
>> >> > would like to avoid.  Is there a non-hackish way of getting the offset
>> >> > of c.a in the serialized buffer?  Can I use the offset and
>> >> > c.a().ByteSize() to write the message into the file descriptor?
>>
>> >> You might want to write the data directly to the stream with the
>> >> available streams instead of copying into a buffer first.
>> >> Second, I wouldn't worry at all about the required CPU serializing
>> >> c.a() and c.b() individually; you should first measure if it actually
>> >> makes a difference before thinking of optimizing it in a hackish way.
>>
>> >> -h
>>
>> > --
>> > You received this message because you are subscribed to the Google Groups 
>> > "Protocol Buffers" group.
>> > To post to this group, send email to proto...@googlegroups.com.
>> > To unsubscribe from this group, send email to 
>> > protobuf+unsubscr...@googlegroups.com.
>> > For more options, visit this group 
>> > athttp://groups.google.com/group/protobuf?hl=en.
>
> --
> You received this message because you are subscribed to the Google Groups 
> "Protocol Buffers" group.
> To post to this group, send email to proto...@googlegroups.com.
> To unsubscribe from this group, send email to 
> protobuf+unsubscr...@googlegroups.com.
> For more options, visit this group at 
> http://groups.google.com/group/protobuf?hl=en.
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To post to this group, send email to proto...@googlegroups.com.
To unsubscribe from this group, send email to 
protobuf+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en.



Re: [protobuf] Re: Serializing nested messages

2010-03-29 Thread Henner Zeller
On Mon, Mar 29, 2010 at 22:53, Nader Salehi  wrote:
> These are valid points, but I need to work within the constraints that
> I have; I need to write serialized buffers into different files.  I
> might end up serializing the second time, but still would like to know
> if there is a way which is NOT a hack to avoid this.

mmh, but what is wrong in having two files open, wrap them in streams
and serialize a() and b() into these individually ? No way to hack
around, two files you write to, minimal copying of data from a buffer
to a file.
So I guess you need to specify exactly what your constraints are.

-h

>
> Nader
>
> On Mar 29, 2:41 pm, Henner Zeller 
> wrote:
>> On Mon, Mar 29, 2010 at 10:31, Nader Salehi  wrote:
>> > Hi,
>>
>> > In my code, I have a PB message which encompass other PB messages.
>> > For instance,
>>
>> > Protocol Buffer File
>> > ===
>> > message A {
>> >  required int32 a = 1;
>> > }
>>
>> > message B {
>> >  required int32 b = 1;
>> > }
>>
>> > message C {
>> >  required A a = 1;
>> >  required B b = 2;
>> > }
>>
>> > C++ File
>> > 
>> > #include "PB File.pb.h"
>>
>> > using namespace 
>>
>> > int main ()
>> > {
>> >  C c;
>> >   ...
>> >  char buff[MAX_SIZE]
>> >  int len = c.SerializeToArray(buff, MAX_SIZE);
>> >  write(fd, buff, len);
>> >  ...
>> > }
>>
>> > As part of the logic, I need to write into separate files the
>> > serialized equivalent of c.a and c.b.  Of course you can always call
>> > c.a().SerializeToArray() but that requires additional CPU time which I
>> > would like to avoid.  Is there a non-hackish way of getting the offset
>> > of c.a in the serialized buffer?  Can I use the offset and
>> > c.a().ByteSize() to write the message into the file descriptor?
>>
>> You might want to write the data directly to the stream with the
>> available streams instead of copying into a buffer first.
>> Second, I wouldn't worry at all about the required CPU serializing
>> c.a() and c.b() individually; you should first measure if it actually
>> makes a difference before thinking of optimizing it in a hackish way.
>>
>> -h
>
> --
> You received this message because you are subscribed to the Google Groups 
> "Protocol Buffers" group.
> To post to this group, send email to proto...@googlegroups.com.
> To unsubscribe from this group, send email to 
> protobuf+unsubscr...@googlegroups.com.
> For more options, visit this group at 
> http://groups.google.com/group/protobuf?hl=en.
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To post to this group, send email to proto...@googlegroups.com.
To unsubscribe from this group, send email to 
protobuf+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en.