[protobuf] Re: Serializing part of a message and writing to disk
Thank you for the response. On Mar 8, 5:59 am, Evan Jones wrote: > On Mar 8, 2011, at 2:12 , Linus wrote: > > > At a later stage in the code, the values of (say) Message A are > > changed by the user. Is there a way of modifying only Message A and > > updating the file on disk, without loading the composite Message C > > updating the Message A and flushing entire contents to disk again? > > Not really: protocol buffers are a variable length encoding, so > changing Message A could change the length, so overwriting doesn't > really work, at least not without additional checks and effort. > > Evan > > --http://evanjones.ca/ -- You received this message because you are subscribed to the Google Groups "Protocol Buffers" group. To post to this group, send email to protobuf@googlegroups.com. To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/protobuf?hl=en.
[protobuf] Serializing part of a message and writing to disk
I have a "composite message", meaning Message C is composed of Message
A and Message B.
Message C
{
required Message A = 1;
required Message B = 2;
}
This composite message C I have serialized to disk ( say output.dat).
At a later stage in the code, the values of (say) Message A are
changed by the user. Is there a way of modifying only Message A and
updating the file on disk, without loading the composite Message C
updating the Message A and flushing entire contents to disk again?
Any help appreciated.
Thanks,
Linus
--
You received this message because you are subscribed to the Google Groups
"Protocol Buffers" group.
To post to this group, send email to protobuf@googlegroups.com.
To unsubscribe from this group, send email to
protobuf+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/protobuf?hl=en.
[protobuf] Re: Chunking a large message
Thanks Evan. Your response makes some things clear to me. I meant to say I'd like chunks of 1MB ( I re-read my email and realize it wasn't worded properly). I now have some ideas that I'm going to try out. Thanks again! On Mar 4, 8:37 am, Evan Jones wrote: > On Mar 3, 2011, at 15:53 , Linus wrote: > > > I am wondering if there are any examples of chunking large PB messages > > (about 1MB) into smaller chunks, to transmit over the wire. > > This is going to be pretty application specific. Typically it involves > taking one message with a huge repeated field and sending it / writing > it as a sequence of messages with fewer items for each repeated field. > So I can't really point you to any examples off the top of my head. > > That said: the documentation suggests keeping protocol buffers to be ~ > <1 MB in size, so if your messages are 1 MB, I personally wouldn't > worry about it. Hope this helps, > > Evan > > --http://evanjones.ca/ -- You received this message because you are subscribed to the Google Groups "Protocol Buffers" group. To post to this group, send email to protobuf@googlegroups.com. To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/protobuf?hl=en.
[protobuf] Chunking a large message
Hello, I am wondering if there are any examples of chunking large PB messages (about 1MB) into smaller chunks, to transmit over the wire. Thanks, Linus -- You received this message because you are subscribed to the Google Groups "Protocol Buffers" group. To post to this group, send email to protobuf@googlegroups.com. To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/protobuf?hl=en.
[protobuf] Re: Nested messages using C++
Cool!
Thank you for the clarification.
On Jan 13, 5:44 pm, Jason Hsueh wrote:
> By the way this is documented
> here:http://code.google.com/apis/protocolbuffers/docs/reference/cpp-genera...
>
> On Thu, Jan 13, 2011 at 5:42 PM, Jason Hsueh wrote:
>
> > On Thu, Jan 13, 2011 at 5:38 PM, Linus wrote:
>
> >> I am new to PB and I just ran into this. Is it possible that the PB
> >> compiler does not generate set_<...> methods for some nested messages?
>
> >> Here is an example: I don't see set_<...> methods for ANY of the
> >> parameters in the db message.
>
> > For message type fields, there is no set_<...> method. The codegen produces
> > mutable_<...> methods instead. Otherwise it is easy to write inefficient
> > code - set would require a potentially expensive copy of the message.
>
> >> What am I missing???
>
> >> package pd;
>
> >> message nv
> >> {
> >> repeated int32 Length = 1;
> >> }
>
> >> message dp
> >> {
> >> required int32 DesignID = 1;
> >> repeated double Design = 2;
> >> }
>
> >> message Shape
> >> {
> >> required int32 size_m = 1;
> >> required int32 size_n = 2;
> >> }
>
> >> message ds
> >> {
> >> required int32 DesignID = 1;
> >> repeated double data = 2;
> >> }
>
> >> message db
> >> {
> >> required nv numVars = 1;
> >> repeated dp despar = 2;
> >> required Shape db_size = 3;
> >> repeated ds dtst = 4;
> >> }
>
> >> On Jan 13, 1:48 pm, Linus wrote:
> >> > Hello,
>
> >> > I have nested messages like the following (it is a little more
> >> > complicated, but i am trying to simplify with this example).
>
> >> > package DB;
>
> >> > message Header
> >> > {
> >> > required int32 ID=1;
> >> > message param
> >> > {
> >> > required int32 size_m = 1;
> >> > required int32 size_n = 2;
> >> > }
> >> > required param p = 2;
>
> >> > }
>
> >> > message data
> >> > {
> >> > repeated double = 1;
>
> >> > }
>
> >> > message DB
> >> > {
> >> > required Header = 1;
> >> > required data = 2;
>
> >> > }
>
> >> > The problem I am having is that the accessor methods generated for the
> >> > "DB " does not have a
> >> > set_param( DB::Header::param ).
>
> >> > Is there something wrong with how I am structuring my messages? Any
> >> > help is appreciated.
> >> > Please let me know if this post is unclear and I will try and explain
> >> > myself better.
>
> >> > Thanks!
> >> > Linus
>
> >> --
> >> You received this message because you are subscribed to the Google Groups
> >> "Protocol Buffers" group.
> >> To post to this group, send email to protobuf@googlegroups.com.
> >> To unsubscribe from this group, send email to
> >> protobuf+unsubscr...@googlegroups.com
> >> .
> >> For more options, visit this group at
> >>http://groups.google.com/group/protobuf?hl=en.
--
You received this message because you are subscribed to the Google Groups
"Protocol Buffers" group.
To post to this group, send email to protobuf@googlegroups.com.
To unsubscribe from this group, send email to
protobuf+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/protobuf?hl=en.
[protobuf] Re: Nested messages using C++
I am new to PB and I just ran into this. Is it possible that the PB
compiler does not generate set_<...> methods for some nested messages?
Here is an example: I don't see set_<...> methods for ANY of the
parameters in the db message.
What am I missing???
package pd;
message nv
{
repeated int32 Length = 1;
}
message dp
{
required int32 DesignID = 1;
repeated double Design = 2;
}
message Shape
{
required int32 size_m = 1;
required int32 size_n = 2;
}
message ds
{
required int32 DesignID = 1;
repeated double data = 2;
}
message db
{
required nv numVars = 1;
repeated dp despar = 2;
required Shape db_size = 3;
repeated ds dtst = 4;
}
On Jan 13, 1:48 pm, Linus wrote:
> Hello,
>
> I have nested messages like the following (it is a little more
> complicated, but i am trying to simplify with this example).
>
> package DB;
>
> message Header
> {
> required int32 ID=1;
> message param
> {
> required int32 size_m = 1;
> required int32 size_n = 2;
> }
> required param p = 2;
>
> }
>
> message data
> {
> repeated double = 1;
>
> }
>
> message DB
> {
> required Header = 1;
> required data = 2;
>
> }
>
> The problem I am having is that the accessor methods generated for the
> "DB " does not have a
> set_param( DB::Header::param ).
>
> Is there something wrong with how I am structuring my messages? Any
> help is appreciated.
> Please let me know if this post is unclear and I will try and explain
> myself better.
>
> Thanks!
> Linus
--
You received this message because you are subscribed to the Google Groups
"Protocol Buffers" group.
To post to this group, send email to protobuf@googlegroups.com.
To unsubscribe from this group, send email to
protobuf+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/protobuf?hl=en.
[protobuf] Nested messages using C++
Hello,
I have nested messages like the following (it is a little more
complicated, but i am trying to simplify with this example).
package DB;
message Header
{
required int32 ID=1;
message param
{
required int32 size_m = 1;
required int32 size_n = 2;
}
required param p = 2;
}
message data
{
repeated double = 1;
}
message DB
{
required Header = 1;
required data = 2;
}
The problem I am having is that the accessor methods generated for the
"DB " does not have a
set_param( DB::Header::param ).
Is there something wrong with how I am structuring my messages? Any
help is appreciated.
Please let me know if this post is unclear and I will try and explain
myself better.
Thanks!
Linus
--
You received this message because you are subscribed to the Google Groups
"Protocol Buffers" group.
To post to this group, send email to protobuf@googlegroups.com.
To unsubscribe from this group, send email to
protobuf+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/protobuf?hl=en.
