Hi,

 This is exactly what I've done before putting arrays into a string.
 When I've implemented arrays via repeated fields, the program was
even slower,
 and the file size was too large (compare to Java serialization
mechanism+ zip).
 This is why I've moved  my array into a  string, thinking that there
 will be no significant overhead storing such object. I guess, each
repeated
 filed has some used additional bits to store them

 Yes, I used [packed=true] for "double" field. I did not check what
 will happen after removing at (probably, the file size will be even
bigger!!)


 cheers, Sergei

On Oct 8, 10:48 am, Marc Gravell <marc.grav...@gmail.com> wrote:
> For basic types, you can also use packed encoding to reduce the space
> required; just add [packed=true] to a "repeated" element.
>
> Marc
>
> On Oct 8, 4:46 pm, Constantinos Michael <constanti...@google.com>
> wrote:
>
> > On Thu, Oct 8, 2009 at 5:35 PM, sergei175 <sergei...@googlemail.com> wrote:
>
> > >  Hi,
>
> > >  I've looked at protocol buffers, and I've noted that there is no
> > > support for arrays
> > >  of values (double, integers). This is a significant drawback, for
> > > example
> > >  JSOM, HDF5 etc they all have this.
>
> > Have you looked at "repeated" fields? You can define one like so:
>
> > repeated double my_number = 1;
>
> > >  One post suggested  that one should put an array  as one single
> > > string in a field
> > >  I've did this, and the performance  was very bad in Java and very
> > > memory consuming
> > >  (compare to the standard Java serialization).
> > >  I've wrote 500 times the same array (10,000 double numbers ), and
> > > after the array 500,
> > >  my computer was out of memory,
>
> > >  Secondly, all tutorials suggest that the file should be written at
> > > once, i.e. at the end
> > >  of the program, when the messages
> > >  are filled. I want to write data to the disk in several steps. say I
> > > want to write one record first (say, one array),
> > >  then I want to append data to the existing file, and so on, this way
> > > I will not need
> > >  to keep all records in the computer memory. The merge mechanism
> > > shown in the tutorial
> > >  seems parses the old file
> > >  first, and then add new record, and write a new file.
>
> > >  Do I understand this correctly? If yes, then the protocol  buffers is
> > > not too good for large data volumes,
> > >  especially with numerical arrays
>
> > >  best wishes, Sergei
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to 
protobuf+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en
-~----------~----~----~----~------~----~------~--~---

Reply via email to