Using something like Fastutil <http://fastutil.dsi.unimi.it/> or the collections library posted would definitely improve performance. Google guys/committers - what do you think about adding a dependency on these libraries? I don't mind contributing code for this.
I still think a huge performance increase could be seen if we could preallocate the ArrayList in the Builder. Looking at generated code for a simple repeated int32 type, whenever an item is added to the list ArrayLists's append is called. If we exposed ArrayList's ensureCapacity method, we could reserve the capacity we needed and wouldn't have to reallocate objects and copy over as we build up the List. 2010/6/23 Marc-André Laverdière <marcandre.laverdi...@gmail.com> > There is an Apache Commons project that provide the primitive lists: > http://commons.apache.org/primitives/ > > I think that those would do the heavy lifting for us :) > > Marc-André LAVERDIÈRE > "Perseverance must finish its work so that you may be mature and > complete, not lacking anything." -James 1:4 > mlaverd.theunixplace.com/blog > > /"\ > \ / ASCII Ribbon Campaign > X against HTML e-mail > / \ > > > > 2010/6/24 Kenton Varda <ken...@google.com>: > > This is a use case that is, unfortunately, not well-optimized by the Java > > implementation. What's needed is versions of ArrayList which store > unboxed > > values. Unfortunately, a separate such implementation would be needed > for > > every primitive field type, since Java generics work only with boxed > types. > > It will be tedious, but you're welcome to work on it. > > > > On Tue, Jun 22, 2010 at 7:16 PM, Joshua Hartman <atomicfo...@gmail.com> > > wrote: > >> > >> Hi, > >> > >> I'm trying to serialize a large primitive array of int in Java, and > >> I'm testing using 1,000,000 elements. My .proto message is just a > >> packed repeated field of type int32. To serialize, I just create a > >> builder and loop through the array, adding elements one at a time. I > >> took a peek at the underlying implementation and it seems that it uses > >> an ArrayList. However, there's no way to reserve the size of the > >> ArrayList, so serializing is slower than it could be. In fact, it's > >> faster for me to just serialize the data an ObjectOutputStream to put > >> the data in a protocol buffer bytes message. > >> > >> Is there a way for me to do the serialization faster? I took a peek > >> through the API but didn't see anything obvious. I saw > >> CodedOutputStream as well, but I didn't understand the intent of that > >> class. Could anyone please explain what it's used for? > >> > >> Thanks, > >> Josh H > >> > >> -- > >> You received this message because you are subscribed to the Google > Groups > >> "Protocol Buffers" group. > >> To post to this group, send email to proto...@googlegroups.com. > >> To unsubscribe from this group, send email to > >> protobuf+unsubscr...@googlegroups.com<protobuf%2bunsubscr...@googlegroups.com> > . > >> For more options, visit this group at > >> http://groups.google.com/group/protobuf?hl=en. > >> > > > > -- > > You received this message because you are subscribed to the Google Groups > > "Protocol Buffers" group. > > To post to this group, send email to proto...@googlegroups.com. > > To unsubscribe from this group, send email to > > protobuf+unsubscr...@googlegroups.com<protobuf%2bunsubscr...@googlegroups.com> > . > > For more options, visit this group at > > http://groups.google.com/group/protobuf?hl=en. > > > > -- > You received this message because you are subscribed to the Google Groups > "Protocol Buffers" group. > To post to this group, send email to proto...@googlegroups.com. > To unsubscribe from this group, send email to > protobuf+unsubscr...@googlegroups.com<protobuf%2bunsubscr...@googlegroups.com> > . > For more options, visit this group at > http://groups.google.com/group/protobuf?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Protocol Buffers" group. To post to this group, send email to proto...@googlegroups.com. To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/protobuf?hl=en.
