On May 7, 2010, at 18:54 , Kenton Varda wrote:
I'd be very interested to hear why the JDK is not optimal here.
I dug into this. I *think* the problem is that the JDK ends up allocating a huge temporary array for the UTF-8 data. Hence, the garbage collection cost is higher for the JDK's implementation, rather than my implementation. Basically the code does this:
* allocate a new byte[] array that is string length * max bytes per character ( = 4 for the UTF encoder) * use the java.nio.charset.CharsetEncoder to encode the char[] into the byte[] (wrapped in CharBuffer / ByteBuffer). * copy the exact number of bytes out of the byte[] into a new byte[], and return that.
The only "trick" the JDK gets to use that "normal" Java code can't is that they can access the string's char[] buffer directly, whereas I need to copy it out into a char[] array.
Hence, I think what is happening is that the JDK allocates 4-5 times as much memory per encode than I do. In the cases where the data is ASCII, my code is faster, since it allocates exactly the right amount of space, and doesn't need an extra copy. When the data is not ASCII, my code may still be faster, since it doesn't overallocate quite as much (in exchange my code does many copies).
Conclusion: there is a legitimate reason for this code to be faster than the JDK's code. But it still may not be worth including this patch in the main line protocol buffer code.
Evan -- Evan Jones http://evanjones.ca/ -- You received this message because you are subscribed to the Google Groups "Protocol Buffers" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/protobuf?hl=en.
