Note: I apparently do not have permission to post to hypergraphdb. On Sun, Apr 11, 2010 at 11:17 PM, Kenton Varda <[email protected]> wrote:
> I'm a bit confused about how constructing a builder could be as slow as you > describe it. It's just an object allocation. Granted, two objects need to > be allocated instead of just one, due to the builder design, but object > allocations aren't very expensive. Can you provide more detail on what you > found slow? A realistic benchmark demonstrating the problem would be ideal. > > The independent benchmarks done by the thrift-protobuf-compare project show > that yes, protobuf object creation is slower than others, but the difference > is so small compared to parsing and serialization time that protobuf ends up > near the front of the pack in "total time": > > http://code.google.com/p/thrift-protobuf-compare/wiki/Benchmarking > > > On Sun, Apr 11, 2010 at 4:57 PM, David Hart <[email protected]> wrote: > >> Hi All, >> >> Please cross-post all (any) followups to this thread. Now, please see >> this: >> >> On 12/04/2010, at 8:19, Drew Botwinick <[email protected]> wrote: >> >> I'm gonna go a bit off-topic and I apologize in advance for doing >>> that. Protocol Buffers are probably not the best match here as Boris >>> has already identified. I figured I'd share my other experience/ >>> concerns with java protobuf here as a sidenote. >>> >>> Google Protocol Buffers are actually a bit disappointing in Java. I've >>> run some tests with protobuf and found that although the performance >>> was great in most respects... the amount of time it takes to generate >>> a message using the "Message.Builder" is painfully slow. It takes way >>> too long to instantiate a Message.Builder and there is no point in >>> profiling the issue away because as soon as u make any changes to >>> your .proto files... you'll compile that into java classes and undo >>> your optimizations. I could only come up with 3 ways to fix protobuf: >>> >>> 1) find the source of the issue once and for all and commit the fixes >>> to the protobuf project so that from now on... all java generated >>> source is improved [which is not exactly trivial... I didn't spend >>> very long looking for the issue, but it certainly wasn't obvious] >>> 2) try to arrange message builders into some kind of work queue so >>> that you can re-use them (which violates their designed functionality >>> and leads to all sorts of ugliness) >>> 3) use python or c instead of java since the issues are java specific. >>> >>> I think protobuf is great... especially since it's well supported, >>> well designed, and bindings are available for many languages and >>> platforms, but it has some quirky java performance issues that don't >>> make it the best choice (in java) if performance is your number one >>> concern. If you're willing to sacrifice a few dozen milliseconds here >>> and there, it's a great solution for an extensible cross-platform >>> messaging format. >>> >>> -Drew >>> >>> On Apr 11, 5:51 am, David Hart <[email protected]> wrote: >>> >>>> Can Google's 'Protocol Buffers', while written in C, be of any >>>> guidance, inspiration or use? Presumably, Google (or someone else) >>>> have written Java wrappers for PB? >>>> >>>> -Dave >>>> >>>> David Hart | phone +61 419 483 263 | sent from iPhone >>>> >>>> On 11/04/2010, at 8:51, Drew Botwinick <[email protected]> >>>> wrote: >>>> >>>> >>>> >>>> Hi all, >>>>> >>>> >>>> I'm new to the project and I'm looking to implement a bunch of new >>>>> features as well as fix existing issues. >>>>> >>>> >>>> I have an idea for an approach (but not a complete solution) to >>>>> resolving issue #21 which is that ObjectInputStream and >>>>> ObjectOutputStream cannot be configured to use a different class >>>>> loader than the original class loader of HGDB. My proposed solution is >>>>> to use "serialization containers". It's a bit of a "hack" solution and >>>>> is sort of half way between providing custom serialization and using >>>>> java serialization (because it can do both). I'll explain in more >>>>> detail below. >>>>> >>>> >>>> The idea is simple. The SerializableType should force the inclusion of >>>>> a writeReplace() method. The writeReplace method should replace the >>>>> type to be serialized with a "serialization container" with an >>>>> interface such as: >>>>> >>>> >>>> public interface SerializedDataContainer<T> extends Externalizable { >>>>> public Object readResolve() throws ObjectStreamException; >>>>> >>>> >>>> public void writeExternal(ObjectOutput out) throws IOException; >>>>> >>>> >>>> public void readExternal(ObjectInput in) throws IOException, >>>>> ClassNotFoundException; >>>>> } >>>>> >>>> >>>> The reason for choosing Externalizable is so that the container can >>>>> dictate either using the standard java serialization or any custom >>>>> serialization. readResolve() should construct the object using >>>>> whatever class loader is appropriate. The downside of this option is >>>>> that by the time you implement the container scheme... you might as >>>>> well be providing a method of serialization other than vanilla java >>>>> serialization anyway. I've used the above container scheme >>>>> successfully elsewhere, but I originally used this approach so that I >>>>> could easily substitute alternate serialization mechanisms into code >>>>> originally designed to be used with java serialization. (My data >>>>> container had 1 private field that was a byte[]). >>>>> >>>> >>>> As I see it, it's very difficult to manipulate the vanilla java >>>>> serialization scheme to use alternate class loaders, and this, >>>>> although it's a roundabout solution, does provide a method of choosing >>>>> your class loader by some other means; however, it'd be hard to >>>>> generalize... it'd be very application specific...and at worst... very >>>>> type specific (which at that point... why not just have an alternate >>>>> serialization mechanism anyway?) >>>>> >>>> >>>> Thoughts? Comments? Concerns? Any feedback is appreciated. :-) >>>>> >>>> >>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "HyperGraphDB" group. >>>>> To post to this group, send email to [email protected]. >>>>> To unsubscribe from this group, send email to >>>>> [email protected]<hypergraphdb%[email protected]> >>>>> . >>>>> For more options, visit this group athttp:// >>>>> groups.google.com/group/hypergraphdb?hl=en >>>>> . >>>>> >>>> >>> -- >>> You received this message because you are subscribed to the Google Groups >>> "HyperGraphDB" group. >>> To post to this group, send email to [email protected]. >>> To unsubscribe from this group, send email to >>> [email protected]<hypergraphdb%[email protected]> >>> . >>> For more options, visit this group at >>> http://groups.google.com/group/hypergraphdb?hl=en. >>> >>> >> -- >> You received this message because you are subscribed to the Google Groups >> "Protocol Buffers" group. >> To post to this group, send email to [email protected]. >> To unsubscribe from this group, send email to >> [email protected]<protobuf%[email protected]> >> . >> For more options, visit this group at >> http://groups.google.com/group/protobuf?hl=en. >> >> > -- You received this message because you are subscribed to the Google Groups "Protocol Buffers" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/protobuf?hl=en.
