Ah, I was just talking about the size of the .jar files here, which I had
thought was what you were worried about.  The runtime memory use could be
much larger; I'm not sure.  In the non-lite runtime, descriptors are
constructed at startup which take space.  The lite library does not allocate
any permanent objects.  Either way, though, I'd expect the descriptors to
take very little memory.  You call your interface "large" but to me it
sounds small.  :)
But that said, I suspect that the runtime memory usage of your existing
system has more to do with the RPC implementation than with the message
objects.  Since you're proposing writing a new RPC implementation to use
with protobufs, it's hard to say how much memory it could end up using.

On Wed, Aug 12, 2009 at 3:27 PM, Noel Fegan <nfe...@gmail.com> wrote:

> We have a UI app (client) that talks to a locally running process
> (localhost) which I am calling the server. So, we're using SOAP for
> IPC so it's not "remote", but it is an invocation of an out-of-process
> functionality. As such both processes are running on the client
> machine. We have a number of clients apps that may be running at the
> same time for this user, so we encapsulated the common functionality
> in a single app that these clients use under-the-hood. Also, the
> background process connects downstream to singular network services
> that it consolidates multiplexing the service on behalf of the
> clients.
> 300KB doesn't sound too bad. Our current Adapter layer (IPC layer of
> our server app) comes in at around 25MB-30MB (Java heap space). Note:
> that's the complete SOAP stack: Web Service, SOAP engine, JAXB. Apart
> from the SOAP generated class our own code in this layer merely maps
> equivalent "core" classes to SOAP generated types and methods. We
> don't figure our code is adding much, so we're assuming most of this
> is down to some part of SOAP.
> Anyway, thanks for the quick response. I might start by trying to
> produce proto file for a portion of our interface, and see how that
> compares.
> You don't happen to have a WSDL to proto file converter by any
> chance? ;-)
> On Aug 12, 10:56 pm, Kenton Varda <ken...@google.com> wrote:
> > Well there's good news and bad news.  The bad news is that using protocol
> > buffers has been known to lead to large code footprints.  The good news
> is
> > that it should be fairly easy to estimate how much it will bloat *your*
> > binary, without having to actually implement the whole system.
> > Protocol buffers involves both generated code and a runtime library.
>  Both
> > will be factors in your code footprint.  You have several options for
> > trading off between speed, code size, and features, as follows:
> >
> > * With optimize_for = SPEED (the default), generated code is large (how
> > large depends on the number of types and fields in your protocol), and
> the
> > runtime library is a 313kB jar file, but you get insanely high throughput
> > and all features including reflection.
> >
> > * With optimize_for = CODE_SIZE, you use the same (large) runtime library
> > and get all the same features, but the generated code is much smaller as
> it
> > falls back to shared reflection-based implementations of many methods.
>  In
> > C++ the generated code is about 40% of the regular size; I don't have
> > precise numbers for Java.  Reflection-based operations tend to be 4x-10x
> > slower than generated code (probably similar in performance to XML).
> >
> > * With optimize_for = LITE_RUNTIME (new in v2.2.0), you get fast (but
> large)
> > generated code, but it depends on a much smaller runtime library.
> >  libprotobuf-lite.jar is only 55kB.  The down side is that features like
> > descriptions and reflection are not available (which is why you can't
> > optimize both for code size and lite runtime at the same time -- the code
> > size optimizations rely on features that aren't in the lite runtime).
>  Note
> > that service definitions also aren't supported in lite mode, but they
> > actually don't provide all that much functionality (since we don't
> include
> > an RPC implementation).
> >
> > Both C++ and Java support these options.
> >
> > What I'd suggest you do is write out the .proto file(s) for your
> interface,
> > then compile them in each mode and see how big the jars end up being.
>  This
> > should not take very long and should answer your question.
> >
> > I'm a little surprised that you seem more worried about the footprint of
> the
> > server than the client.  The usual experience at Google is that we're
> happy
> > to let server programs balloon into gigantic monsters, sometimes hundreds
> of
> > MB in size, but we want to keep client programs small because users have
> to
> > download them.
> >
> >
> >
> > On Wed, Aug 12, 2009 at 1:58 PM, Noel Fegan <nfe...@gmail.com> wrote:
> >
> > > Hi,
> >
> > > This might be a "how long is a piece of string"-type question, but I
> > > was wondering if there's any way for me to know how much protocol
> > > buffers might add to my app's memory footprint, when I have a large
> > > number of messages. I see there's a LITE option which sounds like the
> > > mode I would need to be working in? I'm assuming this affects the code
> > > generational aspects of PB, ie the classes it generates for me. Is
> > > there a runtime engine for PB?
> >
> > > The context of my question, is that I am currently working on a
> > > project that uses SOAP as an IPC mechanism. I know, the decision to
> > > use SOAP pre-dates my involvement, but our interface is quite large,
> > > e.g. 7 main services each exposing say 10 functions, and the "client
> > > end" is also a SOAP server for async callback, so there's an API in
> > > both directions so-to-speak. Server end is a Java app. Between SOAP,
> > > jetty, the generated classes etc, this IPC layer of the server end of
> > > our app represents about a 1/3 of the runtime memory footprint of this
> > > server process. For us, whatever we replace SOAP with would have to
> > > have a much smaller memory footprint, rather than necessarily being
> > > much faster.
> >
> > > I know Protocol Buffers doesn't include an RPC implementation and
> > > merely provides stubs, but I could imagine we could produce a fairly
> > > light-weight RPC implementation, but I'm more concerned that we could
> > > go down this road and find that using Protocol Buffers will also
> > > produce a large memory footprint, really because our API is large. I'm
> > > not sure if this gives enough context to help anyone answer my
> > > question, but I would be interested even in understanding a little
> > > more about how I might go about working this out for myself? Any
> > > pointers would be appreciated.
> >
> > > Thanks.
> > > Noel
> >

You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to 
For more options, visit this group at 

Reply via email to