On Sun, Mar 1, 2009 at 2:12 PM, Joshua Haberman <jhaber...@gmail.com> wrote:

>
> Stripping down protobufs to their essence is *exactly* what I am doing
> with pbstream:
>
> http://github.com/haberman/pbstream
>
> If you can hang tight for just another few weeks, I think you're going
> to like what you see.  The streaming decoder is more or less finished:
> it's just over 500 lines of C99 and compiles to a ~6k object file.
> I'm working now on an in-memory representation, which should be also
> quite small.  And while I don't have any benchmarks yet, I think it's
> going to be extremely fast, despite not doing any code generation.  In
> the case where you're not actually copying the data into a separate
> structure (eg. pure streaming) I think it will likely beat the main
> implementation (malloc is more expensive than people realize).


Note that the C++ implementation only mallocs the *first* time you use an
object.  If you reuse an object, it will reuse memory, only allocating new
memory for parts of the message that weren't used before.  In practice, most
performance-critical software reuses protobuf objects, so it's only fair to
compare your implementation against this case.

Be careful about claiming that your code will be faster before actually
running benchmarks.  I've been burned many times doing that.  :)


>
>
> Josh
>
> On Feb 27, 1:55 pm, Tim <timbla...@gmail.com> wrote:
> > That sounds like an excellent idea!
> >
> > On Feb 27, 11:51 am, Kenton Varda <ken...@google.com> wrote:
> >
> > > Stripping out the use of file descriptors and iostreams should be easy.
>  All
> > > you have to do is remove the relevant classes from
> > > zero_copy_stream_impl.{h,cc} and the corresponding methods from
> > > message.{h,cc}.  The rest of the library doesn't depend on these.  Note
> that
> > > protoc needs them, but presumably you will only run protoc on your
> build
> > > machines, not on the target platform itself.
> > > I had an idea yesterday:  We could define a new superclass of
> > > protobuf::Message called protobuf::LeanMessage.  It would only have
> basic
> > > serialization methods -- no descriptors or reflection.  It could then
> live
> > > in a separate libprotobuf-lean library that contains only the I/O stuff
> and
> > > the LeanMessage interface.  Then we could add an option to protoc which
> > > outputs code that only implements this new "lean" interface.  Hopefully
> this
> > > new library would be far smaller than the current 870k libprotobuf.so.
> >
> > > On Fri, Feb 27, 2009 at 10:58 AM, Tim <timbla...@gmail.com> wrote:
> >
> > > > That's precisely the case I've been hoping to make in my current
> work.
> > > > Since it's a static library to us, the point of stripping down isn't
> > > > so much for reduced footprint (our linker should eliminate the unused
> > > > modules anyway), but instead to simplify the porting effort.
> >
> > > > One of the first things I did was to get libprotobuf 2.0.3 to "build"
> > > > in our environment (Freescale Codewarrior for PPC). But I did this in
> > > > a very incomplete way - just pulling in the missing headers from
> > > > Cygwin - hoping to get a feel for what the problem areas were. These
> > > > were the missing headers:
> >
> > > > include/
> > > >  _ansi.h
> > > >  fcntl.h
> > > >  newlib.h
> > > >  machine/
> > > >    _types.h
> > > >    ieeefp.h
> > > >    types.h
> > > >  sys/
> > > >    _types.h
> > > >    config.h
> > > >    fcntl.h
> > > >    features.h
> > > >    lock.h
> > > >    stat.h
> > > >    types.h
> > > >    unistd.h
> >
> > > > We would of course have to port most of the dependencies to our OS.
> > > > But some of these dependencies are things our system may never
> > > > support. For example, we don't currently support files and streams,
> or
> > > > exceptions. Our application really only needs the data object
> > > > serialization functionality, and the buffer/string interface is
> > > > sufficient. I thought that, like protobuf-c, there might there a past
> > > > release that would be a better starting point for stripping down
> > > > libprotobuf. But from what I can tell, it looks like even the initial
> > > > release supports files and streams. Pretty basic stuff in most
> > > > environments. I realize we are an exception here.
> >
> > > > After briefly evaluating protobuf and protobuf-c, I can see that we
> > > > would have to heavily modify either one to achieve our goal. Given
> > > > this, I would like to lean towards stripping/modifying the c++
> > > > version, since our app is largely in c++ and this language might
> > > > facilitate our design modifications/extensions. Might you have any
> > > > recommendations or comments on an approach for stripping protobuf of
> > > > file and stream support?
> >
> > > > Given our simple requirements and the perceived work involved with
> > > > stripping libprotobuf, however, my instinct tells me that protobuf-
> > > > c-0.7 would be a better starting point. It already is "stripped". I'm
> > > > guessing we could add the functionality we need there quicker than we
> > > > could strip down from protobuf. Maybe even rewrite it in c++. It's
> > > > only 3 files!
> >
> > > > Comments, anyone?
> >
> > > > On Feb 26, 5:41 pm, Kenton Varda <ken...@google.com> wrote:
> > > > > I've tested in on OSX, FreeBSD, and Solaris, but those are not
> terribly
> > > > > different from Linux.  The Google implementations of protobufs
> definitely
> > > > > weren't designed for embedded systems, but there might be a case
> for
> > > > taking
> > > > > the Google C++ implementation and ripping out all the "advanced"
> features
> > > > to
> > > > > make something extremely stripped-down.
> >
> > > > > On Thu, Feb 26, 2009 at 5:27 PM, Tim <timbla...@gmail.com> wrote:
> >
> > > > > > Anyone out there using GPB on an OS other than Linux or Windows?
> I've
> > > > > > been evaluating GPB for use in an embedded OS and at the moment
> am a
> > > > > > bit stuck between protobuf and protobuf-c. We would be using GPB
> for
> > > > > > simple data storage and serialization, and don't need RPC or even
> > > > > > Reflection. I checked out protobuf-c (0.7 is easiest to port, bc
> it's
> > > > > > before RPC was added) and this is definitely at the opposite end
> of
> > > > > > the spectrum. We would need to add quite a bit of functionality
> to it.
> >
>

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to 
protobuf+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en
-~----------~----~----~----~------~----~------~--~---

Reply via email to