Stripping down protobufs to their essence is *exactly* what I am doing with pbstream:
http://github.com/haberman/pbstream If you can hang tight for just another few weeks, I think you're going to like what you see. The streaming decoder is more or less finished: it's just over 500 lines of C99 and compiles to a ~6k object file. I'm working now on an in-memory representation, which should be also quite small. And while I don't have any benchmarks yet, I think it's going to be extremely fast, despite not doing any code generation. In the case where you're not actually copying the data into a separate structure (eg. pure streaming) I think it will likely beat the main implementation (malloc is more expensive than people realize). Josh On Feb 27, 1:55 pm, Tim <timbla...@gmail.com> wrote: > That sounds like an excellent idea! > > On Feb 27, 11:51 am, Kenton Varda <ken...@google.com> wrote: > > > Stripping out the use of file descriptors and iostreams should be easy. All > > you have to do is remove the relevant classes from > > zero_copy_stream_impl.{h,cc} and the corresponding methods from > > message.{h,cc}. The rest of the library doesn't depend on these. Note that > > protoc needs them, but presumably you will only run protoc on your build > > machines, not on the target platform itself. > > I had an idea yesterday: We could define a new superclass of > > protobuf::Message called protobuf::LeanMessage. It would only have basic > > serialization methods -- no descriptors or reflection. It could then live > > in a separate libprotobuf-lean library that contains only the I/O stuff and > > the LeanMessage interface. Then we could add an option to protoc which > > outputs code that only implements this new "lean" interface. Hopefully this > > new library would be far smaller than the current 870k libprotobuf.so. > > > On Fri, Feb 27, 2009 at 10:58 AM, Tim <timbla...@gmail.com> wrote: > > > > That's precisely the case I've been hoping to make in my current work. > > > Since it's a static library to us, the point of stripping down isn't > > > so much for reduced footprint (our linker should eliminate the unused > > > modules anyway), but instead to simplify the porting effort. > > > > One of the first things I did was to get libprotobuf 2.0.3 to "build" > > > in our environment (Freescale Codewarrior for PPC). But I did this in > > > a very incomplete way - just pulling in the missing headers from > > > Cygwin - hoping to get a feel for what the problem areas were. These > > > were the missing headers: > > > > include/ > > > _ansi.h > > > fcntl.h > > > newlib.h > > > machine/ > > > _types.h > > > ieeefp.h > > > types.h > > > sys/ > > > _types.h > > > config.h > > > fcntl.h > > > features.h > > > lock.h > > > stat.h > > > types.h > > > unistd.h > > > > We would of course have to port most of the dependencies to our OS. > > > But some of these dependencies are things our system may never > > > support. For example, we don't currently support files and streams, or > > > exceptions. Our application really only needs the data object > > > serialization functionality, and the buffer/string interface is > > > sufficient. I thought that, like protobuf-c, there might there a past > > > release that would be a better starting point for stripping down > > > libprotobuf. But from what I can tell, it looks like even the initial > > > release supports files and streams. Pretty basic stuff in most > > > environments. I realize we are an exception here. > > > > After briefly evaluating protobuf and protobuf-c, I can see that we > > > would have to heavily modify either one to achieve our goal. Given > > > this, I would like to lean towards stripping/modifying the c++ > > > version, since our app is largely in c++ and this language might > > > facilitate our design modifications/extensions. Might you have any > > > recommendations or comments on an approach for stripping protobuf of > > > file and stream support? > > > > Given our simple requirements and the perceived work involved with > > > stripping libprotobuf, however, my instinct tells me that protobuf- > > > c-0.7 would be a better starting point. It already is "stripped". I'm > > > guessing we could add the functionality we need there quicker than we > > > could strip down from protobuf. Maybe even rewrite it in c++. It's > > > only 3 files! > > > > Comments, anyone? > > > > On Feb 26, 5:41 pm, Kenton Varda <ken...@google.com> wrote: > > > > I've tested in on OSX, FreeBSD, and Solaris, but those are not terribly > > > > different from Linux. The Google implementations of protobufs > > > > definitely > > > > weren't designed for embedded systems, but there might be a case for > > > taking > > > > the Google C++ implementation and ripping out all the "advanced" > > > > features > > > to > > > > make something extremely stripped-down. > > > > > On Thu, Feb 26, 2009 at 5:27 PM, Tim <timbla...@gmail.com> wrote: > > > > > > Anyone out there using GPB on an OS other than Linux or Windows? I've > > > > > been evaluating GPB for use in an embedded OS and at the moment am a > > > > > bit stuck between protobuf and protobuf-c. We would be using GPB for > > > > > simple data storage and serialization, and don't need RPC or even > > > > > Reflection. I checked out protobuf-c (0.7 is easiest to port, bc it's > > > > > before RPC was added) and this is definitely at the opposite end of > > > > > the spectrum. We would need to add quite a bit of functionality to it. --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Protocol Buffers" group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~----------~----~----~----~------~----~------~--~---
