GZIPOutputStream (for C++)

2009-03-01 Thread bolson
Has anyone written a ZeroCopyOutputStream that compresses its output with zlib on the way out? (Ideally there'd be a matching GZIPInputStream.) This is of course trivial in Java, but my project is in C++. I recently noticed that one of my protobuf-data-files still benefitted from gzip at a ratio

Re: GPB on non-Linux, non-Windows OS?

2009-03-01 Thread Joshua Haberman
Stripping down protobufs to their essence is *exactly* what I am doing with pbstream: http://github.com/haberman/pbstream If you can hang tight for just another few weeks, I think you're going to like what you see. The streaming decoder is more or less finished: it's just over 500 lines of C99

Re: GZIPOutputStream (for C++)

2009-03-01 Thread Kenton Varda
On Sun, Mar 1, 2009 at 2:01 PM, bolson brian.ol...@gmail.com wrote: I recently noticed that one of my protobuf-data-files still benefitted from gzip at a ratio slightly better than 7 to 1. Just to note, this is not surprising. PB is a compact format but does not actual compression, so if you

Re: GPB on non-Linux, non-Windows OS?

2009-03-01 Thread Kenton Varda
On Sun, Mar 1, 2009 at 3:42 PM, Joshua Haberman jhaber...@gmail.com wrote: What about strings and repeated elements? String assignment will at least sometimes call malloc() to re-allocate the string, though I don't know enough about STL to know how often this happens. And string assignment

Re: GPB on non-Linux, non-Windows OS?

2009-03-01 Thread Kenton Varda
On Sun, Mar 1, 2009 at 6:56 PM, Kenton Varda ken...@google.com wrote: I believe GCC's std::string will only malloc if it needs to grow, so on reuse there should be no new allocation. I meant, reuse to store a string of the same or lesser size, of course.

Re: GZIPOutputStream (for C++)

2009-03-01 Thread Mark Assad
Hi, I've implemented something like this (attached to this email). You'll probably have to do a bit of work to get it working with what you want to do. The InputStream takes as input another ZeroCopyInput Stream. Unfortunatly, the output stream hasn't been implemented this way, instead it

Textmate bundle for protobufs

2009-03-01 Thread Kevin Weil
I made a syntax highlighting bundle for editing .proto files in Textmate. It's pretty rough, but it's helped me, so I thought others might like it as well. Feedback is welcome, and feel free to improve it too. http://github.com/kevinweil/protobuf.tmbundle Thanks, Kevin