Re: [protobuf] Re: Protocol Buffers using Lzip

2009-12-11 Thread Jacob Rief
Hello Chris, 2009/12/10 Christopher Smith cbsm...@gmail.com: One compression algo that I thought would be particularly useful with PB's would be LZO. It lines up nicely with PB's goals of being fast and compact. Have you thought about allowing an integrated LZO stream? --Chris My goal is to

Re: [protobuf] Re: Protocol Buffers using Lzip

2009-12-11 Thread Daniel Wright
As I'm sure you can imagine, we store a lot of data in protocol buffer format at Google, so we often want to store very large files with many serialized protocol buffers. The technique we use is to batch a bunch of records together, compress them and write the compressed block to the file (with

Re: [protobuf] Re: Protocol Buffers using Lzip

2009-12-10 Thread Christopher Smith
One compression algo that I thought would be particularly useful with PB's would be LZO. It lines up nicely with PB's goals of being fast and compact. Have you thought about allowing an integrated LZO stream? --Chris On Wed, Dec 9, 2009 at 12:21 PM, Kenton Varda ken...@google.com wrote: Thanks