I think there is a bit of ambiguity in what you said. I think what you mean is by "can be optionally compressed..." is that the TRecordStream itself will do the compression if you ask, not that you can do it for yourself.
Correct? On Thu, Sep 4, 2008 at 11:46 AM, Pete Wyckoff <[EMAIL PROTECTED]> wrote: > > I'll just give another plug for Thrift's TRecordStream which has fixed > sized > frames that can be optionally compressed or checksummed; since the frames > are fixed sized, it can be split on frame boundaries. > > You can write whatever data you want with it - it doesn't have to be > thrift, > it just takes whatever is written and writes it to a FD or a socket or > whatever. > > There is the issue of spill over between frames just like the sequence file > case. > > -- pete > > > On 9/4/08 11:32 AM, "Ted Dunning" <[EMAIL PROTECTED]> wrote: > > > On Thu, Sep 4, 2008 at 10:51 AM, Owen O'Malley <[EMAIL PROTECTED]> > wrote: > > > >> ... > >> It is also not splittable. It would be really nice to have a codec that > was > >> similar in compression/cpu cost to gzip that was splittable. > >> > > > > Indeed. > > > > What happened to the effort to build a splittable gzip codec by inserting > > dummy compression resets with a known pattern? > > > > -- ted
