Re: Avro file Compression

2013-08-22 Thread Scott Carey
objects carefully so that neighbor records have more in common. From: Bill Baird Reply-To: "user@avro.apache.org" Date: Thursday, August 22, 2013 7:47 AM To: "user@avro.apache.org" Subject: Re: Avro file Compression As with any compression, how much you get depends on

Re: Avro file Compression

2013-08-22 Thread Bill Baird
As with any compression, how much you get depends on the size and nature of the data. I have objects where unserialized they take 4 or 5k, and they serialize to 1.5 to 3k, or about 2 to 1. However, for the same object structure (which contains several nested arrays ... lots of strings, numbers ..

Re: Avro file Compression

2013-08-21 Thread Harsh J
Can you share your test? There is an example at http://svn.apache.org/repos/asf/avro/trunk/lang/c/examples/quickstop.c which has the right calls for using a file writer with a deflate codec - is yours similar? On Mon, Aug 19, 2013 at 9:42 PM, amit nanda wrote: > I am try to compress the avro file

Avro file Compression

2013-08-19 Thread amit nanda
I am try to compress the avro files that i am writing, for that i am using the latest Avro C, with "deflate" option, but i am not able to see any difference in the file size. Is there any special type to data that this works on, or is there any more setting that needs to be done for this to work.