> The 1600MB number looks like record-level compression. Are you sure
> you've turned on block compression?
Here's the exact snippet from my shell script. Do I have to set these
configuration parameters directly in the hadoop configuration file:
${HIVE_COMMAND} -e "set hive.exec.compress.output=true; set
io.seqfile.compression.type=BLOCK; set
mapred.output.compression.codec=org.apache.hadoop.io.compress.GzipCodec; set
io.seqfile.compress.blocksize=50000000; insert overwrite table
raw_compressed partition(dt='${D}') select line from raw where dt='${D}'"
Saurabh.
--
http://nandz.blogspot.com
http://foodieforlife.blogspot.com
