-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/24792/
-----------------------------------------------------------
Review request for hive.
Bugs: HIVE-4788
https://issues.apache.org/jira/browse/HIVE-4788
Repository: hive-git
Description
-------
The issue is that Bzip2 compressed rcfile data is encountering an error when
being queried even the most simple query "select *". The issue is easily
reproducible using the following.
Create a table and load the sample data below.
DDL: create table source_data (a string, b string) row format delimited fields
terminated by ',';
Sample data:
apple,sauce
Test:
Do the following and you should receive the error listed below for the rcfile
table with bz2 compression.
create table rc_nobz2 (a string, b string) stored as rcfile;
insert into table rc_nobz2 select * from source_txt;
SET io.seqfile.compression.type=BLOCK;
SET hive.exec.compress.output=true;
SET mapred.compress.map.output=true;
SET mapred.output.compress=true;
SET mapred.output.compression.codec=org.apache.hadoop.io.compress.BZip2Codec;
create table rc_bz2 (a string, b string) stored as rcfile;
insert into table rc_bz2 select * from source_txt;
hive> select * from rc_bz2;
Failed with exception java.io.IOException:java.io.IOException: Stream is not
BZip2 formatted: expected 'h' as first byte but got 'ï¿¿'
hive> select * from rc_nobz2;
apple sauce
Diffs
-----
ql/src/java/org/apache/hadoop/hive/ql/io/RCFile.java 2a27676
ql/src/test/queries/clientpositive/rcfile_compress.q PRE-CREATION
ql/src/test/results/clientpositive/rcfile_compress.q.out PRE-CREATION
Diff: https://reviews.apache.org/r/24792/diff/
Testing
-------
Thanks,
Navis Ryu
