----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/935/#review875 -----------------------------------------------------------
Can you make sure that in the test cases, the query need the merge step? - Siying On 2011-06-20 19:20:53, Franklin Hu wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/935/ > ----------------------------------------------------------- > > (Updated 2011-06-20 19:20:53) > > > Review request for hive. > > > Summary > ------- > > For a table stored as RCFile, intermediate results are sometimes merged if > those files are below a certain threshold. For RCFiles, we can do a block > level merge that does not deserialize the blocks and is more efficient. This > patch leverages the existing code used to merge for ALTER TABLE ... > CONCATENATE. > > > This addresses bug HIVE-2035. > https://issues.apache.org/jira/browse/HIVE-2035 > > > Diffs > ----- > > trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 1136090 > trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/TaskFactory.java 1136090 > > trunk/ql/src/java/org/apache/hadoop/hive/ql/io/rcfile/merge/BlockMergeTask.java > 1136090 > trunk/ql/src/java/org/apache/hadoop/hive/ql/io/rcfile/merge/MergeWork.java > 1136090 > > trunk/ql/src/java/org/apache/hadoop/hive/ql/io/rcfile/merge/RCFileBlockMergeRecordReader.java > 1136090 > > trunk/ql/src/java/org/apache/hadoop/hive/ql/io/rcfile/merge/RCFileKeyBufferWrapper.java > 1136090 > > trunk/ql/src/java/org/apache/hadoop/hive/ql/io/rcfile/merge/RCFileMergeMapper.java > 1136090 > trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMRFileSink1.java > 1136090 > > trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/ConditionalResolverMergeFiles.java > 1136090 > trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/MapredWork.java 1136090 > trunk/ql/src/test/queries/clientpositive/rcfile_createas1.q PRE-CREATION > trunk/ql/src/test/queries/clientpositive/rcfile_merge1.q PRE-CREATION > trunk/ql/src/test/queries/clientpositive/rcfile_merge2.q PRE-CREATION > trunk/ql/src/test/queries/clientpositive/rcfile_merge3.q PRE-CREATION > trunk/ql/src/test/queries/clientpositive/rcfile_merge4.q PRE-CREATION > trunk/ql/src/test/results/clientpositive/rcfile_createas1.q.out > PRE-CREATION > trunk/ql/src/test/results/clientpositive/rcfile_merge1.q.out PRE-CREATION > trunk/ql/src/test/results/clientpositive/rcfile_merge2.q.out PRE-CREATION > trunk/ql/src/test/results/clientpositive/rcfile_merge3.q.out PRE-CREATION > trunk/ql/src/test/results/clientpositive/rcfile_merge4.q.out PRE-CREATION > > Diff: https://reviews.apache.org/r/935/diff > > > Testing > ------- > > > Thanks, > > Franklin > >
