----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/14221/#review26284 -----------------------------------------------------------
ql/src/java/org/apache/hadoop/hive/ql/exec/SMBMapJoinOperator.java <https://reviews.apache.org/r/14221/#comment51360> removed in diff3 ql/src/java/org/apache/hadoop/hive/ql/exec/SMBMapJoinOperator.java <https://reviews.apache.org/r/14221/#comment51361> From the context, I think op should be a TS. So I will remove this else. ql/src/java/org/apache/hadoop/hive/ql/exec/mr/MapredLocalTask.java <https://reviews.apache.org/r/14221/#comment51362> removed in diff3 ql/src/java/org/apache/hadoop/hive/ql/exec/mr/MapredLocalTask.java <https://reviews.apache.org/r/14221/#comment51363> will remove it. serde/src/java/org/apache/hadoop/hive/serde2/ColumnProjectionUtils.java <https://reviews.apache.org/r/14221/#comment51364> it is only used in tests. I will remove it. serde/src/java/org/apache/hadoop/hive/serde2/ColumnProjectionUtils.java <https://reviews.apache.org/r/14221/#comment51365> will remove it. serde/src/java/org/apache/hadoop/hive/serde2/ColumnProjectionUtils.java <https://reviews.apache.org/r/14221/#comment51366> will remove it serde/src/java/org/apache/hadoop/hive/serde2/ColumnProjectionUtils.java <https://reviews.apache.org/r/14221/#comment51367> If we use LinkedHashSet, there will be no duplicate in ids. But, we also need to check if there is any duplicate in the read column string we are appending to (it will happen in the node running the compiler). The current version will leave the deduplication work in getReadColumnIDs (it will happen in every task). I think your suggestion is better. Will change it serde/src/java/org/apache/hadoop/hive/serde2/ColumnProjectionUtils.java <https://reviews.apache.org/r/14221/#comment51370> right now, we do the deduplication work in getReadColumnIDs (at every task). I think your suggestion is better. Will change it. serde/src/java/org/apache/hadoop/hive/serde2/ColumnProjectionUtils.java <https://reviews.apache.org/r/14221/#comment51369> I will check it usage and make sure ids will not be null. Since ids is an input parameter, is it better to add an annotation or have an assertion? serde/src/java/org/apache/hadoop/hive/serde2/ColumnProjectionUtils.java <https://reviews.apache.org/r/14221/#comment51371> ok will remove it. serde/src/java/org/apache/hadoop/hive/serde2/ColumnProjectionUtils.java <https://reviews.apache.org/r/14221/#comment51372> ok, will also remove other null check - Yin Huai On Sept. 19, 2013, 5:48 p.m., Yin Huai wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/14221/ > ----------------------------------------------------------- > > (Updated Sept. 19, 2013, 5:48 p.m.) > > > Review request for hive. > > > Bugs: HIVE-4113 > https://issues.apache.org/jira/browse/HIVE-4113 > > > Repository: hive-git > > > Description > ------- > > Modifies ColumnProjectionUtils such there are two flags. One for the column > ids and one indicating whether all columns should be read. Additionally the > patch updates all locations which uses the old method of empty string > indicating all columns should be read. > > The automatic formatter generated by ant eclipse-files is fairly aggressive > so there are some unrelated import/whitespace cleanup. > > This one is based on https://reviews.apache.org/r/11770/ and has been rebased > to the latest trunk. > > > Diffs > ----- > > common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 9f37d0c > conf/hive-default.xml.template 545026d > > hbase-handler/src/java/org/apache/hadoop/hive/hbase/HiveHBaseTableInputFormat.java > 766056b > > hcatalog/core/src/main/java/org/apache/hive/hcatalog/mapreduce/HCatBaseInputFormat.java > 553446a > > hcatalog/core/src/main/java/org/apache/hive/hcatalog/mapreduce/HCatRecordReader.java > 3ee6157 > > hcatalog/core/src/main/java/org/apache/hive/hcatalog/mapreduce/InitializeInput.java > 1980ef5 > > hcatalog/core/src/test/java/org/apache/hive/hcatalog/mapreduce/TestHCatPartitioned.java > 577e06d > > hcatalog/hcatalog-pig-adapter/src/test/java/org/apache/hive/hcatalog/pig/TestHCatLoader.java > d38bb8d > ql/src/java/org/apache/hadoop/hive/ql/Driver.java 31a52ba > ql/src/java/org/apache/hadoop/hive/ql/exec/SMBMapJoinOperator.java ab0494e > ql/src/java/org/apache/hadoop/hive/ql/exec/TableScanOperator.java a5a8943 > ql/src/java/org/apache/hadoop/hive/ql/exec/mr/MapredLocalTask.java 0f29a0e > ql/src/java/org/apache/hadoop/hive/ql/io/BucketizedHiveInputFormat.java > 49145b7 > ql/src/java/org/apache/hadoop/hive/ql/io/HiveInputFormat.java cccdc1b > ql/src/java/org/apache/hadoop/hive/ql/io/RCFile.java a83f223 > ql/src/java/org/apache/hadoop/hive/ql/io/RCFileRecordReader.java 9521060 > ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java 50c5093 > > ql/src/java/org/apache/hadoop/hive/ql/io/rcfile/merge/RCFileBlockMergeRecordReader.java > cbdc2db > > ql/src/java/org/apache/hadoop/hive/ql/optimizer/ColumnPrunerProcFactory.java > ed14e82 > ql/src/java/org/apache/hadoop/hive/ql/optimizer/Optimizer.java b97d869 > > ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/MetadataOnlyOptimizer.java > 0550bf6 > ql/src/test/org/apache/hadoop/hive/ql/io/PerformTestRCFileAndSeqFile.java > fb9fca1 > ql/src/test/org/apache/hadoop/hive/ql/io/TestRCFile.java dd1276d > ql/src/test/org/apache/hadoop/hive/ql/io/orc/TestInputOutputFormat.java > 83c5c38 > serde/src/java/org/apache/hadoop/hive/serde2/ColumnProjectionUtils.java > 0b3ef7b > serde/src/java/org/apache/hadoop/hive/serde2/columnar/ColumnarSerDe.java > 11f5f07 > serde/src/java/org/apache/hadoop/hive/serde2/columnar/ColumnarStruct.java > 1335446 > > serde/src/java/org/apache/hadoop/hive/serde2/columnar/ColumnarStructBase.java > e1270cc > > serde/src/java/org/apache/hadoop/hive/serde2/columnar/LazyBinaryColumnarSerDe.java > b717278 > > serde/src/java/org/apache/hadoop/hive/serde2/columnar/LazyBinaryColumnarStruct.java > 0317024 > serde/src/test/org/apache/hadoop/hive/serde2/TestColumnProjectionUtils.java > PRE-CREATION > > Diff: https://reviews.apache.org/r/14221/diff/ > > > Testing > ------- > > > Thanks, > > Yin Huai > >
