Re: Review Request 36939: HIVE-11376: CombineHiveInputFormat is falling back to HiveInputFormat in case codecs are found for one of the input files

[Rajat Khandelwal]

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/36939/
-----------------------------------------------------------

(Updated July 30, 2015, 6:49 p.m.)


Review request for hive.


Bugs: HIVE-11376
    https://issues.apache.org/jira/browse/HIVE-11376


Repository: hive-git


Description
-------

https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/io/CombineHiveInputFormat.java#L379

This is the exact code snippet:

{noformat}

/ Since there is no easy way of knowing whether MAPREDUCE-1597 is present in 
the tree or not,
      // we use a configuration variable for the same
      if (this.mrwork != null && !this.mrwork.getHadoopSupportsSplittable()) {
        // The following code should be removed, once
        // https://issues.apache.org/jira/browse/MAPREDUCE-1597 is fixed.
        // Hadoop does not handle non-splittable files correctly for 
CombineFileInputFormat,
        // so don't use CombineFileInputFormat for non-splittable files

        //ie, dont't combine if inputformat is a TextInputFormat and has 
compression turned on

{noformat}


Diffs (updated)
-----

  common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 
33b67dd7b0fde41f81f8d86ea8c83d29c631e3d7 
  ql/src/java/org/apache/hadoop/hive/ql/io/CombineHiveInputFormat.java 
1de7e4073f5eea4c7be8423a7ebe6a89cb51d9f1 
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMapRedUtils.java 
693d8c7e9f956999b2da33593d780e37ddf2b3b8 
  ql/src/java/org/apache/hadoop/hive/ql/plan/MapWork.java 
3217df27bb5731a1dcd5db1ae17c5bdff2e3fbfc 
  ql/src/java/org/apache/hadoop/hive/ql/plan/PlanUtils.java 
76926e79729d9ea4823de0ffc9b1e5bac6364842 

Diff: https://reviews.apache.org/r/36939/diff/


Testing
-------


Thanks,

Rajat Khandelwal