[
https://issues.apache.org/jira/browse/HIVE-6572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sushanth Sowmyan updated HIVE-6572:
-----------------------------------
Description:
HadoopShims has a method to fetch config parameters by name so that they return
the appropriate config param name for the appropriate hadoop version. We need
to be consistent about using these versions.
For eg:. mapred.min.split.size is deprecated with hadoop 2.x, and is instead
called mapreduce.input.fileinputformat.split.minsize .
Also, there is a bug in Hadoop23Shims, Hadoop20SShims and Hadoop20Shims that
defines MAPREDMINSPLITSIZEPERNODE as mapred.min.split.size.per.rack and
MAPREDMINSPLITSIZEPERRACK as mapred.min.split.size.per.node. This is wrong and
confusing.
was:
HadoopShims has a method to fetch config parameters by name so that they return
the appropriate config param name for the appropriate hadoop version. We need
to be consistent about using these versions.
For eg:. mapred.min.split.size is deprecated with hadoop 2.x, and is instead
called mapreduce.input.fileinputformat.split.minsize .
Also, there is a bug in Hadoop20SShims and Hadoop20Shims that defines
MAPREDMINSPLITSIZEPERNODE as mapred.min.split.size.per.rack and
MAPREDMINSPLITSIZEPERRACK as mapred.min.split.size.per.node. This is wrong and
confusing.
> Use shimmed version of hadoop conf names for mapred.{min,max}.split.size{.*}
> ----------------------------------------------------------------------------
>
> Key: HIVE-6572
> URL: https://issues.apache.org/jira/browse/HIVE-6572
> Project: Hive
> Issue Type: Bug
> Affects Versions: 0.13.0, 0.14.0
> Reporter: Sushanth Sowmyan
> Assignee: Sushanth Sowmyan
> Attachments: HIVE-6572.patch
>
>
> HadoopShims has a method to fetch config parameters by name so that they
> return the appropriate config param name for the appropriate hadoop version.
> We need to be consistent about using these versions.
> For eg:. mapred.min.split.size is deprecated with hadoop 2.x, and is instead
> called mapreduce.input.fileinputformat.split.minsize .
> Also, there is a bug in Hadoop23Shims, Hadoop20SShims and Hadoop20Shims that
> defines MAPREDMINSPLITSIZEPERNODE as mapred.min.split.size.per.rack and
> MAPREDMINSPLITSIZEPERRACK as mapred.min.split.size.per.node. This is wrong
> and confusing.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
