[
https://issues.apache.org/jira/browse/HIVE-6500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14003107#comment-14003107
]
Lefty Leverenz edited comment on HIVE-6500 at 10/6/14 8:02 PM:
---------------------------------------------------------------
Unfortunately my review board advice not to patch hive-default.xml.template led
to release 0.13.0 having the obsolete default value for *hive.stats.dbclass* in
the template file. But it's updated in the most recent patch for HIVE-6037, so
presumably it will be corrected by release 0.14.0.
Sorry about that.
Edit: The updated parameter description didn't make it into the new version of
HiveConf.java, so it needs to be fixed in another patch. (I suggest HIVE-6586.)
was (Author: le...@hortonworks.com):
Unfortunately my review board advice not to patch hive-default.xml.template led
to release 0.13.0 having the obsolete default value for *hive.stats.dbclass* in
the template file. But it's updated in the most recent patch for HIVE-6037, so
presumably it will be corrected by release 0.14.0.
Sorry about that.
> Stats collection via filesystem
> -------------------------------
>
> Key: HIVE-6500
> URL: https://issues.apache.org/jira/browse/HIVE-6500
> Project: Hive
> Issue Type: New Feature
> Components: Statistics
> Reporter: Ashutosh Chauhan
> Assignee: Ashutosh Chauhan
> Labels: TODOC13, TODOC14
> Fix For: 0.13.0
>
> Attachments: HIVE-6500.2.patch, HIVE-6500.3.patch, HIVE-6500.patch
>
>
> Recently, support for stats gathering via counter was [added |
> https://issues.apache.org/jira/browse/HIVE-4632] Although, its useful it has
> following issues:
> * [Length of counter group name is limited |
> https://github.com/apache/hadoop-common/blob/branch-2.3/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/MRJobConfig.java?source=c#L340]
> * [Length of counter name is limited |
> https://github.com/apache/hadoop-common/blob/branch-2.3/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/MRJobConfig.java?source=c#L337]
> * [Number of distinct counter groups are limited |
> https://github.com/apache/hadoop-common/blob/branch-2.3/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/MRJobConfig.java?source=c#L343]
> * [Number of distinct counters are limited |
> https://github.com/apache/hadoop-common/blob/branch-2.3/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/MRJobConfig.java?source=c#L334]
> Although, these limits are configurable, but setting them to higher value
> implies increased memory load on AM and job history server.
> Now, whether these limits makes sense or not is [debatable |
> https://issues.apache.org/jira/browse/MAPREDUCE-5680] it is desirable that
> Hive doesn't make use of counters features of framework so that it we can
> evolve this feature without relying on support from framework. Filesystem
> based counter collection is a step in that direction.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
