[ https://issues.apache.org/jira/browse/HIVE-4518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13848991#comment-13848991 ]
Lefty Leverenz commented on HIVE-4518: -------------------------------------- This patch removed *hive.task.progress* from HiveConf.java and hive-default.xml.template, so I revised the Configuration Properties wiki with a version note: "Removed in: Hive 0.13.0 with HIVE-4518" (see [https://cwiki.apache.org/confluence/display/Hive/Configuration+Properties] and search for "hive.task.progress"). But the patch also added *hive.counters.group.name* to HiveConf.java without defining it in hive-default.xml.template. Would someone please put a definition in the release note for this ticket? Then it can be added to hive-default.xml.template, either with the spelling fixes in HIVE-5879 or in a follow-up ticket when the Config Props wiki gets synchronized with hive-default.xml.template, and I'll add it to the wiki. > Counter Strike: Operation Operator > ---------------------------------- > > Key: HIVE-4518 > URL: https://issues.apache.org/jira/browse/HIVE-4518 > Project: Hive > Issue Type: Improvement > Reporter: Gunther Hagleitner > Assignee: Gunther Hagleitner > Fix For: 0.13.0 > > Attachments: HIVE-4518.1.patch, HIVE-4518.10.patch, > HIVE-4518.11.patch, HIVE-4518.2.patch, HIVE-4518.3.patch, HIVE-4518.4.patch, > HIVE-4518.5.patch, HIVE-4518.6.patch.txt, HIVE-4518.7.patch, > HIVE-4518.8.patch, HIVE-4518.9.patch > > > Queries of the form: > from foo > insert overwrite table bar partition (p) select ... > insert overwrite table bar partition (p) select ... > insert overwrite table bar partition (p) select ... > Generate a huge amount of counters. The reason is that task.progress is > turned on for dynamic partitioning queries. > The counters not only make queries slower than necessary (up to 50%) you will > also eventually run out. That's because we're wrapping them in enum values to > comply with hadoop 0.17. > The real reason we turn task.progress on is that we need CREATED_FILES and > FATAL counters to ensure dynamic partitioning queries don't go haywire. > The counters have counter-intuitive names like C1 through C1000 and don't > seem really useful by themselves. > With hadoop 20+ you don't need to wrap the counters anymore, each operator > can simply create and increment counters. That should simplify the code a lot. -- This message was sent by Atlassian JIRA (v6.1.4#6159)