[
https://issues.apache.org/jira/browse/HIVE-12411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pengcheng Xiong updated HIVE-12411:
-----------------------------------
Attachment: HIVE-12411.02.patch
> Remove counter based stats collection mechanism
> -----------------------------------------------
>
> Key: HIVE-12411
> URL: https://issues.apache.org/jira/browse/HIVE-12411
> Project: Hive
> Issue Type: Task
> Components: Statistics
> Reporter: Pengcheng Xiong
> Assignee: Pengcheng Xiong
> Attachments: HIVE-12411.01.patch, HIVE-12411.02.patch
>
>
> Following HIVE-12005, HIVE-12164, we have removed jdbc and hbase stats
> collection mechanism. Now we are targeting counter based stats collection
> mechanism. The main advantages are as follows (1) counter based stats has
> limitation on the length of the counter itself, if it is too long, MD5 will
> be applied. (2) when there are a large number of partitions and columns, we
> need to create a large number of counters in memory. This will put a heavy
> load on the M/R AM or Tez AM etc. FS based stats will do a better job.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
