Bhushan Mandhani created HIVE-3959:
--------------------------------------
Summary: Update Partition Statistics in Metastore Layer
Key: HIVE-3959
URL: https://issues.apache.org/jira/browse/HIVE-3959
Project: Hive
Issue Type: Improvement
Components: Metastore, Statistics
Reporter: Bhushan Mandhani
Assignee: Bhushan Mandhani
Priority: Minor
When partitions are created using queries ("insert overwrite" and "insert
into") then the StatsTask updates all stats. However, when partitions are added
directly through metadata-only partitions (either CLI or direct calls to Thrift
Metastore) no stats are populated even if hive.stats.reliable is set to true.
This puts us in a situation where we can't decide if stats are truly reliable
or not.
We propose that the "fast stats" (numFiles and totalSize) which don't require a
scan of the data should always be populated and be completely reliable. For now
we are still excluding rowCount and rawDataSize because that will make these
operations very expensive. Currently they are quick metadata-only ops.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
