[
https://issues.apache.org/jira/browse/HIVE-17634?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
liyunzhang_intel updated HIVE-17634:
------------------------------------
Description: In the statistics estimation([StatsRulesProcFactory|), we do
not estimate the column stats once we set hive.stats.fetch.column.stats as
false.Suggest to estimate the data size by column type when
{{hive.stats.fetch.column.stats}} as false like HIVE-17634.1.patch does. (was:
in
[RelOptHiveTable#updateColStats|https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/RelOptHiveTable.java#L309],
we set {{fetchColStats}},{{fetchPartStats}} as true when call
{{StatsUtils.collectStatistics}}
{code}
if (!hiveTblMetadata.isPartitioned()) {
// 2.1 Handle the case for unpartitioned table.
try {
Statistics stats = StatsUtils.collectStatistics(hiveConf, null,
hiveTblMetadata, hiveNonPartitionCols,
nonPartColNamesThatRqrStats,
colStatsCached, nonPartColNamesThatRqrStats, true, true);
...
{code}
This will cause querying columns statistic from metastore even we set
{{hive.stats.fetch.column.stats}} and {{hive.stats.fetch.partition.stats}} as
false in HiveConf. If we these two properties as false, we can not any column
statistics from metastore. Suggest to set the properties from HiveConf. )
> Estimate the column stats even not retrieve columns from
> metastore(hive.stats.fetch.column.stats as false)
> ----------------------------------------------------------------------------------------------------------
>
> Key: HIVE-17634
> URL: https://issues.apache.org/jira/browse/HIVE-17634
> Project: Hive
> Issue Type: Bug
> Reporter: liyunzhang_intel
> Assignee: liyunzhang_intel
> Attachments: HIVE-17634.1.patch, HIVE-17634.patch
>
>
> In the statistics estimation([StatsRulesProcFactory|), we do not estimate
> the column stats once we set hive.stats.fetch.column.stats as false.Suggest
> to estimate the data size by column type when
> {{hive.stats.fetch.column.stats}} as false like HIVE-17634.1.patch does.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
