[
https://issues.apache.org/jira/browse/HIVE-11786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14946194#comment-14946194
]
Chaoyu Tang commented on HIVE-11786:
------------------------------------
[~szehon] Yes, all joins occur on primary keys and indexed foreign keys. But
some columns (like column_name etc) have not been indexed but they are used in
the query filters, which I wonder if is the main cause.
Before trying splitting a big query to two small ones, [~sseth], could you help
to run the queries again with following indexes created in MySQL test bed?
{code}
CREATE INDEX COLNAME_TBLID_IDX ON TAB_COL_STATS (COLUMN_NAME, TBL_ID);
CREATE INDEX COLNAME_IDX ON TAB_COL_STATS (COLUMN_NAME);
CREATE INDEX COLNAME_PARTID_IDX ON PART_COL_STATS (COLUMN_NAME, PART_ID);
CREATE INDEX COLNAME_IDX ON PART_COL_STATS (COLUMN_NAME);
CREATE INDEX PARTNAME_IDX ON PARTITIONS (PART_NAME);
CREATE INDEX TBLNAME_IDX ON TBLS (TBL_NAME);
{code}
if it works, I can create a patch for all RDBMS. Thanks
> Deprecate the use of redundant column in colunm stats related tables
> --------------------------------------------------------------------
>
> Key: HIVE-11786
> URL: https://issues.apache.org/jira/browse/HIVE-11786
> Project: Hive
> Issue Type: Bug
> Components: Metastore
> Reporter: Chaoyu Tang
> Assignee: Chaoyu Tang
> Fix For: 1.3.0, 2.0.0
>
> Attachments: HIVE-11786.1.patch, HIVE-11786.1.patch,
> HIVE-11786.2.patch, HIVE-11786.patch
>
>
> The stats tables such as TAB_COL_STATS, PART_COL_STATS have redundant columns
> such as DB_NAME, TABLE_NAME, PARTITION_NAME since these tables already have
> foreign key like TBL_ID, or PART_ID referencing to TBLS or PARTITIONS.
> These redundant columns violate database normalization rules and cause a lot
> of inconvenience (sometimes difficult) in column stats related feature
> implementation. For example, when renaming a table, we have to update
> TABLE_NAME column in these tables as well which is unnecessary.
> This JIRA is first to deprecate the use of these columns at HMS code level. A
> followed JIRA is to be opened to focus on DB schema change and upgrade.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
