[
https://issues.apache.org/jira/browse/KYLIN-4679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
chuxiao updated KYLIN-4679:
---------------------------
Description:
我的库有几千张表。StorageCleanupJob的行为是获取库里所有表的元数据再过滤表名。首先，取所有表元数据的行为会给metastore较大压力。其次，我们的metastore当一次取超过1千张表的元数据，会直接报错。所以需要增加提前用表名前缀过滤数据的功能。
My database has thousands of tables. StorageCleanupJob's behavior is to get the
metadata of all tables in the database and then filter table name.
First, the behavior of getting all tables metadata will give metastore great
pressure. Secondly, when getting metadata of more than 1,000 tables at one
time, our metastore would directly return errors. So it needs to add the
function of filtering data with table name prefix in advance
was:my database has 1,000+ tables. so I need set hive table prefix replace
all tables.
> StorageCleanupJob clean hive table support hive table prefix
> ------------------------------------------------------------
>
> Key: KYLIN-4679
> URL: https://issues.apache.org/jira/browse/KYLIN-4679
> Project: Kylin
> Issue Type: Improvement
> Reporter: chuxiao
> Priority: Major
>
> 我的库有几千张表。StorageCleanupJob的行为是获取库里所有表的元数据再过滤表名。首先，取所有表元数据的行为会给metastore较大压力。其次，我们的metastore当一次取超过1千张表的元数据，会直接报错。所以需要增加提前用表名前缀过滤数据的功能。
> My database has thousands of tables. StorageCleanupJob's behavior is to get
> the metadata of all tables in the database and then filter table name.
> First, the behavior of getting all tables metadata will give metastore great
> pressure. Secondly, when getting metadata of more than 1,000 tables at one
> time, our metastore would directly return errors. So it needs to add the
> function of filtering data with table name prefix in advance
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
