[jira] [Updated] (HIVE-23721) MetaStoreDirectSql.ensureDbInit() need to optimize QuerySQL
[ https://issues.apache.org/jira/browse/HIVE-23721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Denys Kuzmenko updated HIVE-23721: -- Priority: Major (was: Critical) > MetaStoreDirectSql.ensureDbInit() need to optimize QuerySQL > --- > > Key: HIVE-23721 > URL: https://issues.apache.org/jira/browse/HIVE-23721 > Project: Hive > Issue Type: Bug > Components: Standalone Metastore >Affects Versions: 3.1.2, 4.0.0 > Environment: Hadoop 3.1（1700+ nodes） > YARN 3.1 （with timelineserver enabled，https enabled) > Hive 3.1 (15 HS2 instance) > 6+ YARN Applications every day >Reporter: YulongZ >Assignee: Butao Zhang >Priority: Major > Labels: pull-request-available > Attachments: HIVE-23721.01.patch > > Time Spent: 40m > Remaining Estimate: 0h > > From Hive3.0，catalog added to hivemeta，many schema of metastore added column > “catName”，and index for table added column “catName”。 > In MetaStoreDirectSql.ensureDbInit() ，two queries below > “ > initQueries.add(pm.newQuery(MTableColumnStatistics.class, "dbName == > ''")); > initQueries.add(pm.newQuery(MPartitionColumnStatistics.class, "dbName > == ''")); > ” > should use "catName == ''" instead of "dbName == ''"，because “catName” is the > first index column。 > When data of metastore become large，for example， table of > MPartitionColumnStatistics have millions of lines。The > “newQuery(MPartitionColumnStatistics.class, "dbName == ''")” for metastore > executed very slowly，and the query “show tables“ for hiveserver2 executed > very slowly too。 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-23721) MetaStoreDirectSql.ensureDbInit() need to optimize QuerySQL
[ https://issues.apache.org/jira/browse/HIVE-23721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Denys Kuzmenko updated HIVE-23721: -- Target Version/s: 4.1.0 > MetaStoreDirectSql.ensureDbInit() need to optimize QuerySQL > --- > > Key: HIVE-23721 > URL: https://issues.apache.org/jira/browse/HIVE-23721 > Project: Hive > Issue Type: Bug > Components: Standalone Metastore >Affects Versions: 3.1.2, 4.0.0 > Environment: Hadoop 3.1（1700+ nodes） > YARN 3.1 （with timelineserver enabled，https enabled) > Hive 3.1 (15 HS2 instance) > 6+ YARN Applications every day >Reporter: YulongZ >Assignee: Butao Zhang >Priority: Critical > Labels: pull-request-available > Attachments: HIVE-23721.01.patch > > Time Spent: 40m > Remaining Estimate: 0h > > From Hive3.0，catalog added to hivemeta，many schema of metastore added column > “catName”，and index for table added column “catName”。 > In MetaStoreDirectSql.ensureDbInit() ，two queries below > “ > initQueries.add(pm.newQuery(MTableColumnStatistics.class, "dbName == > ''")); > initQueries.add(pm.newQuery(MPartitionColumnStatistics.class, "dbName > == ''")); > ” > should use "catName == ''" instead of "dbName == ''"，because “catName” is the > first index column。 > When data of metastore become large，for example， table of > MPartitionColumnStatistics have millions of lines。The > “newQuery(MPartitionColumnStatistics.class, "dbName == ''")” for metastore > executed very slowly，and the query “show tables“ for hiveserver2 executed > very slowly too。 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-23721) MetaStoreDirectSql.ensureDbInit() need to optimize QuerySQL
[ https://issues.apache.org/jira/browse/HIVE-23721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Denys Kuzmenko updated HIVE-23721: -- Component/s: Standalone Metastore > MetaStoreDirectSql.ensureDbInit() need to optimize QuerySQL > --- > > Key: HIVE-23721 > URL: https://issues.apache.org/jira/browse/HIVE-23721 > Project: Hive > Issue Type: Bug > Components: Standalone Metastore >Affects Versions: 3.1.2, 4.0.0 > Environment: Hadoop 3.1（1700+ nodes） > YARN 3.1 （with timelineserver enabled，https enabled) > Hive 3.1 (15 HS2 instance) > 6+ YARN Applications every day >Reporter: YulongZ >Assignee: Butao Zhang >Priority: Critical > Labels: pull-request-available > Attachments: HIVE-23721.01.patch > > Time Spent: 40m > Remaining Estimate: 0h > > From Hive3.0，catalog added to hivemeta，many schema of metastore added column > “catName”，and index for table added column “catName”。 > In MetaStoreDirectSql.ensureDbInit() ，two queries below > “ > initQueries.add(pm.newQuery(MTableColumnStatistics.class, "dbName == > ''")); > initQueries.add(pm.newQuery(MPartitionColumnStatistics.class, "dbName > == ''")); > ” > should use "catName == ''" instead of "dbName == ''"，because “catName” is the > first index column。 > When data of metastore become large，for example， table of > MPartitionColumnStatistics have millions of lines。The > “newQuery(MPartitionColumnStatistics.class, "dbName == ''")” for metastore > executed very slowly，and the query “show tables“ for hiveserver2 executed > very slowly too。 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-23721) MetaStoreDirectSql.ensureDbInit() need to optimize QuerySQL
[ https://issues.apache.org/jira/browse/HIVE-23721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stamatis Zampetakis updated HIVE-23721: --- Fix Version/s: (was: 4.0.0) I cleared the fixVersion field since this ticket is not resolved. Please review this ticket and if the fix is already committed to a specific version please set the version accordingly and mark the ticket as RESOLVED. According to the JIRA guidelines (https://cwiki.apache.org/confluence/display/Hive/HowToContribute) the fixVersion should be set only when the issue is resolved/closed. > MetaStoreDirectSql.ensureDbInit() need to optimize QuerySQL > --- > > Key: HIVE-23721 > URL: https://issues.apache.org/jira/browse/HIVE-23721 > Project: Hive > Issue Type: Bug >Affects Versions: 3.1.2, 4.0.0 > Environment: Hadoop 3.1（1700+ nodes） > YARN 3.1 （with timelineserver enabled，https enabled) > Hive 3.1 (15 HS2 instance) > 6+ YARN Applications every day >Reporter: YulongZ >Assignee: zhangbutao >Priority: Critical > Labels: pull-request-available > Attachments: HIVE-23721.01.patch > > Time Spent: 40m > Remaining Estimate: 0h > > From Hive3.0，catalog added to hivemeta，many schema of metastore added column > “catName”，and index for table added column “catName”。 > In MetaStoreDirectSql.ensureDbInit() ，two queries below > “ > initQueries.add(pm.newQuery(MTableColumnStatistics.class, "dbName == > ''")); > initQueries.add(pm.newQuery(MPartitionColumnStatistics.class, "dbName > == ''")); > ” > should use "catName == ''" instead of "dbName == ''"，because “catName” is the > first index column。 > When data of metastore become large，for example， table of > MPartitionColumnStatistics have millions of lines。The > “newQuery(MPartitionColumnStatistics.class, "dbName == ''")” for metastore > executed very slowly，and the query “show tables“ for hiveserver2 executed > very slowly too。 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-23721) MetaStoreDirectSql.ensureDbInit() need to optimize QuerySQL
[ https://issues.apache.org/jira/browse/HIVE-23721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhangbutao updated HIVE-23721: -- Affects Version/s: 4.0.0 > MetaStoreDirectSql.ensureDbInit() need to optimize QuerySQL > --- > > Key: HIVE-23721 > URL: https://issues.apache.org/jira/browse/HIVE-23721 > Project: Hive > Issue Type: Bug >Affects Versions: 4.0.0, 3.1.2 > Environment: Hadoop 3.1（1700+ nodes） > YARN 3.1 （with timelineserver enabled，https enabled) > Hive 3.1 (15 HS2 instance) > 6+ YARN Applications every day >Reporter: YulongZ >Assignee: zhangbutao >Priority: Critical > Labels: pull-request-available > Fix For: 4.0.0 > > Attachments: HIVE-23721.01.patch > > Time Spent: 10m > Remaining Estimate: 0h > > From Hive3.0，catalog added to hivemeta，many schema of metastore added column > “catName”，and index for table added column “catName”。 > In MetaStoreDirectSql.ensureDbInit() ，two queries below > “ > initQueries.add(pm.newQuery(MTableColumnStatistics.class, "dbName == > ''")); > initQueries.add(pm.newQuery(MPartitionColumnStatistics.class, "dbName > == ''")); > ” > should use "catName == ''" instead of "dbName == ''"，because “catName” is the > first index column。 > When data of metastore become large，for example， table of > MPartitionColumnStatistics have millions of lines。The > “newQuery(MPartitionColumnStatistics.class, "dbName == ''")” for metastore > executed very slowly，and the query “show tables“ for hiveserver2 executed > very slowly too。 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23721) MetaStoreDirectSql.ensureDbInit() need to optimize QuerySQL
[ https://issues.apache.org/jira/browse/HIVE-23721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-23721: -- Labels: pull-request-available (was: ) > MetaStoreDirectSql.ensureDbInit() need to optimize QuerySQL > --- > > Key: HIVE-23721 > URL: https://issues.apache.org/jira/browse/HIVE-23721 > Project: Hive > Issue Type: Bug >Affects Versions: 3.1.2 > Environment: Hadoop 3.1（1700+ nodes） > YARN 3.1 （with timelineserver enabled，https enabled) > Hive 3.1 (15 HS2 instance) > 6+ YARN Applications every day >Reporter: YulongZ >Assignee: zhangbutao >Priority: Critical > Labels: pull-request-available > Fix For: 4.0.0 > > Attachments: HIVE-23721.01.patch > > Time Spent: 10m > Remaining Estimate: 0h > > From Hive3.0，catalog added to hivemeta，many schema of metastore added column > “catName”，and index for table added column “catName”。 > In MetaStoreDirectSql.ensureDbInit() ，two queries below > “ > initQueries.add(pm.newQuery(MTableColumnStatistics.class, "dbName == > ''")); > initQueries.add(pm.newQuery(MPartitionColumnStatistics.class, "dbName > == ''")); > ” > should use "catName == ''" instead of "dbName == ''"，because “catName” is the > first index column。 > When data of metastore become large，for example， table of > MPartitionColumnStatistics have millions of lines。The > “newQuery(MPartitionColumnStatistics.class, "dbName == ''")” for metastore > executed very slowly，and the query “show tables“ for hiveserver2 executed > very slowly too。 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23721) MetaStoreDirectSql.ensureDbInit() need to optimize QuerySQL
[ https://issues.apache.org/jira/browse/HIVE-23721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhangbutao updated HIVE-23721: -- Attachment: HIVE-23721.01.patch Fix Version/s: 4.0.0 Status: Patch Available (was: Open) > MetaStoreDirectSql.ensureDbInit() need to optimize QuerySQL > --- > > Key: HIVE-23721 > URL: https://issues.apache.org/jira/browse/HIVE-23721 > Project: Hive > Issue Type: Bug >Affects Versions: 3.1.2 > Environment: Hadoop 3.1（1700+ nodes） > YARN 3.1 （with timelineserver enabled，https enabled) > Hive 3.1 (15 HS2 instance) > 6+ YARN Applications every day >Reporter: YulongZ >Assignee: zhangbutao >Priority: Critical > Fix For: 4.0.0 > > Attachments: HIVE-23721.01.patch > > > From Hive3.0，catalog added to hivemeta，many schema of metastore added column > “catName”，and index for table added column “catName”。 > In MetaStoreDirectSql.ensureDbInit() ，two queries below > “ > initQueries.add(pm.newQuery(MTableColumnStatistics.class, "dbName == > ''")); > initQueries.add(pm.newQuery(MPartitionColumnStatistics.class, "dbName > == ''")); > ” > should use "catName == ''" instead of "dbName == ''"，because “catName” is the > first index column。 > When data of metastore become large，for example， table of > MPartitionColumnStatistics have millions of lines。The > “newQuery(MPartitionColumnStatistics.class, "dbName == ''")” for metastore > executed very slowly，and the query “show tables“ for hiveserver2 executed > very slowly too。 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23721) MetaStoreDirectSql.ensureDbInit() need to optimize QuerySQL
[ https://issues.apache.org/jira/browse/HIVE-23721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] YulongZ updated HIVE-23721: --- Environment: Hadoop 3.1（1700+ nodes） YARN 3.1 （with timelineserver enabled，https enabled) Hive 3.1 (15 HS2 instance) 6+ YARN Applications every day > MetaStoreDirectSql.ensureDbInit() need to optimize QuerySQL > --- > > Key: HIVE-23721 > URL: https://issues.apache.org/jira/browse/HIVE-23721 > Project: Hive > Issue Type: Bug >Affects Versions: 3.1.2 > Environment: Hadoop 3.1（1700+ nodes） > YARN 3.1 （with timelineserver enabled，https enabled) > Hive 3.1 (15 HS2 instance) > 6+ YARN Applications every day >Reporter: YulongZ >Priority: Critical > > From Hive3.0，catalog added to hivemeta，many schema of metastore added column > “catName”，and index for table added column “catName”。 > In MetaStoreDirectSql.ensureDbInit() ，two queries below > “ > initQueries.add(pm.newQuery(MTableColumnStatistics.class, "dbName == > ''")); > initQueries.add(pm.newQuery(MPartitionColumnStatistics.class, "dbName > == ''")); > ” > should use "catName == ''" instead of "dbName == ''"，because “catName” is the > first index column。 > When data of metastore become large，for example， table of > MPartitionColumnStatistics have millions of lines。The > “newQuery(MPartitionColumnStatistics.class, "dbName == ''")” for metastore > executed very slowly，and the query “show tables“ for hiveserver2 executed > very slowly too。 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23721) MetaStoreDirectSql.ensureDbInit() need to optimize QuerySQL
[ https://issues.apache.org/jira/browse/HIVE-23721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] YulongZ updated HIVE-23721: --- Priority: Critical (was: Minor) > MetaStoreDirectSql.ensureDbInit() need to optimize QuerySQL > --- > > Key: HIVE-23721 > URL: https://issues.apache.org/jira/browse/HIVE-23721 > Project: Hive > Issue Type: Bug >Affects Versions: 3.1.2 >Reporter: YulongZ >Priority: Critical > > From Hive3.0，catalog added to hivemeta，many schema of metastore added column > “catName”，and index for table added column “catName”。 > In MetaStoreDirectSql.ensureDbInit() ，two queries below > “ > initQueries.add(pm.newQuery(MTableColumnStatistics.class, "dbName == > ''")); > initQueries.add(pm.newQuery(MPartitionColumnStatistics.class, "dbName > == ''")); > ” > should use "catName == ''" instead of "dbName == ''"，because “catName” is the > first index column。 > When data of metastore become large，for example， table of > MPartitionColumnStatistics have millions of lines。The > “newQuery(MPartitionColumnStatistics.class, "dbName == ''")” for metastore > executed very slowly，and the query “show tables“ for hiveserver2 executed > very slowly too。 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23721) MetaStoreDirectSql.ensureDbInit() need to optimize QuerySQL
[ https://issues.apache.org/jira/browse/HIVE-23721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] YulongZ updated HIVE-23721: --- Description: >From Hive3.0，catalog added to hivemeta，many schema of metastore added column >“catName”，and index for table added column “catName”。 In MetaStoreDirectSql.ensureDbInit() ，two queries below “ initQueries.add(pm.newQuery(MTableColumnStatistics.class, "dbName == ''")); initQueries.add(pm.newQuery(MPartitionColumnStatistics.class, "dbName == ''")); ” should use "catName == ''" instead of "dbName == ''"，because “catName” is the first index column。 When data of metastore become large，for example， table of MPartitionColumnStatistics have millions of lines。The “newQuery(MPartitionColumnStatistics.class, "dbName == ''")” for metastore executed very slowly，and the query “show tables“ for hiveserver2 executed very slowly too。 was:From Hive3.0，catalog added to hivemeta，many schema of metastore added column “catName”，and index for table added column “catName”。For Hive2.0， > MetaStoreDirectSql.ensureDbInit() need to optimize QuerySQL > --- > > Key: HIVE-23721 > URL: https://issues.apache.org/jira/browse/HIVE-23721 > Project: Hive > Issue Type: Bug >Affects Versions: 3.1.2 >Reporter: YulongZ >Priority: Minor > > From Hive3.0，catalog added to hivemeta，many schema of metastore added column > “catName”，and index for table added column “catName”。 > In MetaStoreDirectSql.ensureDbInit() ，two queries below > “ > initQueries.add(pm.newQuery(MTableColumnStatistics.class, "dbName == > ''")); > initQueries.add(pm.newQuery(MPartitionColumnStatistics.class, "dbName > == ''")); > ” > should use "catName == ''" instead of "dbName == ''"，because “catName” is the > first index column。 > When data of metastore become large，for example， table of > MPartitionColumnStatistics have millions of lines。The > “newQuery(MPartitionColumnStatistics.class, "dbName == ''")” for metastore > executed very slowly，and the query “show tables“ for hiveserver2 executed > very slowly too。 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23721) MetaStoreDirectSql.ensureDbInit() need to optimize QuerySQL
[ https://issues.apache.org/jira/browse/HIVE-23721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] YulongZ updated HIVE-23721: --- Description: From Hive3.0，catalog added to hivemeta，many schema of metastore added column “catName”，and index for table added column “catName”。For Hive2.0， > MetaStoreDirectSql.ensureDbInit() need to optimize QuerySQL > --- > > Key: HIVE-23721 > URL: https://issues.apache.org/jira/browse/HIVE-23721 > Project: Hive > Issue Type: Bug >Affects Versions: 3.1.2 >Reporter: YulongZ >Priority: Minor > > From Hive3.0，catalog added to hivemeta，many schema of metastore added column > “catName”，and index for table added column “catName”。For Hive2.0， -- This message was sent by Atlassian Jira (v8.3.4#803005)
