[jira] [Work logged] (HIVE-23721) MetaStoreDirectSql.ensureDbInit() need to optimize QuerySQL
[ https://issues.apache.org/jira/browse/HIVE-23721?focusedWorklogId=480512=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-480512 ] ASF GitHub Bot logged work on HIVE-23721: - Author: ASF GitHub Bot Created on: 09/Sep/20 00:46 Start Date: 09/Sep/20 00:46 Worklog Time Spent: 10m Work Description: github-actions[bot] closed pull request #1202: URL: https://github.com/apache/hive/pull/1202 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 480512) Time Spent: 40m (was: 0.5h) > MetaStoreDirectSql.ensureDbInit() need to optimize QuerySQL > --- > > Key: HIVE-23721 > URL: https://issues.apache.org/jira/browse/HIVE-23721 > Project: Hive > Issue Type: Bug >Affects Versions: 4.0.0, 3.1.2 > Environment: Hadoop 3.1（1700+ nodes） > YARN 3.1 （with timelineserver enabled，https enabled) > Hive 3.1 (15 HS2 instance) > 6+ YARN Applications every day >Reporter: YulongZ >Assignee: zhangbutao >Priority: Critical > Labels: pull-request-available > Fix For: 4.0.0 > > Attachments: HIVE-23721.01.patch > > Time Spent: 40m > Remaining Estimate: 0h > > From Hive3.0，catalog added to hivemeta，many schema of metastore added column > “catName”，and index for table added column “catName”。 > In MetaStoreDirectSql.ensureDbInit() ，two queries below > “ > initQueries.add(pm.newQuery(MTableColumnStatistics.class, "dbName == > ''")); > initQueries.add(pm.newQuery(MPartitionColumnStatistics.class, "dbName > == ''")); > ” > should use "catName == ''" instead of "dbName == ''"，because “catName” is the > first index column。 > When data of metastore become large，for example， table of > MPartitionColumnStatistics have millions of lines。The > “newQuery(MPartitionColumnStatistics.class, "dbName == ''")” for metastore > executed very slowly，and the query “show tables“ for hiveserver2 executed > very slowly too。 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-23721) MetaStoreDirectSql.ensureDbInit() need to optimize QuerySQL
[ https://issues.apache.org/jira/browse/HIVE-23721?focusedWorklogId=477545=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-477545 ] ASF GitHub Bot logged work on HIVE-23721: - Author: ASF GitHub Bot Created on: 02/Sep/20 00:41 Start Date: 02/Sep/20 00:41 Worklog Time Spent: 10m Work Description: github-actions[bot] commented on pull request #1202: URL: https://github.com/apache/hive/pull/1202#issuecomment-685209535 This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Feel free to reach out on the d...@hive.apache.org list if the patch is in need of reviews. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 477545) Time Spent: 0.5h (was: 20m) > MetaStoreDirectSql.ensureDbInit() need to optimize QuerySQL > --- > > Key: HIVE-23721 > URL: https://issues.apache.org/jira/browse/HIVE-23721 > Project: Hive > Issue Type: Bug >Affects Versions: 4.0.0, 3.1.2 > Environment: Hadoop 3.1（1700+ nodes） > YARN 3.1 （with timelineserver enabled，https enabled) > Hive 3.1 (15 HS2 instance) > 6+ YARN Applications every day >Reporter: YulongZ >Assignee: zhangbutao >Priority: Critical > Labels: pull-request-available > Fix For: 4.0.0 > > Attachments: HIVE-23721.01.patch > > Time Spent: 0.5h > Remaining Estimate: 0h > > From Hive3.0，catalog added to hivemeta，many schema of metastore added column > “catName”，and index for table added column “catName”。 > In MetaStoreDirectSql.ensureDbInit() ，two queries below > “ > initQueries.add(pm.newQuery(MTableColumnStatistics.class, "dbName == > ''")); > initQueries.add(pm.newQuery(MPartitionColumnStatistics.class, "dbName > == ''")); > ” > should use "catName == ''" instead of "dbName == ''"，because “catName” is the > first index column。 > When data of metastore become large，for example， table of > MPartitionColumnStatistics have millions of lines。The > “newQuery(MPartitionColumnStatistics.class, "dbName == ''")” for metastore > executed very slowly，and the query “show tables“ for hiveserver2 executed > very slowly too。 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-23721) MetaStoreDirectSql.ensureDbInit() need to optimize QuerySQL
[ https://issues.apache.org/jira/browse/HIVE-23721?focusedWorklogId=454250=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-454250 ] ASF GitHub Bot logged work on HIVE-23721: - Author: ASF GitHub Bot Created on: 03/Jul/20 02:51 Start Date: 03/Jul/20 02:51 Worklog Time Spent: 10m Work Description: butaozhang commented on pull request #1202: URL: https://github.com/apache/hive/pull/1202#issuecomment-653309334 Failed tests seem not be relate to this pr , and I can run successfully in my local env. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 454250) Time Spent: 20m (was: 10m) > MetaStoreDirectSql.ensureDbInit() need to optimize QuerySQL > --- > > Key: HIVE-23721 > URL: https://issues.apache.org/jira/browse/HIVE-23721 > Project: Hive > Issue Type: Bug >Affects Versions: 4.0.0, 3.1.2 > Environment: Hadoop 3.1（1700+ nodes） > YARN 3.1 （with timelineserver enabled，https enabled) > Hive 3.1 (15 HS2 instance) > 6+ YARN Applications every day >Reporter: YulongZ >Assignee: zhangbutao >Priority: Critical > Labels: pull-request-available > Fix For: 4.0.0 > > Attachments: HIVE-23721.01.patch > > Time Spent: 20m > Remaining Estimate: 0h > > From Hive3.0，catalog added to hivemeta，many schema of metastore added column > “catName”，and index for table added column “catName”。 > In MetaStoreDirectSql.ensureDbInit() ，two queries below > “ > initQueries.add(pm.newQuery(MTableColumnStatistics.class, "dbName == > ''")); > initQueries.add(pm.newQuery(MPartitionColumnStatistics.class, "dbName > == ''")); > ” > should use "catName == ''" instead of "dbName == ''"，because “catName” is the > first index column。 > When data of metastore become large，for example， table of > MPartitionColumnStatistics have millions of lines。The > “newQuery(MPartitionColumnStatistics.class, "dbName == ''")” for metastore > executed very slowly，and the query “show tables“ for hiveserver2 executed > very slowly too。 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-23721) MetaStoreDirectSql.ensureDbInit() need to optimize QuerySQL
[ https://issues.apache.org/jira/browse/HIVE-23721?focusedWorklogId=453816=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-453816 ] ASF GitHub Bot logged work on HIVE-23721: - Author: ASF GitHub Bot Created on: 02/Jul/20 07:47 Start Date: 02/Jul/20 07:47 Worklog Time Spent: 10m Work Description: butaozhang opened a new pull request #1202: URL: https://github.com/apache/hive/pull/1202 ## NOTICE Please create an issue in ASF JIRA before opening a pull request, and you need to set the title of the pull request which starts with the corresponding JIRA issue number. (e.g. HIVE-X: Fix a typo in YYY) For more details, please see https://cwiki.apache.org/confluence/display/Hive/HowToContribute This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 453816) Remaining Estimate: 0h Time Spent: 10m > MetaStoreDirectSql.ensureDbInit() need to optimize QuerySQL > --- > > Key: HIVE-23721 > URL: https://issues.apache.org/jira/browse/HIVE-23721 > Project: Hive > Issue Type: Bug >Affects Versions: 3.1.2 > Environment: Hadoop 3.1（1700+ nodes） > YARN 3.1 （with timelineserver enabled，https enabled) > Hive 3.1 (15 HS2 instance) > 6+ YARN Applications every day >Reporter: YulongZ >Assignee: zhangbutao >Priority: Critical > Fix For: 4.0.0 > > Attachments: HIVE-23721.01.patch > > Time Spent: 10m > Remaining Estimate: 0h > > From Hive3.0，catalog added to hivemeta，many schema of metastore added column > “catName”，and index for table added column “catName”。 > In MetaStoreDirectSql.ensureDbInit() ，two queries below > “ > initQueries.add(pm.newQuery(MTableColumnStatistics.class, "dbName == > ''")); > initQueries.add(pm.newQuery(MPartitionColumnStatistics.class, "dbName > == ''")); > ” > should use "catName == ''" instead of "dbName == ''"，because “catName” is the > first index column。 > When data of metastore become large，for example， table of > MPartitionColumnStatistics have millions of lines。The > “newQuery(MPartitionColumnStatistics.class, "dbName == ''")” for metastore > executed very slowly，and the query “show tables“ for hiveserver2 executed > very slowly too。 -- This message was sent by Atlassian Jira (v8.3.4#803005)
