[
https://issues.apache.org/jira/browse/DRILL-7115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16809265#comment-16809265
]
ASF GitHub Bot commented on DRILL-7115:
---------------------------------------
vdiravka commented on pull request #1706: DRILL-7115: Improve Hive schema show
tables performance
URL: https://github.com/apache/drill/pull/1706#discussion_r271930881
##########
File path:
contrib/storage-hive/core/src/main/java/org/apache/drill/exec/store/hive/schema/HiveDatabaseSchema.java
##########
@@ -63,89 +58,38 @@ public Table getTable(String tableName) {
return hiveSchema.getDrillTable(this.name, tableName);
}
+ @Override
+ public Collection<Map.Entry<String, TableType>> getTableNamesAndTypes() {
+ ensureInitTables();
+ return tables.entrySet();
+ }
+
@Override
public Set<String> getTableNames() {
+ ensureInitTables();
+ return tables.keySet();
+ }
+
+ private void ensureInitTables() {
if (tables == null) {
try {
- tables = Sets.newHashSet(mClient.getTableNames(this.name,
schemaConfig.getIgnoreAuthErrors()));
- } catch (final TException e) {
- logger.warn("Failure while attempting to access HiveDatabase '{}'.",
this.name, e.getCause());
- tables = Sets.newHashSet(); // empty set.
+ tables = mClient.getTableNamesAndTypes(this.name,
schemaConfig.getIgnoreAuthErrors());
+ } catch (TException e) {
+ logger.warn(String.format(
Review comment:
I am ok about `e` instead of `e.getCause()`, but in both cases it is the
last argument for `warn` method. So not sure what is the difference will a
String formatting approach. However WARN level is used by default, users will
not have benefit from placeholders. And this is not important. So you can keep
your approach here.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Improve Hive schema show tables performance
> -------------------------------------------
>
> Key: DRILL-7115
> URL: https://issues.apache.org/jira/browse/DRILL-7115
> Project: Apache Drill
> Issue Type: Improvement
> Components: Storage - Hive, Storage - Information Schema
> Affects Versions: 1.15.0
> Reporter: Igor Guzenko
> Assignee: Igor Guzenko
> Priority: Major
> Fix For: 1.16.0
>
>
> In Sqlline(Drill), "show tables" on a Hive schema is taking nearly 15mins to
> 20mins. The schema has nearly ~8000 tables.
> Whereas the same in beeline(Hive) is throwing the result in a split second(~
> 0.2 secs).
> I tested the same in my test cluster by creating 6000 tables(empty!) in Hive
> and then doing "show tables" in Drill. It took more than 2 mins(~140 secs).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
