[
https://issues.apache.org/jira/browse/DRILL-4826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15507911#comment-15507911
]
ASF GitHub Bot commented on DRILL-4826:
---------------------------------------
Github user parthchandra commented on a diff in the pull request:
https://github.com/apache/drill/pull/592#discussion_r79721140
--- Diff:
exec/java-exec/src/main/java/org/apache/drill/exec/store/ischema/InfoSchemaRecordGenerator.java
---
@@ -290,28 +290,30 @@ public Tables(OptionManager optionManager) {
return new PojoRecordReader<>(Records.Table.class,
records.iterator());
}
- @Override
- public void visitTables(String schemaPath, SchemaPlus schema) {
+ @Override public void visitTables(String schemaPath, SchemaPlus
schema) {
final AbstractSchema drillSchema =
schema.unwrap(AbstractSchema.class);
+ final List<Pair<String, TableType>> tableNamesAndTypes = drillSchema
+
.getTableNamesAndTypes(optionManager.getOption(ExecConstants.ENABLE_BULK_LOAD_TABLE_LIST),
+
(int)optionManager.getOption(ExecConstants.BULK_LOAD_TABLE_LIST_BULK_SIZE));
- final List<String> tableNames =
Lists.newArrayList(schema.getTableNames());
- final List<Pair<String, ? extends Table>> tableNameToTables;
-
if(optionManager.getOption(ExecConstants.ENABLE_BULK_LOAD_TABLE_LIST)) {
- tableNameToTables =
drillSchema.getTablesByNamesByBulkLoad(tableNames);
- } else {
- tableNameToTables = drillSchema.getTablesByNames(tableNames);
- }
-
- for(Pair<String, ? extends Table> tableNameToTable :
tableNameToTables) {
- final String tableName = tableNameToTable.getKey();
- final Table table = tableNameToTable.getValue();
+ for (Pair<String, TableType> tableNameAndType : tableNamesAndTypes) {
+ final String tableName = tableNameAndType.getKey();
+ final TableType tableType = tableNameAndType.getValue();
// Visit the table, and if requested ...
- if(shouldVisitTable(schemaPath, tableName)) {
- visitTable(schemaPath, tableName, table);
+ if (shouldVisitTable(schemaPath, tableName)) {
+ visitTableWithType(schemaPath, tableName, tableType);
}
}
}
+ public boolean visitTableWithType(String schemaName, String tableName,
TableType type) {
+ Preconditions
+ .checkNotNull(type, "Error. Type information for table %s.%s
provided is null.", schemaName,
+ tableName);
+ records.add(new Records.Table(IS_CATALOG_NAME, schemaName,
tableName, type.toString()));
+ return false;
--- End diff --
<sigh/> to keep it similar to visitTable which does the same,
unnecessarily. I suppose I could change it to return void.
> Query against INFORMATION_SCHEMA.TABLES degrades as the number of views
> increases
> ---------------------------------------------------------------------------------
>
> Key: DRILL-4826
> URL: https://issues.apache.org/jira/browse/DRILL-4826
> Project: Apache Drill
> Issue Type: Bug
> Reporter: Parth Chandra
> Assignee: Parth Chandra
>
> Queries against INFORMATION_SCHEMA.TABLES and INFORMATION_SCHEMA.VIEWS slow
> down as the number of views increases.
> BI tools like Tableau issue a query like the following at connection time:
> {code}
> select TABLE_CATALOG, TABLE_SCHEMA, TABLE_NAME, TABLE_TYPE from
> INFORMATION_SCHEMA.`TABLES` WHERE TABLE_CATALOG LIKE 'DRILL' ESCAPE '\' AND
> TABLE_SCHEMA <> 'sys' AND TABLE_SCHEMA <> 'INFORMATION_SCHEMA'ORDER BY
> TABLE_TYPE, TABLE_CATALOG, TABLE_SCHEMA, TABLE_NAME
> {code}
> The time to query the information schema tables degrades as the number of
> views increases. On a test system:
> || Views || Time(secs) ||
> |500 | 6 |
> |1000 | 19 |
> |1500 | 33 |
> This can result in a single connection taking more than a minute to establish.
> The problem occurs because we read the view file for every view and this
> appears to take most of the time.
> Querying information_schema.tables does not, in fact, need to open the view
> file at all, it merely needs to get a listing of the view files. Eliminating
> the view file read will speed up the query tremendously.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
