[
https://issues.apache.org/jira/browse/HIVE-25213?focusedWorklogId=614323&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-614323
]
ASF GitHub Bot logged work on HIVE-25213:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 24/Jun/21 03:54
Start Date: 24/Jun/21 03:54
Worklog Time Spent: 10m
Work Description: nrg4878 commented on a change in pull request #2371:
URL: https://github.com/apache/hive/pull/2371#discussion_r657601470
##########
File path:
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HMSHandler.java
##########
@@ -3799,10 +3799,37 @@ public Table get_table_core(GetTableRequest
getTableRequest) throws MetaExceptio
@Override
public GetTablesResult get_table_objects_by_name_req(GetTablesRequest req)
throws TException {
String catName = req.isSetCatName() ? req.getCatName() :
getDefaultCatalog(conf);
+ if (isDatabaseRemote(req.getDbName())) {
+ return new
GetTablesResult(getRemoteTableObjectsInternal(req.getDbName(),
req.getTblNames(), req.getTablesPattern()));
+ }
return new GetTablesResult(getTableObjectsInternal(catName,
req.getDbName(),
req.getTblNames(), req.getCapabilities(), req.getProjectionSpec(),
req.getTablesPattern()));
}
+ private List<Table> filterTablesByName(List<Table> tables, List<String>
tableNames) {
+ List<Table> filteredTables = new ArrayList<>();
+ for (Table table : tables) {
+ if (tableNames.contains(table.getTableName())) {
+ filteredTables.add(table);
+ }
+ }
+ return filteredTables;
+ }
+
+ private List<Table> getRemoteTableObjectsInternal(String dbname,
List<String> tableNames, String pattern) throws MetaException {
+ String[] parsedDbName = parseDbName(dbname, conf);
+ try {
+ Database db = get_database_core(parsedDbName[CAT_NAME],
parsedDbName[DB_NAME]);
+ List<Table> tables =
DataConnectorProviderFactory.getDataConnectorProvider(db).getTables(null);
+ if (tableNames != null) {
+ tables = filterTablesByName(tables, tableNames);
+ }
+ return FilterUtils.filterTablesIfEnabled(isServerFilterEnabled,
filterHook, tables);
+ } catch (Exception e) {
+ return new ArrayList<Table>();
Review comment:
Could you log this exception as a WARN message with the message
##########
File path:
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/dataconnector/jdbc/AbstractJDBCConnectorProvider.java
##########
@@ -192,6 +215,7 @@ protected Connection getConnection() {
}
table = buildTableFromColsList(tableName, cols);
+ table.setDbName(scoped_db);
Review comment:
I think this might be wrong.
So scoped_db is the name of the remote database thats being mapped. and
table.setDbName() is meant to be the name of the hive database for this table.
So for example,
MySQL has a database called "employees" with 2 tables (profiles and
salaries).
Using a MySQL dataconnector, we map this remote db to a hive database called
"companyx_employees" like so
create remote database companyx_employees using mysql_conn
DBPROPERTIES("connector.remoteDbName"="employees");
show tables in this db should show the 2 tables "profiiles" and "salaries"
But to reference them in hive queries, you will have to use
"companyx_employees.profiles" and NOT "employees.profiiles" as employees
database in hive does not exist.
##########
File path:
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/dataconnector/jdbc/AbstractJDBCConnectorProvider.java
##########
@@ -53,7 +53,6 @@
private static final String JDBC_OUTPUTFORMAT_CLASS =
"org.apache.hive.storage.jdbc.JdbcOutputFormat".intern();
String type = null; // MYSQL, POSTGRES, ORACLE, DERBY, MSSQL, DB2 etc.
- String driverClassName = null;
Review comment:
This should not be removed as it is used by the providers to set the
driver classname. Please add it back
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 614323)
Time Spent: 1h 10m (was: 1h)
> Implement List<Table> getTables() for existing connectors.
> ----------------------------------------------------------
>
> Key: HIVE-25213
> URL: https://issues.apache.org/jira/browse/HIVE-25213
> Project: Hive
> Issue Type: Sub-task
> Reporter: Naveen Gangam
> Assignee: Dantong Dong
> Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> In the initial implementation, connector providers do not implement the
> getTables(string pattern) spi. We had deferred it for later. Only
> getTableNames() and getTable() were implemented.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
