[
https://issues.apache.org/jira/browse/HIVE-24396?focusedWorklogId=572664&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-572664
]
ASF GitHub Bot logged work on HIVE-24396:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 26/Mar/21 13:47
Start Date: 26/Mar/21 13:47
Worklog Time Spent: 10m
Work Description: nrg4878 commented on a change in pull request #2037:
URL: https://github.com/apache/hive/pull/2037#discussion_r602306938
##########
File path:
ql/src/java/org/apache/hadoop/hive/ql/ddl/database/create/CreateDatabaseAnalyzer.java
##########
@@ -70,19 +73,43 @@ public void analyzeInternal(ASTNode root) throws
SemanticException {
managedLocationUri =
unescapeSQLString(childNode.getChild(0).getText());
outputs.add(toWriteEntity(managedLocationUri));
break;
+ case HiveParser.TOK_DATACONNECTOR:
+ type = "REMOTE";
+ // locationUri = "REMOTE_DATABASE"; // TODO
+ ASTNode nextNode = (ASTNode) root.getChild(i);
+ connectorName = ((ASTNode)nextNode).getChild(0).getText();
+ outputs.add(toWriteEntity(connectorName));
+ // outputs.remove(toWriteEntity(locationUri));
+ if (managedLocationUri != null) {
+ outputs.remove(toWriteEntity(managedLocationUri));
+ managedLocationUri = null;
+ }
+ break;
default:
throw new SemanticException("Unrecognized token in CREATE DATABASE
statement");
}
}
- CreateDatabaseDesc desc = new CreateDatabaseDesc(databaseName, comment,
locationUri, managedLocationUri,
- ifNotExists, props);
- rootTasks.add(TaskFactory.get(new DDLWork(getInputs(), getOutputs(),
desc)));
-
+ CreateDatabaseDesc desc = null;
Database database = new Database(databaseName, comment, locationUri,
props);
- if (managedLocationUri != null) {
- database.setManagedLocationUri(managedLocationUri);
+ if (type.equalsIgnoreCase("NATIVE")) {
+ desc = new CreateDatabaseDesc(databaseName, comment, locationUri,
managedLocationUri, ifNotExists, props);
+ database.setType(DatabaseType.NATIVE);
+ // database = new Database(databaseName, comment, locationUri, props);
+ if (managedLocationUri != null) {
+ database.setManagedLocationUri(managedLocationUri);
+ }
+ } else {
+ String remoteDbName = databaseName;
+ if (props != null && props.get("connector.remoteDbName") != null) //
TODO finalize the property name
+ remoteDbName = props.get("connector.remoteDbName");
+ desc = new CreateDatabaseDesc(databaseName, comment, locationUri, null,
ifNotExists, props, type,
Review comment:
When you create a table using JDBCStorageHandler pointing to the remote
MySQL server, that table's metadata resides in HMS. Everything is local and
there is never a need for HMS to connect to remote SQL instance. This table
still belongs to a NATIVE database that can contain local tables as well. So
the paths are very much relevant for NATIVE databases. When a query is run
against this table, then HS2 (not HMS) uses the info on the table to connect to
remote MySQL server for table data.
For remote Databases, only the database metadata is local in HMS and the
table info is built at runtime. Hope that helps.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 572664)
Time Spent: 9h (was: 8h 50m)
> [New Feature] Add data connector support for remote datasources
> ---------------------------------------------------------------
>
> Key: HIVE-24396
> URL: https://issues.apache.org/jira/browse/HIVE-24396
> Project: Hive
> Issue Type: Improvement
> Components: Hive
> Reporter: Naveen Gangam
> Assignee: Naveen Gangam
> Priority: Major
> Labels: pull-request-available
> Time Spent: 9h
> Remaining Estimate: 0h
>
> This feature work is to be able to support in Hive Metastore to be able to
> configure data connectors for remote datasources and map databases. We
> currently have support for remote tables via StorageHandlers like
> JDBCStorageHandler and HBaseStorageHandler.
> Data connectors are a natural extension to this where we can map an entire
> database or catalogs instead of individual tables. The tables within are
> automagically mapped at runtime. The metadata for these tables are not
> persisted in Hive. They are always mapped and built at runtime.
> With this feature, we introduce a concept of type for Databases in Hive.
> NATIVE vs REMOTE. All current databases are NATIVE. To create a REMOTE
> database, the following syntax is to be used
> CREATE REMOTE DATABASE remote_db USING <dataconnector> WITH DCPROPERTIES
> (....);
> Will attach a design doc to this jira.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
