[
https://issues.apache.org/jira/browse/HIVE-24396?focusedWorklogId=576202&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-576202
]
ASF GitHub Bot logged work on HIVE-24396:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 02/Apr/21 17:34
Start Date: 02/Apr/21 17:34
Worklog Time Spent: 10m
Work Description: nrg4878 commented on a change in pull request #2037:
URL: https://github.com/apache/hive/pull/2037#discussion_r606340774
##########
File path:
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/dataconnector/DataConnectorProviderFactory.java
##########
@@ -0,0 +1,101 @@
+package org.apache.hadoop.hive.metastore.dataconnector;
+
+import org.apache.hadoop.hive.metastore.IHMSHandler;
+import org.apache.hadoop.hive.metastore.api.DataConnector;
+import org.apache.hadoop.hive.metastore.api.Database;
+import org.apache.hadoop.hive.metastore.api.DatabaseType;
+import org.apache.hadoop.hive.metastore.api.MetaException;
+import org.apache.hadoop.hive.metastore.api.NoSuchObjectException;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.util.HashMap;
+import java.util.Map;
+
+import static
org.apache.hadoop.hive.metastore.dataconnector.IDataConnectorProvider.*;
+
+public class DataConnectorProviderFactory {
+ Logger LOG = LoggerFactory.getLogger(DataConnectorProviderFactory.class);
+
+ private static Map<String, IDataConnectorProvider> cache = null;
+ private static DataConnectorProviderFactory singleton = null;
+ private static IHMSHandler hmsHandler = null;
+
+ private DataConnectorProviderFactory(IHMSHandler hmsHandler) {
+ cache = new HashMap<String, IDataConnectorProvider>();
+ this.hmsHandler = hmsHandler;
+ }
+
+ public static synchronized DataConnectorProviderFactory
getInstance(IHMSHandler hmsHandler) {
+ if (singleton == null) {
+ singleton = new DataConnectorProviderFactory(hmsHandler);
+ }
+ return singleton;
+ }
+
+ public static synchronized IDataConnectorProvider
getDataConnectorProvider(Database db) throws MetaException {
+ IDataConnectorProvider provider = null;
+ DataConnector connector = null;
+ if (db.getType() == DatabaseType.NATIVE) {
+ throw new MetaException("Database " + db.getName() + " is of type
NATIVE, no connector available");
+ }
+
+ String scopedDb = (db.getRemote_dbname() != null) ? db.getRemote_dbname()
: db.getName();
+ if (cache.containsKey(db.getConnector_name().toLowerCase())) {
+ provider = cache.get(db.getConnector_name().toLowerCase());
+ if (provider != null) {
+ provider.setScope(scopedDb);
+ }
+ return provider;
+ }
+
+ try {
+ connector = hmsHandler.get_dataconnector_core(db.getConnector_name());
+ } catch (NoSuchObjectException notexists) {
+ throw new MetaException("Data connector " + db.getConnector_name() + "
associated with database "
+ + db.getName() + " does not exist");
+ }
+ String type = connector.getType();
+ switch (type) {
+ case DERBY_TYPE:
+ case MSSQL_TYPE:
+ case MYSQL_TYPE:
+ case ORACLE_TYPE:
+ case POSTGRES_TYPE:
+ try {
+ provider = JDBCConnectorProviderFactory.get(scopedDb, connector);
+ } catch (Exception e) {
+ throw new MetaException("Could not instantiate a provider for database
" + db.getName());
+ }
+ break;
+ default:
+ throw new MetaException("Data connector of type " + connector.getType()
+ " not implemented yet");
+ }
+ cache.put(connector.getName().toLowerCase(), provider);
+ return provider;
+ }
+
+ IDataConnectorProvider getDataConnectorProvider(String connectorName) {
+ if (connectorName == null || connectorName.isEmpty()) {
+ return null;
+ }
+
+ if (cache.containsKey(connectorName.toLowerCase() != null)) {
+ return cache.get(connectorName.toLowerCase());
+ }
+ return null;
+ // return getDataConnectorProvider();
Review comment:
this method is currently not used at all, but removing it makes sense
because without the Database object, the returned provided will not have a
scope. So I think we can delete this method entirely.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
-------------------
Worklog Id: (was: 576202)
Time Spent: 9h 50m (was: 9h 40m)
> [New Feature] Add data connector support for remote datasources
> ---------------------------------------------------------------
>
> Key: HIVE-24396
> URL: https://issues.apache.org/jira/browse/HIVE-24396
> Project: Hive
> Issue Type: Improvement
> Components: Hive
> Reporter: Naveen Gangam
> Assignee: Naveen Gangam
> Priority: Major
> Labels: pull-request-available
> Time Spent: 9h 50m
> Remaining Estimate: 0h
>
> This feature work is to be able to support in Hive Metastore to be able to
> configure data connectors for remote datasources and map databases. We
> currently have support for remote tables via StorageHandlers like
> JDBCStorageHandler and HBaseStorageHandler.
> Data connectors are a natural extension to this where we can map an entire
> database or catalogs instead of individual tables. The tables within are
> automagically mapped at runtime. The metadata for these tables are not
> persisted in Hive. They are always mapped and built at runtime.
> With this feature, we introduce a concept of type for Databases in Hive.
> NATIVE vs REMOTE. All current databases are NATIVE. To create a REMOTE
> database, the following syntax is to be used
> CREATE REMOTE DATABASE remote_db USING <dataconnector> WITH DCPROPERTIES
> (....);
> Will attach a design doc to this jira.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
