FranMorilloAWS commented on code in PR #191:
URL:
https://github.com/apache/flink-connector-aws/pull/191#discussion_r2034972205
##########
flink-catalog-aws-glue/src/main/java/com/amazonaws/services/msf/util/ConnectorRegistry.java:
##########
@@ -0,0 +1,53 @@
+package com.amazonaws.services.msf.util;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.util.HashMap;
+import java.util.Map;
+
+/**
+ * This class is responsible for storing and retrieving location-specific keys
for different connectors.
+ * It maps connector types to their corresponding location keys (e.g.,
Kinesis, Kafka).
+ */
+public class ConnectorRegistry {
+
+ /** Logger for logging connector registry actions. */
+ private static final Logger LOG =
LoggerFactory.getLogger(ConnectorRegistry.class);
+
+ /** Map to store connector types and their corresponding location-specific
keys. */
+ private static final Map<String, String> connectorLocationKeys = new
HashMap<>();
+
+ // Static block to initialize the connector keys mapping.
+ static {
+ connectorLocationKeys.put("kinesis", "stream.arn");
+ connectorLocationKeys.put("kafka", "properties.bootstrap.servers");
+ connectorLocationKeys.put("jdbc", "url");
+ connectorLocationKeys.put("filesystem", "path");
+ connectorLocationKeys.put("elasticsearch", "hosts");
+ connectorLocationKeys.put("opensearch", "hosts");
+ connectorLocationKeys.put("hbase", "zookeeper.quorum");
+ connectorLocationKeys.put("dynamodb", "table.name");
+ connectorLocationKeys.put("mongodb", "uri");
+ connectorLocationKeys.put("hive", "hive-conf-dir");
Review Comment:
Can you elaborate more? So here what we are doing is that in Glue Catalog
we can have a Location property that usually it would point to an S3 Table, so
What we are doing is based on the available Flink SQL Connectors, based on the
connector they used when creating the table we extract from the connectors
properties what would be its appropriate location, such as for elasticsearch
connector, its location would be the hosts (endpoint) or for the kinesis
connector we would get the stream.arn. Adding qualifier prefix wouldnt work as
we are using the connector type qualifier and their exact config name from
properties
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
