[
https://issues.apache.org/jira/browse/HUDI-1848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17372944#comment-17372944
]
ASF GitHub Bot commented on HUDI-1848:
--------------------------------------
jsbali commented on a change in pull request #2879:
URL: https://github.com/apache/hudi/pull/2879#discussion_r662461892
##########
File path:
hudi-sync/hudi-hive-sync/src/main/java/org/apache/hudi/hive/util/HiveSchemaUtil.java
##########
@@ -150,11 +172,26 @@ private static boolean isFieldExistsInSchema(Map<String,
String> newTableSchema,
result.append(convertField(parquetType, supportTimestamp));
}
- schema.put(hiveCompatibleFieldName(key, false), result.toString());
+ schema.put(key, result.toString());
}
return schema;
}
+ public static Map<String, String>
convertMapSchemaToHiveSchema(LinkedHashMap<String, String> schema) throws
IOException {
+ Map<String, String> hiveSchema = new LinkedHashMap<>();
+ for (Map.Entry<String,String> entry: schema.entrySet()) {
+ hiveSchema.put(hiveCompatibleFieldName(entry.getKey(), false),
entry.getValue());
+ }
+ return hiveSchema;
+ }
+
+ public static List<FieldSchema>
convertMapSchemaToHiveFieldSchema(LinkedHashMap<String, String> schema,
HiveSyncConfig syncConfig) throws IOException {
+ return schema.keySet().stream()
+ .map(key -> new FieldSchema(key, schema.get(key), ""))
+ .filter(field -> !syncConfig.partitionFields.contains(field.getName()))
Review comment:
This is only for the HMS api as it expects cols and partCols to be
provided separately.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
> Add support for HMS in Hive-sync-tool
> -------------------------------------
>
> Key: HUDI-1848
> URL: https://issues.apache.org/jira/browse/HUDI-1848
> Project: Apache Hudi
> Issue Type: New Feature
> Reporter: Jagmeet Bali
> Priority: Minor
> Labels: pull-request-available, sev:normal
>
> Add support for HMS in Hive-sync-tool
> Currently there are two ways to sun DDL queries in hive-sync-tool.
> This work adds on top of
> [https://github.com/apache/hudi/pull/2532|https://github.com/apache/hudi/pull/2532/files]
> and adds a pluggable way to support
> new way to run DDL queries using HMS.
>
> Different DDL executors can be selected via diff syncConfig options
> useJDBC true -> JDBCExecutor will be used
> useJDBC false -> QlHiveQueryExecutor will be used
> useHMS true -> HMSDDLExecutor will be used.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
