[
https://issues.apache.org/jira/browse/HUDI-1848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17385730#comment-17385730
]
ASF GitHub Bot commented on HUDI-1848:
--------------------------------------
satishkotha commented on a change in pull request #2879:
URL: https://github.com/apache/hudi/pull/2879#discussion_r675097053
##########
File path:
hudi-sync/hudi-hive-sync/src/main/java/org/apache/hudi/hive/util/HiveSchemaUtil.java
##########
@@ -143,38 +145,73 @@ private static boolean isFieldExistsInSchema(Map<String,
String> newTableSchema,
* @param messageType : Parquet Schema
* @return : Hive Table schema read from parquet file MAP[String,String]
*/
- private static Map<String, String>
convertParquetSchemaToHiveSchema(MessageType messageType, boolean
supportTimestamp) throws IOException {
- Map<String, String> schema = new LinkedHashMap<>();
+ public static Map<String, String>
convertParquetSchemaToHiveSchema(MessageType messageType, boolean
supportTimestamp) throws IOException {
+ return convertMapSchemaToHiveSchema(parquetSchemaToMapSchema(messageType,
supportTimestamp, true));
+ }
+
+ /**
+ * Returns equivalent Hive table Field schema read from a parquet file.
+ *
+ * @param messageType : Parquet Schema
+ * @return : Hive Table schema read from parquet file List[FieldSchema]
without partitionField
+ */
+ public static List<FieldSchema>
convertParquetSchemaToHiveFieldSchema(MessageType messageType, HiveSyncConfig
syncConfig) throws IOException {
+ return
convertMapSchemaToHiveFieldSchema(parquetSchemaToMapSchema(messageType,
syncConfig.supportTimestamp, false), syncConfig);
Review comment:
@jsbali why is doFormat false here? Can we use doFormat true everywhere
and simplify this code? Or does this break backward compatibiity for some sync
executors?
if we cannot make code change for backward compatibility, could you add
comment why doFormat is false here?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
> Add support for HMS in Hive-sync-tool
> -------------------------------------
>
> Key: HUDI-1848
> URL: https://issues.apache.org/jira/browse/HUDI-1848
> Project: Apache Hudi
> Issue Type: New Feature
> Reporter: Jagmeet Bali
> Priority: Minor
> Labels: pull-request-available, sev:normal
>
> Add support for HMS in Hive-sync-tool
> Currently there are two ways to sun DDL queries in hive-sync-tool.
> This work adds on top of
> [https://github.com/apache/hudi/pull/2532|https://github.com/apache/hudi/pull/2532/files]
> and adds a pluggable way to support
> new way to run DDL queries using HMS.
>
> Different DDL executors can be selected via diff syncConfig options
> useJDBC true -> JDBCExecutor will be used
> useJDBC false -> QlHiveQueryExecutor will be used
> useHMS true -> HMSDDLExecutor will be used.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
