xushiyan commented on a change in pull request #4438:
URL: https://github.com/apache/hudi/pull/4438#discussion_r774815485
##########
File path:
hudi-spark-datasource/hudi-spark-common/src/main/java/org/apache/hudi/DataSourceUtils.java
##########
@@ -309,6 +309,8 @@ public static HiveSyncConfig
buildHiveSyncConfig(TypedProperties props, String b
DataSourceWriteOptions.HIVE_SKIP_RO_SUFFIX_FOR_READ_OPTIMIZED_TABLE().defaultValue()));
hiveSyncConfig.supportTimestamp =
Boolean.valueOf(props.getString(DataSourceWriteOptions.HIVE_SUPPORT_TIMESTAMP_TYPE().key(),
DataSourceWriteOptions.HIVE_SUPPORT_TIMESTAMP_TYPE().defaultValue()));
+ hiveSyncConfig.isConditionalSync =
Boolean.valueOf(props.getString(DataSourceWriteOptions.HIVE_CONDITIONAL_SYNC().key(),
+ DataSourceWriteOptions.HIVE_CONDITIONAL_SYNC().defaultValue()));
Review comment:
irrelevant change.. moving to separate PR
##########
File path:
hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/DataSourceOptions.scala
##########
@@ -495,6 +495,16 @@ object DataSourceWriteOptions {
.withDocumentation("‘INT64’ with original type TIMESTAMP_MICROS is
converted to hive ‘timestamp’ type. " +
"Disabled by default for backward compatibility.")
+ /**
+ * Flag to indicate whether to use conditional syncing in HiveSync.
+ * If set true, the Hive sync procedure will only run if partition or schema
changes are detected.
+ * By default true.
+ */
+ val HIVE_CONDITIONAL_SYNC: ConfigProperty[String] = ConfigProperty
+ .key("hoodie.datasource.hive_sync.conditional_sync")
+ .defaultValue("false")
+ .withDocumentation("Enables conditional hive sync, where partition or
schema change must exist to perform sync to hive.")
Review comment:
irrelevant change.. moving to separate PR
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
