danny0405 commented on code in PR #19794:
URL: https://github.com/apache/hudi/pull/19794#discussion_r3891625985
##########
hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieCLIUtils.scala:
##########
@@ -60,13 +59,26 @@ object HoodieCLIUtils extends Logging {
}
// Priority: defaults < catalog props < table config < sparkSession conf <
specified conf
- val finalParameters = HoodieWriterUtils.parametersWithWriteDefaults(
+ val parameters = HoodieWriterUtils.parametersWithWriteDefaults(
(catalogProps ++
metaClient.getTableConfig.getProps.asScala.toMap ++
filterHoodieConfigs(sparkSession.sqlContext.getAllConfs) ++
conf).toMap
)
+ // Auto-config a DFS-based lock for the metadata table when the table has
an MDT and no lock
+ // provider is configured at any layer. Table-service writers created here
(compaction,
+ // clustering, clean, TTL, restore, rollback, savepoint, ...) can update
the metadata table, so
+ // they must be mutually exclusive with concurrent writers on the shared
lock path. This must be
+ // applied before building the client so the lock config actually takes
effect.
+ val finalParameters =
Review Comment:
do we need lock for all the procedures, if not, keeping it in separate
procedure looks fine.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
