codope commented on a change in pull request #3722:
URL: https://github.com/apache/hudi/pull/3722#discussion_r725986918
##########
File path:
hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/DataSourceOptions.scala
##########
@@ -381,6 +381,11 @@ object DataSourceWriteOptions {
val HIVE_URL: ConfigProperty[String] = ConfigProperty
.key("hoodie.datasource.hive_sync.jdbcurl")
.defaultValue("jdbc:hive2://localhost:10000")
+ .withDocumentation("Hive jdbc url")
+
+ val METASTOR_URIS: ConfigProperty[String] = ConfigProperty
Review comment:
Minor: rename to `METASTORE_URI`
##########
File path:
hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/hudi/HoodieSparkSqlWriter.scala
##########
@@ -517,6 +517,9 @@ object HoodieSparkSqlWriter {
val hiveSyncConfig: HiveSyncConfig = buildSyncConfig(basePath,
hoodieConfig, sqlConf)
val hiveConf: HiveConf = new HiveConf()
hiveConf.addResource(fs.getConf)
+
if(!DataSourceWriteOptions.METASTOR_URIS.defaultValue.equals(hiveSyncConfig.metastoreUris)){
Review comment:
Can you please add a test case in the writer test suite or elsewhere to
cover this scenario?
##########
File path:
hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/DataSourceOptions.scala
##########
@@ -381,6 +381,11 @@ object DataSourceWriteOptions {
val HIVE_URL: ConfigProperty[String] = ConfigProperty
.key("hoodie.datasource.hive_sync.jdbcurl")
.defaultValue("jdbc:hive2://localhost:10000")
+ .withDocumentation("Hive jdbc url")
+
+ val METASTOR_URIS: ConfigProperty[String] = ConfigProperty
+ .key("hoodie.datasource.hive_sync.metastore.uris")
Review comment:
Perhaps we need to set this config in
`DataSourceUtils#buildHiveSyncConfig`. Please check.
##########
File path:
hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/DataSourceOptions.scala
##########
@@ -381,6 +381,11 @@ object DataSourceWriteOptions {
val HIVE_URL: ConfigProperty[String] = ConfigProperty
.key("hoodie.datasource.hive_sync.jdbcurl")
.defaultValue("jdbc:hive2://localhost:10000")
+ .withDocumentation("Hive jdbc url")
+
+ val METASTOR_URIS: ConfigProperty[String] = ConfigProperty
+ .key("hoodie.datasource.hive_sync.metastore.uris")
Review comment:
Let's have consistency with DeltaStreamer. There should be a way to set
this config with deltastreamer as well.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
