XuQianJin-Stars commented on a change in pull request #4291:
URL: https://github.com/apache/hudi/pull/4291#discussion_r836131643
##########
File path:
hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/spark/sql/hudi/command/AlterHoodieTableDropPartitionCommand.scala
##########
@@ -82,16 +84,26 @@ extends RunnableCommand {
}.mkString("/")
}.mkString(",")
+ val enableHive = isEnableHive(sparkSession)
withSparkConf(sparkSession, Map.empty) {
Map(
"path" -> hoodieCatalogTable.tableLocation,
TBL_NAME.key -> hoodieCatalogTable.tableName,
TABLE_TYPE.key -> hoodieCatalogTable.tableTypeName,
OPERATION.key ->
DataSourceWriteOptions.DELETE_PARTITION_OPERATION_OPT_VAL,
- PARTITIONS_TO_DELETE.key -> partitionsToDelete,
+ PARTITIONS_TO_DELETE.key -> partitionsToDrop,
RECORDKEY_FIELD.key -> hoodieCatalogTable.primaryKeys.mkString(","),
PRECOMBINE_FIELD.key -> hoodieCatalogTable.preCombineKey.getOrElse(""),
- PARTITIONPATH_FIELD.key ->
hoodieCatalogTable.partitionFields.mkString(",")
+ PARTITIONPATH_FIELD.key -> partitionFields,
+ HIVE_SYNC_ENABLED.key -> enableHive.toString,
+ META_SYNC_ENABLED.key -> enableHive.toString,
+ HIVE_SYNC_MODE.key -> HiveSyncMode.HMS.name(),
+ HIVE_USE_JDBC.key -> "false",
+ HIVE_DATABASE.key ->
hoodieCatalogTable.table.identifier.database.getOrElse("default"),
+ HIVE_TABLE.key -> hoodieCatalogTable.table.identifier.table,
+ HIVE_SUPPORT_TIMESTAMP_TYPE.key -> "true",
+ HIVE_PARTITION_FIELDS.key -> partitionFields,
+ HIVE_PARTITION_EXTRACTOR_CLASS.key ->
classOf[MultiPartKeysValueExtractor].getCanonicalName
Review comment:
> is this hardcoding of configs okay? is the user still able to override
these with properties supplied at the SQL statement itself?
Regarding this configuration, I need a pr to handle it uniformly. As you
said, if spark sql has hive support enabled, you can replace this information
from the sql configuration without hard configuration.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
