xushiyan commented on code in PR #7951:
URL: https://github.com/apache/hudi/pull/7951#discussion_r1120838073
##########
hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/DataSourceOptions.scala:
##########
@@ -295,30 +295,35 @@ object DataSourceWriteOptions {
def translateSqlOptions(optParams: Map[String, String]): Map[String, String]
= {
var translatedOptParams = optParams
// translate the api partitionBy of spark DataFrameWriter to
PARTITIONPATH_FIELD
- if (optParams.contains(SparkDataSourceUtils.PARTITIONING_COLUMNS_KEY)) {
+ // we should set hoodie's partition path only if its not set by the user.
+ if (optParams.contains(SparkDataSourceUtils.PARTITIONING_COLUMNS_KEY)
Review Comment:
the reasoning is: if people use partitionBy() and set
PARTITIONPATH_FIELD_NAME right now, they are likely to be matched. so now we
make PARTITIONPATH_FIELD_NAME higher precedence, it'll be compatible.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
