nsivabalan commented on code in PR #5424:
URL: https://github.com/apache/hudi/pull/5424#discussion_r857784124
##########
hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieSparkSqlWriter.scala:
##########
@@ -137,6 +137,9 @@ object HoodieSparkSqlWriter {
val partitionColumns =
HoodieSparkUtils.getPartitionColumns(keyGenerator, toProperties(parameters))
// Create the table if not present
if (!tableExists) {
+ val preCombineField = hoodieConfig.getString(PRECOMBINE_FIELD)
Review Comment:
will remove this line. just a note to reviewer that here the value is
already deduced to "ts" if not explicitly set by the user.
##########
hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieSparkSqlWriter.scala:
##########
@@ -151,7 +154,7 @@ object HoodieSparkSqlWriter {
.setBaseFileFormat(baseFileFormat)
.setArchiveLogFolder(archiveLogFolder)
.setPayloadClassName(hoodieConfig.getString(PAYLOAD_CLASS_NAME))
-
.setPreCombineField(hoodieConfig.getStringOrDefault(PRECOMBINE_FIELD, null))
+ .setPreCombineField(optParams.getOrElse(PRECOMBINE_FIELD.key(),
null))
Review Comment:
if not for this fix, tests written in TestMORDatasource fails
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
