nsivabalan commented on code in PR #7951:
URL: https://github.com/apache/hudi/pull/7951#discussion_r1120583549
##########
hudi-spark-datasource/hudi-spark/src/test/scala/org/apache/hudi/functional/TestCOWDataSource.scala:
##########
@@ -643,8 +736,9 @@ class TestCOWDataSource extends HoodieSparkClientTestBase
with ScalaAssertionSup
def testSparkPartitionByWithCustomKeyGenerator(recordType:
HoodieRecordType): Unit = {
val (writeOpts, readOpts) = getWriterReaderOpts(recordType)
+ val updatedWriteOpts =
writeOpts.-(KeyGeneratorOptions.PARTITIONPATH_FIELD_NAME.key())
Review Comment:
in all tests where we are testing df.partitionBy, we don't want to override
partition field and hence. bcoz, the main purpose of these test is, hudi should
infer df.partitionBy columns if partition path config is not explicitly
overriden by the user. getWriterReaderOpts() unfortunately already has
partition path set and hence for these tests I had to remove them. I can add a
comment here explaining why we are doing this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
