SEPURI-SAI-KRISHNA commented on code in PR #19648:
URL: https://github.com/apache/hudi/pull/19648#discussion_r3815133707
##########
hudi-spark-datasource/hudi-spark/src/test/scala/org/apache/spark/sql/hudi/common/TestSlashSeparatedPartitionValue.scala:
##########
@@ -90,6 +90,61 @@ class TestSlashSeparatedPartitionValue extends
HoodieSparkSqlTestBase {
}
}
+ test("Test slash separated date partitions written through the row writer") {
+ withSQLConf("hoodie.sql.bulk.insert.enable" -> "true",
"hoodie.sql.insert.mode" -> "non-strict",
+ "hoodie.datasource.write.row.writer.enable" -> "true") {
+ withTempDir { tmp =>
+ val targetTable = generateTableName
+ val tablePath = s"${tmp.getCanonicalPath}/$targetTable"
+
+ spark.sql(
+ s"""
+ |create table $targetTable (
+ | `id` string,
+ | `name` string,
+ | `ts` bigint,
+ | `datestr` STRING
Review Comment:
DATE case added, parameterized over insert and bulk_insert, asserting both
the directory on disk and that datestr reads back as date'2026-01-05'.
Worth noting why it needed the rebase: on the previous base the DATE case
failed at read time with
ClassCastException: UTF8String cannot be cast to Integer
at ColumnVectorUtils.populate / VectorizedParquetRecordReader.initBatch
because castStringToType returned UTF8String for DateType while Catalyst
wants an Int. That is exactly what #19652 fixed, and this branch predated it.
Rebased onto master, the case passes -- so it now guards the write-side
formatting from this PR and the read-side typing from #19652 together.
For TIMESTAMP I have filed a follow-up rather than widening this PR: the
slash arm in SqlKeyGenerator sits ahead of the TimestampType normalization and
shadows it, so the Avro path writes raw micros while the row writer writes the
normalized rendering. Filed as #19667.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
