rangareddy commented on issue #16408: URL: https://github.com/apache/hudi/issues/16408#issuecomment-5056575188
This issue was reviewed as part of the JIRA-migrated backlog triage. Confirmed reproducible on current master (this is NOT fixed; JIRA HUDI-7461 is still Open). Root cause: CDC before/after JSON images are built by `InternalRowToJsonStringConverter` (hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/cdc/InternalRowToJsonStringConverter.scala). Its `convertField` handles `String`/`Array`/`Map`/`Struct` explicitly but routes all other types — including `DateType` and `TimestampType` — through the `case _ => value` fallthrough. That returns the raw `InternalRow` storage (epoch-days `Int` for date, epoch-micros `Long` for timestamp) instead of a formatted value, so date/timestamp columns appear as integers in the CDC read output. There is no date/timestamp coverage in `TestInternalRowToJsonStringConverter`. Fix scope is a contained change in that converter (add `DateType`/`TimestampType` cases that render proper values) plus regression tests. Tracking as `stream:fix` / incr-processing (read path). Keeping open. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
