Harsha Teja Kanna created HUDI-2909:
---------------------------------------
Summary: KeyGenerator is broken in 0.10.0
Key: HUDI-2909
URL: https://issues.apache.org/jira/browse/HUDI-2909
Project: Apache Hudi
Issue Type: Bug
Components: DeltaStreamer
Reporter: Harsha Teja Kanna
Fix For: 0.10.0
Existing table has timebased keygen config show below
hoodie.deltastreamer.keygen.timebased.timestamp.type=SCALAR
hoodie.deltastreamer.keygen.timebased.output.timezone=GMT
hoodie.deltastreamer.keygen.timebased.output.dateformat=yyyy/MM/dd
hoodie.deltastreamer.keygen.timebased.timestamp.scalar.time.unit=MICROSECONDS
hoodie.deltastreamer.keygen.timebased.input.timezone=GMT
hoodie.datasource.write.partitionpath.field=lastdate:timestamp
hoodie.datasource.write.operation=upsert
hoodie.deltastreamer.transformer.sql=SELECT session.id, session.rid,
session.mid, to_timestamp(session.lastdate) as lastdate,
to_timestamp(session.updatedate) as updatedate FROM <SRC> a
Upgrading to 0.10.0 from 0.9.0 fails with exception
org.apache.hudi.exception.HoodieKeyGeneratorException: Unable to parse input
partition field :2021-12-01 10:13:34.702
Caused by: org.apache.hudi.exception.HoodieNotSupportedException: Unexpected
type for partition field: java.sql.Timestamp
at
org.apache.hudi.keygen.TimestampBasedAvroKeyGenerator.getPartitionPath(TimestampBasedAvroKeyGenerator.java:211)
at
org.apache.hudi.keygen.TimestampBasedAvroKeyGenerator.getPartitionPath(TimestampBasedAvroKeyGenerator.java:133)
*Workaround fix:*
Reverting this
https://github.com/apache/hudi/pull/3944/files#diff-22fb52b5cf28727ba23cb8bd4be820432a4e396ce663ac472a4677e889b7491eR543
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
