rangareddy commented on issue #17224: URL: https://github.com/apache/hudi/issues/17224#issuecomment-5366110461
This issue was reviewed as part of the JIRA-migrated backlog triage (HUDI-4430). **Findings: needs a reproduction against a current build before it can be dispositioned.** The report is concrete: reading a table written with CustomKeyGenerator and a `create_time_epoch_seconds:timestamp` partition field casts the partition value incorrectly - the problem is specific to CustomKeyGenerator rather than TimestampBasedKeyGenerator or TimestampBasedAvroKeyGenerator. Reproducing it looks tractable - the description lists the exact write configs, so the table can be recreated and read back directly. I have not called this from the source. The surrounding code has been rewritten substantially since this was filed - table versions 8 and 9, the `BufferedRecordMerger` family replacing payload-based merging, the file-group-reader migration, the indexer abstraction in #18348 / #18372, and the `hudi-common` package moves in #19193 / #19195. Inferring either "still broken" or "silently fixed" through that much churn would be a guess. There is also no commit anywhere in the tree referencing HUDI-4430, so if it was fixed it was incidental rather than deliberate - another reason to verify rather than assume. Worth triaging together with #17247 (HUDI-7306), which reports `CustomKeyGenerator` dropping the microseconds part of a TIMESTAMP partition value. Both isolate the fault to `CustomKeyGenerator` specifically while the dedicated timestamp keygens behave correctly, which suggests `CustomKeyGenerator`'s delegation to the timestamp keygen loses type information rather than the timestamp logic being wrong. Keeping this open. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
